23 replies to this topic

[kenkino]

OH NO, CANT LOG IN! >

[Miii]

OH NOES...

[iiNote]

xD
I'm getting the same thing.. LOL

[Luckywhiterabbit]

oh good, so it's not just me.

...what population?

[Alicia]

The bots.

[kenkino]

YAY THERE'S PPL REPLYING TO MY 1ST POST! WOOOOOOOO LMFAO

[Cleffy]

10

lol Korean Gravity having same issue.

[Alicia]

Just the bare minimum :

Quadruple Exp Event Tripple All Spawns No Exp Upon Death Double Drops Bonus Kafra Points Discount Kafra Shop Items.

[Kareemah]

i can't log in either =(

[iiNote]

Now I can...
Waiting for the next disconnect... and counting...

[Yuske]

Yea free ocas too, free zeny and make all monsters have 1 HP So we level faster!!

Edited by Yuske, 30 November 2010 - 04:53 PM.

[Xenyph]

If we put this metaphor to life, it's like saying; let us live faster, so we can die already. (no emo) Thou must learn that the journey towards the end is more important then the end.

[LazyStevz]

If we put this metaphor to life, it's like saying; let us live faster, so we can die already. (no emo) Thou must learn that the journey towards the end is more important then the end.

You, my good sir/madame, win Internets.

As much as compensation is wanted I think we should cut the iRO team some slack. It's not been easy for anyone ATM.

[TheMarX]

If we put this metaphor to life, it's like saying; let us live faster, so we can die already. (no emo) Thou must learn that the journey towards the end is more important then the end.

"Like"

[Heimdallr]

Many comments and complaints have been heard regarding the latency and disconnections that have been occuring recently. We have had several seperate incidents that have accumulated together creating an overall bad game play experience in the past 7 weeks.

I want to talk about what has happened in the past week. We have been making numerous optimizations over the past months to find sources of lag and eliminate it. Things were going as well as expected with such a long term project, when our normally stable ISP began having its own problems. We had several downed routers throughout the internet (not all of them from our ISP) when that happens the network traffic redirects to less optimal routes that creates lag for the users. This is normal, it sucks when it happens but it is not an odd occurance in the internet world.

And then things got real.. a constuction project severed the lines from our Data Center completely cutting WarpPortal off from the world for almost a full day. This was repaired, but things were never quite the same. We began analyzing all packets coming in with a monitoring tool that was on loan, and it was illuminating for where bottlenecks were on our back end network (game servers, routers and DBs). Many optimizations were made over the weeks and regular improvements were noticed, which is expected given the size of such a project. However, at that time those pesky routers were not behaving and many connections between ISPs were very poor routings which causes more latency than would otherwise be there. We could see these problems as well, and reported them to the ISP which then negotiated and worked with the responsible people for the routers.

Things were fairly well sorted out by Tuesday of last week, there was 1 bad router that was being looked into on NTT side, and then the DDOS started on Thursday which made our services unavailable. This DDOS continues through now. Early this morning we hoped to perform a repair and check session at our IDC, this requires help from the Data Center and ISP staff. It was cancelled due to a work scheduling problem. We are doing the appropriate steps to mitigate the DDOS, which is extra annoying because the attack vector is fluctuating, and not persistent. We have taken appropriate prevention and repair procedures to lessen the damage, but due to the way internet traffic must work it is not possible to "turtle up" and still get legit traffic to get through. Our ISP and IDC is working with us but a final solution that doesn't kill normal traffic only comes with time or when the DDOS person stops (or is stopped).

I do expect the the latency and DCs to continue at some level for the next day or so. I do know we will be providing some treats as an apology and to help get you where you should have been. We will weather the storm and come out ahead in the end.

[BlackPotato]

then up, then down, then up, then down, then... timegap... then down... this is rediculous...

[Alicia]

If we put this metaphor to life, it's like saying; let us live faster, so we can die already. (no emo) Thou must learn that the journey towards the end is more important then the end.

So I take it you have no sense of humor?

That's like taking your methaphor and shoving it up a Camel's ass, right?

[fearfulways]

i think i just died again, there goes 710k exp and hebm time

[SisterGoat]

QQ more.

[Alicia]

Yup got a whole 5 minutes in.

[fenryl]

they should really just take down the servers for as long as it takes until it's fixed (doenst matter what needs to be fixed)... This week has been the worst by far.

[Alicia]

i think i just died again, there goes 710k exp and hebm time

FFS STOP USING HEBMS

Edited by Alicia, 30 November 2010 - 05:26 PM.

[Heimdallr]

Many comments and complaints have been heard regarding the latency and disconnections that have been occuring recently. We have had several seperate incidents that have accumulated together creating an overall bad game play experience in the past 7 weeks.

I want to talk about what has happened in the past week. We have been making numerous optimizations over the past months to find sources of lag and eliminate it. Things were going as well as expected with such a long term project, when our normally stable ISP began having its own problems. We had several downed routers throughout the internet (not all of them from our ISP) when that happens the network traffic redirects to less optimal routes that creates lag for the users. This is normal, it sucks when it happens but it is not an odd occurance in the internet world.

And then things got real.. a constuction project severed the lines from our Data Center completely cutting WarpPortal off from the world for almost a full day. This was repaired, but things were never quite the same. We began analyzing all packets coming in with a monitoring tool that was on loan, and it was illuminating for where bottlenecks were on our back end network (game servers, routers and DBs). Many optimizations were made over the weeks and regular improvements were noticed, which is expected given the size of such a project. However, at that time those pesky routers were not behaving and many connections between ISPs were very poor routings which causes more latency than would otherwise be there. We could see these problems as well, and reported them to the ISP which then negotiated and worked with the responsible people for the routers.

Things were fairly well sorted out by Tuesday of last week, there was 1 bad router that was being looked into on NTT side, and then the DDOS started on Thursday which made our services unavailable. This DDOS continues through now. Early this morning we hoped to perform a repair and check session at our IDC, this requires help from the Data Center and ISP staff. It was cancelled due to a work scheduling problem. We are doing the appropriate steps to mitigate the DDOS, which is extra annoying because the attack vector is fluctuating, and not persistent. We have taken appropriate prevention and repair procedures to lessen the damage, but due to the way internet traffic must work it is not possible to "turtle up" and still get legit traffic to get through. Our ISP and IDC is working with us but a final solution that doesn't kill normal traffic only comes with time or when the DDOS person stops (or is stopped).

I do expect the the latency and DCs to continue at some level for the next day or so. I do know we will be providing some treats as an apology and to help get you where you should have been. We will weather the storm and come out ahead in the end.

Merging all lag stuff here.

[Freedom]

Thanks for the update heim. This is the reason why I have gave up on lvling for the past 2 days. No sence in logging in and trying to lvl when I know this lag is going on and I will die and loss exp. I hope you get this fixed soon