/10 chars......I really think to avoid confusion or ppl posting IP addresses they don't want others to see..this information should be sent to a moderator via Private mail.
0
vargr01Member, Neverwinter Beta Users, Neverwinter Hero UsersPosts: 0Arc User
edited June 2013
tracert patchserver.crypticstudios.com
Tracing route to patchserver.crypticstudios.com [208.95.185.41] over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms ___.___.____.___
2 23 ms 23 ms 23 ms 184.99.0.12
3 22 ms 23 ms 23 ms cosp-agw1.inet.qwest.net [184.99.1.89]
4 59 ms 59 ms 69 ms lap-brdr-03.inet.qwest.net [67.14.22.78]
5 58 ms 58 ms 59 ms 63-235-40-86.dia.static.qwest.net [63.235.40.86]
6 59 ms 60 ms 60 ms cr2-tengig-0-7-4-0.lay.savvis.net [204.70.198.9]
7 105 ms 105 ms 105 ms cr2-te-0-13-2-0.nyr.savvis.net [206.28.96.53]
8 105 ms 105 ms 105 ms er1-te-2-1.newyorknyd.savvis.net [204.70.196.70]
9 115 ms 115 ms 114 ms msr1-te-0-3-0-0.bos.savvis.net [206.28.97.205]
10 111 ms 112 ms 111 ms 206.28.97.137
11 109 ms 109 ms 108 ms internap.Boston.savvis.net [208.172.51.230]
12 110 ms 109 ms 109 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.104]
13 172 ms 162 ms 162 ms perfectworldent-4.border11.bsn.pnap.net [216.52.61.78]
14 163 ms 162 ms 163 ms 208.95.185.41
Trace complete.
~~~~~~~~~~~~
We grow not old so long as we continue to chase our dreams.
0
llewelyn89Member, NW M9 PlaytestPosts: 129Arc User
edited June 2013
Tracing route to patchserver.crypticstudios.com [208.95.185.41] over a maximum of 30 hops:
1 8 ms 9 ms 19 ms 10.83.0.1
2 9 ms 9 ms 7 ms bla1-ge0-0-3.gw.optusnet.com.au [198.142.192.169]
3 * * * Request timed out.
4 8 ms 11 ms 9 ms bla5-ge13-0.gw.optusnet.com.au [211.29.125.249]
5 173 ms 163 ms 159 ms 203.208.131.57
6 157 ms 160 ms 160 ms 203.208.153.46
7 158 ms 157 ms 161 ms be2020.ccr22.lax01.atlas.cogentco.com [154.54.88.17]
8 193 ms 195 ms 193 ms te0-2-0-3.mpd22.iah01.atlas.cogentco.com [154.54.45.9]
9 209 ms 211 ms 218 ms te0-2-0-2.ccr22.atl01.atlas.cogentco.com [154.54.42.214]
10 231 ms 220 ms 220 ms te0-0-0-3.mpd22.dca01.atlas.cogentco.com [154.54.28.209]
11 226 ms 227 ms 227 ms te0-3-0-3.mpd22.jfk02.atlas.cogentco.com [154.54.5.74]
12 232 ms 232 ms 232 ms te0-3-1-1.ccr22.bos01.atlas.cogentco.com [154.54.44.62]
13 234 ms 233 ms 233 ms te4-2.ccr01.bos06.atlas.cogentco.com [66.28.4.254]
14 239 ms 246 ms 237 ms 38.111.40.114
15 234 ms 231 ms 240 ms 208.95.185.41
I have constant spikes on every 3-7 seconds everywhere in NW!!! This makes the game unplayable for me... This problem started on June 2/2013 and did not get resolved so far ;( I really have no idea whats going on with your servers but I'm really frustrated now...
I log from Cyprus/EU, my ISP name is MTN
Here is my tracing route:
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1
2 20 ms 22 ms 22 ms 213.207.173.1
3 19 ms 19 ms 22 ms 213.207.129.28
4 22 ms 57 ms 23 ms 213.207.128.234
5 88 ms 90 ms 88 ms 41-208-29-154.uk.mtnns.net [41.208.29.154]
6 90 ms 88 ms 87 ms ls-pr-1.uk--ls-cr-2.uk-a.mtn.net [209.212.111.201]
7 89 ms 87 ms 87 ms TenGigabitEthernet8-2.ar6.LON3.gblx.net [208.48.238.185]
8 162 ms 161 ms 165 ms te0-7-0-21.ccr21.jfk07.atlas.cogentco.com [154.54.14.69]
9 166 ms 157 ms 170 ms te0-1-0-0.mpd22.jfk02.atlas.cogentco.com [154.54.1.221]
10 179 ms 179 ms 166 ms te0-6-0-5.ccr22.bos01.atlas.cogentco.com [154.54.44.26]
11 167 ms 181 ms 164 ms te4-2.ccr01.bos06.atlas.cogentco.com [66.28.4.254]
12 169 ms 177 ms 177 ms 38.111.40.114
13 186 ms 184 ms 184 ms 208.95.185.41
We have investigated this issue and it appears that some players are encountering an issue with a routing provider (Savvis) that exists between their ISP and our servers. We recommend waiting some time to see if the issue is resolved on their end, or alternatively, contacting your ISP to report the issue.
We will continue to monitor for additional issues and will investigate accordingly if new information is discovered. Thank you for letting us know about this!
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms
2 25 ms 25 ms 25 ms desm-dsl-gw26.desm.qwest.net
3 22 ms 23 ms 23 ms desm-agw1.inet.qwest.net [75.160.210.201]
4 38 ms 38 ms 39 ms chp-brdr-03.inet.qwest.net [67.14.8.190]
5 38 ms 38 ms 38 ms 208.174.226.229
6 38 ms 39 ms 39 ms cr1-tengig-0-5-0-0.chicago.savvis.net [204.70.19
6.21]
7 60 ms 67 ms 60 ms cr2-ten-0-13-0-0.nyr.savvis.net [204.70.200.77]
8 60 ms 61 ms 61 ms er1-te-2-1.newyorknyd.savvis.net [204.70.196.70]
9 68 ms 69 ms 69 ms msr1-te-0-3-0-0.bos.savvis.net [206.28.97.205]
10 65 ms 66 ms 66 ms 206.28.97.137
11 67 ms 67 ms 66 ms internap.Boston.savvis.net [208.172.51.230]
12 68 ms 67 ms 67 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.1
04]
13 130 ms 126 ms 125 ms perfectworldent-4.border11.bsn.pnap.net [216.52.
61.78]
14 118 ms 118 ms 118 ms 208.95.185.41
We have investigated this issue and it appears that some players are encountering an issue with a routing provider (Savvis) that exists between their ISP and our servers. We recommend waiting some time to see if the issue is resolved on their end, or alternatively, contacting your ISP to report the issue.
We will continue to monitor for additional issues and will investigate accordingly if new information is discovered. Thank you for letting us know about this!
I do not use the provider Savvis... what about my problem that persist for more then a week already?
the problem is between your ISP and Neverwinter. Third party.
When I look at my trace routes there is nothing wrong with them... and I play any other game perfectly fine! With latency below 200ms... when I play NW is always above 200ms and I get constant spikes...
0
cythrulMember, Neverwinter Beta UsersPosts: 0Arc User
edited June 2013
I am having the same issue.
Massive lag spikes every few seconds... some disconnects.
Started yesterday around 7PM PST.
My ISP is Centurylink.
0
cythrulMember, Neverwinter Beta UsersPosts: 0Arc User
edited June 2013
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.0.1
2 22 ms 23 ms 22 ms 67.40.227.130
3 23 ms 23 ms 23 ms phnx-agw1.inet.qwest.net [75.160.238.1]
4 34 ms 34 ms 35 ms lap-brdr-03.inet.qwest.net [67.14.22.78]
5 35 ms 35 ms 34 ms 63-235-40-86.dia.static.qwest.net [63.235.40.86]
6 35 ms 36 ms 36 ms cr1-tengig-0-5-0-0.lay.savvis.net [204.70.198.5]
7 103 ms 104 ms 104 ms cr2-bundle-pos-1.newyork.savvis.net [204.70.197.
34]
8 102 ms 110 ms 103 ms er1-te-2-1.newyorknyd.savvis.net [204.70.196.70]
9 110 ms 111 ms 110 ms msr1-te-0-3-0-0.bos.savvis.net [206.28.97.205]
10 108 ms 108 ms 109 ms 206.28.97.137
11 107 ms 106 ms 106 ms internap.Boston.savvis.net [208.172.51.230]
12 107 ms 106 ms 106 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.1
04]
13 163 ms 163 ms 175 ms perfectworldent-4.border11.bsn.pnap.net [216.52.
61.78]
14 160 ms 160 ms 161 ms 208.95.185.41
Put a check mark in front of Disable on-demand patching in the settings of the launcher, then it is something pumped and lag reduced. Maybe I should be so lucky, but it looks like a problem with the patch. I apologize for my English.
We have investigated this issue and it appears that some players are encountering an issue with a routing provider (Savvis) that exists between their ISP and our servers. We recommend waiting some time to see if the issue is resolved on their end, or alternatively, contacting your ISP to report the issue.
We will continue to monitor for additional issues and will investigate accordingly if new information is discovered. Thank you for letting us know about this!
I noticed this issue a couple days ago when it was really bad. I just came back later when the network wasn't as saturated and gameplay was somewhat better. The problem wasn't as visible on the PatchServer as it was on the PatchRedirectServer.
I go through Cogento, not Savvis, which indicated this probably affects multiple ISP's. The problem appears to exist in the Boston area. I have limited a view from here, but it looks like the problem possibly exists between the last hop leaving the tier 1 provider before going into the 38.111.40.114 network. The problem was definitely cyclic packet loss (please see attached picture), which is far worse than latency issues. Latency = slow or delayed connectivity. Packet loss = no connectivity for a specified time period. Packet loss usually occurs due to over-utilization of a particular network element. Unfortunately pushing networks to saturation is a standard ISP practice, and unless many complaints are filed AND ESCALATED, nothing will likely be done unless the culprit element pops up on someone's internal monitoring software. This issue might well be related to a measure of success in that there may be more client connectivity that you anticipated at this point, but it appears so far to only reveal itself on the "weakest" links. In other words, this problem will only affect specific people on the same ISP's that happen to have their routing through this particular over-utilized network element.
I investigated realtime and historically and there are currently no peering issues between any of the major US tier 1 provider handoffs. That's good news. But the bad news is I've dealt with problems like this before(on the corporate level) and they can be very difficult to resolve without putting someone on permanent escalation duty with Cryptic's provider of choice. I know you don't have control of that connectivity outside your network, but neither do the clients ISP's, and that's the biggest problem that this issue exists somewhere in the middle.
How to Help:
1) if you're a gamer that's experiencing these issues, it cant hurt to open a ticket with your ISP, but be prepared, they will have no idea what you are talking about, and will likely just tell you to reset your router. You will need to ask directly to escalate to at least tier II or tier III support, these are the folks that deal with ISP interconnectivity. You ISP will probably deny your escalation request unless you just get lucky. You can also ask them to provide you with a tier II or III email address, which they may, or may not provide. If you happen to get any Tier II or III contact, give them your traceroute to the cryptic server and tell them you are experiencing high packet loss on this connection...after that, its up to them. If they don't see the problem, they will close the ticket. If they see the problem, they will open a ticket with the middleman, and it might get addressed.
2) If you are Cryptic. Its a similar scenario in that your facility provider can only open a ticket with the suspect tier 1 ISP. However, unless you really put the pressure on your facility provider, they will likely not put the pressure on the tier 1 provider to find a suitable solution. Most likely they will claim that their connectivity is operating within agreed QOS terms, so it wont get the attention that it needs unless you continue to escalate, and escalate, and escalate the issue. Note on the attached picture the 13% packet loss...this level of packet loss definitely would not be in compliance with any QOS.
What to Do Short Term as a Gamer:
Unfortunately, if this problem persists, you only have 2 choices. Either play earlier or later when less people are online, or get a new ISP and hope it doesn't follow the same exact network route to Cryptic's servers. If you are on a laptop, you could try a friends house with a different ISP and see if the problem exists there. Also it might be helpful if people experiencing the problem state their ISP in the comments, then we can make a short list of culprit ISP's.
EDIT: The next time I see the problem getting bad, I'll locate the exact hop where the problem exists and post a screenshot of it. Right now its just mild latency spikes which probably aren't noticeable on the client side.
Contacted century link and after 2 hours of proving to them that savvis.net had an issue. They contacted Savvis. It seems there was a port issue and has been resolved.
I do not work for Savvis or Century Link. If it fails again, just contact century link and have them go through the steps yet again.
But since it appears to be resolved... I'll see you all in game.
Contacted century link and after 2 hours of proving to them that savvis.net had an issue. They contacted Savvis. It seems there was a port issue and has been resolved.
I do not work for Savvis or Century Link. If it fails again, just contact century link and have them go through the steps yet again.
But since it appears to be resolved... I'll see you all in game.
great work tommy6pack !
I cant test through the savvis network, but the cogent network is looking a little better tonight too @ 2%-4% packet loss, compared to 13% packet loss previously. I'll try to check a couple hours sooner tomorrow to see if peak time is occurring earlier.
if anyone going through the savvis network can do a: ping -t 208.95.185.41
and let it run for a minute, then you can see how stable/steady the ping is now. (note: hit ctrl-c to exit and see ping summary)
You are going through savvis network if you see savvis.net a lot in the latter 1/2 of your trace route: tracert 208.95.185.41
I was just having some really bad delays in-game with skills executing, etc. I took this ping check 5 minutes ago and again finding horrible packet loss:
This is once again only seen on the PatchRedirectServer , not the PatchServer which looked fine. Is it possible some prep work you're doing for the server maintenance might be causing this?
Pinging 208.95.185.41 with 32 bytes of data:
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=67ms TTL=52
Reply from 208.95.185.41: bytes=32 time=66ms TTL=52
Reply from 208.95.185.41: bytes=32 time=64ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Reply from 208.95.185.41: bytes=32 time=63ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=63ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Ping statistics for 208.95.185.41:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 61ms, Maximum = 67ms, Average = 62ms
Pinging 208.95.185.41 with 32 bytes of data:
Ping statistics for 208.95.185.41:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 61ms, Maximum = 67ms, Average = 62ms
tommy6pack, which network do you go through on your tracert, savvis or cogentco?
Also please note I am pinging 208.95.184.176.
The IP you pinged, 208.95.185.41, also looks good for me now.
You guys are perfectly fine... here is how it looks when you log from EU!
I have contacted my ISP and Cogentco... both of them say's that their networks are perfectly fine!!!
But the game is unplayable for me on EU peak hours... I'm getting tired of this now!
You guys are perfectly fine... here is how it looks when you log from EU!
I have contacted my ISP and Cogentco... both of them say's that their networks are perfectly fine!!!
But the game is unplayable for me on EU peak hours... I'm getting tired of this now!
Wow! That's terrible. You have 1700MS of jitter plus some packet loss and overall high latency.
Here's how you can tell where the bad part of the network is for you:
When your are in peak hours and have the peak problem, use your tracroute (tracert) to get the list of numbers you will need to ping, each for about 30 seconds or until you establish a problem exists.
1) ping the last ip address to verify the issue exists: ping -t 208.95.185.41
2) now work down the list 1 hop at a time and see what you have: ping -t 38.11.40.114
3) continue down the list until you no longer see the issue: ping -t 68.48.4.22
4) once you find where the bad ping stops, you will know where on your IP route that the problem lives.
Most likely you will only need to go backwards 1 or 2 hops. Once you find the culprit it helps you to know exactly who to contact with the problem, and you also have some proof to show them, because 90% of the time they will say they have no problems, because they only look at a very high level. If you post your findings here, it may also prove helpful to the Cryptic team to isolate a bad carrier.
0
delph3Member, Neverwinter Beta Users, Neverwinter Guardian UsersPosts: 3Arc User
edited June 2013
Verizon
Please HALP! RANDOM DCs at times...
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 7 ms 7 ms 6 ms L100.DLLSTX-VFTTP-72.verizon-gni.net [173.71.10.
1]
3 13 ms 8 ms 6 ms G0-14-0-2.DLLSTX-LCR-22.verizon-gni.net [130.81.
219.106]
4 8 ms 8 ms 6 ms so-5-0-0-0.DFW9-BB-RTR2.verizon-gni.net [130.81.
199.36]
5 60 ms 59 ms 59 ms 0.so-7-1-0.XL4.BOS4.ALTER.NET [152.63.0.221]
6 60 ms 59 ms 59 ms 0.ge-9-0-0.GW15.BOS4.ALTER.NET [152.63.24.42]
7 153 ms 151 ms 152 ms internap-gw.customer.alter.net [152.179.134.214]
8 60 ms 59 ms 59 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.1
04]
9 * 138 ms * perfectworldent-4.border11.bsn.pnap.net [216.52.
61.78]
10 112 ms 107 ms 111 ms 208.95.185.41
I am also having similar problems. For me they started right after the patch today. Random lag spikes, packet loss and rollbacks, then random disconnects.
Was fine yesterday but sucks today. Pretty sure this is a PWE issue... Been playing with no problems for long time until today. I will post my stats in a little bit but pretty sure it is useless.
Edit: So far seems to only be a problem in dungeons.
Edit: Ok take that back it is happening everywhere....
I am very disappointed, I Was looking forward to trying out the new NERFS to all my characters too......
Maybe I am missing it but I do not see a problem here?
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 4 ms 2 ms 9 ms 192.168.1.1
2 41 ms 15 ms 36 ms 74-130-48-1.dhcp.insightbb.com [74.130.48.1]
3 13 ms 12 ms 13 ms 74-128-16-241.dhcp.insightbb.com [74.128.16.241]
4 14 ms 15 ms 16 ms tge0-14-0-3.lsvmkyzo-ccr01.mwrtn.rr.com [65.29.25.190]
5 26 ms 31 ms 21 ms network-065-189-140-162.mwrtn.rr.com [65.189.140.162]
6 43 ms 39 ms 38 ms ae-4-0.cr0.chi30.tbone.rr.com [66.109.6.68]
7 31 ms 35 ms 34 ms ae-0-0.cr0.chi10.tbone.rr.com [66.109.6.20]
8 33 ms 31 ms 30 ms ae0.pr1.chi10.tbone.rr.com [107.14.17.192]
9 69 ms 77 ms 73 ms te0-3-0-27.ccr21.ord03.atlas.cogentco.com [154.54.11.145]
10 82 ms 81 ms 78 ms be2004.mpd22.ord01.atlas.cogentco.com [154.54.5.9]
11 106 ms 67 ms * te0-7-0-32.ccr21.bos01.atlas.cogentco.com [154.54.45.218]
12 116 ms 206 ms 277 ms te4-2.ccr01.bos06.atlas.cogentco.com [66.28.4.254]
13 72 ms 72 ms 73 ms 38.111.40.114
14 89 ms * 90 ms 208.95.185.41
Trace complete.
ping 208.95.185.41
Pinging 208.95.185.41 with 32 bytes of data:
Reply from 208.95.185.41: bytes=32 time=67ms TTL=47
Reply from 208.95.185.41: bytes=32 time=78ms TTL=47
Reply from 208.95.185.41: bytes=32 time=67ms TTL=47
Reply from 208.95.185.41: bytes=32 time=75ms TTL=47
Ping statistics for 208.95.185.41:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 67ms, Maximum = 78ms, Average = 71ms
I was just having some really bad delays in-game with skills executing, etc. I took this ping check 5 minutes ago and again finding horrible packet loss:
I wish I had your packet loss. I'm at about 50% right now, with a slightly better 86mS ping.
Tracing route to 208.95.184.176 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 7 ms 6 ms 6 ms L100.BSTNMA-VFTTP-109.verizon-gni.net [96.252.62.1]
3 10 ms 9 ms 9 ms G0-15-2-1.BSTNMA-LCR-22.verizon-gni.net [130.81.32.98]
4 20 ms 9 ms 9 ms so-5-0-0-0.BOS-BB-RTR2.verizon-gni.net [130.81.151.220]
5 12 ms 12 ms 11 ms 0.ge-1-0-0.XL4.BOS4.ALTER.NET [152.63.19.245]
6 10 ms 11 ms 10 ms 0.xe-11-2-0.GW15.BOS4.ALTER.NET [152.63.21.162]
7 98 ms 97 ms 97 ms internap-gw.customer.alter.net [152.179.134.214]
8 18 ms 17 ms 16 ms border11.te7-1-bbnet1.bsn.pnap.net [63.251.128.41]
9 75 ms 69 ms 72 ms perfectworldent-4.border11.bsn.pnap.net [216.52.61.78]
10 86 ms * * 208.95.184.176
11 87 ms * 84 ms 208.95.184.176
Prior to today the game has always been playable with the notable execption of foundry quests. For whatever reason when I first load a new zone it takes at least 5+ minutes to get the patches with minutes sitting idle where the game reports the patch is downloading between 0-5kB/s. (Speed test always shows at least 30Mb/s download so the problem isn't my connection to my ISP)
Comments
Century Link
S.S. Doff Lundgren
[SIGPIC][/SIGPIC]
Tracing route to patchserver.crypticstudios.com [208.95.185.41] over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms ___.___.____.___
2 23 ms 23 ms 23 ms 184.99.0.12
3 22 ms 23 ms 23 ms cosp-agw1.inet.qwest.net [184.99.1.89]
4 59 ms 59 ms 69 ms lap-brdr-03.inet.qwest.net [67.14.22.78]
5 58 ms 58 ms 59 ms 63-235-40-86.dia.static.qwest.net [63.235.40.86]
6 59 ms 60 ms 60 ms cr2-tengig-0-7-4-0.lay.savvis.net [204.70.198.9]
7 105 ms 105 ms 105 ms cr2-te-0-13-2-0.nyr.savvis.net [206.28.96.53]
8 105 ms 105 ms 105 ms er1-te-2-1.newyorknyd.savvis.net [204.70.196.70]
9 115 ms 115 ms 114 ms msr1-te-0-3-0-0.bos.savvis.net [206.28.97.205]
10 111 ms 112 ms 111 ms 206.28.97.137
11 109 ms 109 ms 108 ms internap.Boston.savvis.net [208.172.51.230]
12 110 ms 109 ms 109 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.104]
13 172 ms 162 ms 162 ms perfectworldent-4.border11.bsn.pnap.net [216.52.61.78]
14 163 ms 162 ms 163 ms 208.95.185.41
Trace complete.
We grow not old so long as we continue to chase our dreams.
1 8 ms 9 ms 19 ms 10.83.0.1
2 9 ms 9 ms 7 ms bla1-ge0-0-3.gw.optusnet.com.au [198.142.192.169]
3 * * * Request timed out.
4 8 ms 11 ms 9 ms bla5-ge13-0.gw.optusnet.com.au [211.29.125.249]
5 173 ms 163 ms 159 ms 203.208.131.57
6 157 ms 160 ms 160 ms 203.208.153.46
7 158 ms 157 ms 161 ms be2020.ccr22.lax01.atlas.cogentco.com [154.54.88.17]
8 193 ms 195 ms 193 ms te0-2-0-3.mpd22.iah01.atlas.cogentco.com [154.54.45.9]
9 209 ms 211 ms 218 ms te0-2-0-2.ccr22.atl01.atlas.cogentco.com [154.54.42.214]
10 231 ms 220 ms 220 ms te0-0-0-3.mpd22.dca01.atlas.cogentco.com [154.54.28.209]
11 226 ms 227 ms 227 ms te0-3-0-3.mpd22.jfk02.atlas.cogentco.com [154.54.5.74]
12 232 ms 232 ms 232 ms te0-3-1-1.ccr22.bos01.atlas.cogentco.com [154.54.44.62]
13 234 ms 233 ms 233 ms te4-2.ccr01.bos06.atlas.cogentco.com [66.28.4.254]
14 239 ms 246 ms 237 ms 38.111.40.114
15 234 ms 231 ms 240 ms 208.95.185.41
Trace complete.
Optus
What Class Are You?
I have constant spikes on every 3-7 seconds everywhere in NW!!! This makes the game unplayable for me... This problem started on June 2/2013 and did not get resolved so far ;( I really have no idea whats going on with your servers but I'm really frustrated now...
I log from Cyprus/EU, my ISP name is MTN
Here is my tracing route:
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1
2 20 ms 22 ms 22 ms 213.207.173.1
3 19 ms 19 ms 22 ms 213.207.129.28
4 22 ms 57 ms 23 ms 213.207.128.234
5 88 ms 90 ms 88 ms 41-208-29-154.uk.mtnns.net [41.208.29.154]
6 90 ms 88 ms 87 ms ls-pr-1.uk--ls-cr-2.uk-a.mtn.net [209.212.111.201]
7 89 ms 87 ms 87 ms TenGigabitEthernet8-2.ar6.LON3.gblx.net [208.48.238.185]
8 162 ms 161 ms 165 ms te0-7-0-21.ccr21.jfk07.atlas.cogentco.com [154.54.14.69]
9 166 ms 157 ms 170 ms te0-1-0-0.mpd22.jfk02.atlas.cogentco.com [154.54.1.221]
10 179 ms 179 ms 166 ms te0-6-0-5.ccr22.bos01.atlas.cogentco.com [154.54.44.26]
11 167 ms 181 ms 164 ms te4-2.ccr01.bos06.atlas.cogentco.com [66.28.4.254]
12 169 ms 177 ms 177 ms 38.111.40.114
13 186 ms 184 ms 184 ms 208.95.185.41
Trace complete.
We have investigated this issue and it appears that some players are encountering an issue with a routing provider (Savvis) that exists between their ISP and our servers. We recommend waiting some time to see if the issue is resolved on their end, or alternatively, contacting your ISP to report the issue.
We will continue to monitor for additional issues and will investigate accordingly if new information is discovered. Thank you for letting us know about this!
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms
2 25 ms 25 ms 25 ms desm-dsl-gw26.desm.qwest.net
3 22 ms 23 ms 23 ms desm-agw1.inet.qwest.net [75.160.210.201]
4 38 ms 38 ms 39 ms chp-brdr-03.inet.qwest.net [67.14.8.190]
5 38 ms 38 ms 38 ms 208.174.226.229
6 38 ms 39 ms 39 ms cr1-tengig-0-5-0-0.chicago.savvis.net [204.70.19
6.21]
7 60 ms 67 ms 60 ms cr2-ten-0-13-0-0.nyr.savvis.net [204.70.200.77]
8 60 ms 61 ms 61 ms er1-te-2-1.newyorknyd.savvis.net [204.70.196.70]
9 68 ms 69 ms 69 ms msr1-te-0-3-0-0.bos.savvis.net [206.28.97.205]
10 65 ms 66 ms 66 ms 206.28.97.137
11 67 ms 67 ms 66 ms internap.Boston.savvis.net [208.172.51.230]
12 68 ms 67 ms 67 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.1
04]
13 130 ms 126 ms 125 ms perfectworldent-4.border11.bsn.pnap.net [216.52.
61.78]
14 118 ms 118 ms 118 ms 208.95.185.41
Trace complete.
I do not use the provider Savvis... what about my problem that persist for more then a week already?
the problem is between your ISP and Neverwinter. Third party.
When I look at my trace routes there is nothing wrong with them... and I play any other game perfectly fine! With latency below 200ms... when I play NW is always above 200ms and I get constant spikes...
Massive lag spikes every few seconds... some disconnects.
Started yesterday around 7PM PST.
My ISP is Centurylink.
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.0.1
2 22 ms 23 ms 22 ms 67.40.227.130
3 23 ms 23 ms 23 ms phnx-agw1.inet.qwest.net [75.160.238.1]
4 34 ms 34 ms 35 ms lap-brdr-03.inet.qwest.net [67.14.22.78]
5 35 ms 35 ms 34 ms 63-235-40-86.dia.static.qwest.net [63.235.40.86]
6 35 ms 36 ms 36 ms cr1-tengig-0-5-0-0.lay.savvis.net [204.70.198.5]
7 103 ms 104 ms 104 ms cr2-bundle-pos-1.newyork.savvis.net [204.70.197.
34]
8 102 ms 110 ms 103 ms er1-te-2-1.newyorknyd.savvis.net [204.70.196.70]
9 110 ms 111 ms 110 ms msr1-te-0-3-0-0.bos.savvis.net [206.28.97.205]
10 108 ms 108 ms 109 ms 206.28.97.137
11 107 ms 106 ms 106 ms internap.Boston.savvis.net [208.172.51.230]
12 107 ms 106 ms 106 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.1
04]
13 163 ms 163 ms 175 ms perfectworldent-4.border11.bsn.pnap.net [216.52.
61.78]
14 160 ms 160 ms 161 ms 208.95.185.41
Trace complete.
I noticed this issue a couple days ago when it was really bad. I just came back later when the network wasn't as saturated and gameplay was somewhat better. The problem wasn't as visible on the PatchServer as it was on the PatchRedirectServer.
I go through Cogento, not Savvis, which indicated this probably affects multiple ISP's. The problem appears to exist in the Boston area. I have limited a view from here, but it looks like the problem possibly exists between the last hop leaving the tier 1 provider before going into the 38.111.40.114 network. The problem was definitely cyclic packet loss (please see attached picture), which is far worse than latency issues. Latency = slow or delayed connectivity. Packet loss = no connectivity for a specified time period. Packet loss usually occurs due to over-utilization of a particular network element. Unfortunately pushing networks to saturation is a standard ISP practice, and unless many complaints are filed AND ESCALATED, nothing will likely be done unless the culprit element pops up on someone's internal monitoring software. This issue might well be related to a measure of success in that there may be more client connectivity that you anticipated at this point, but it appears so far to only reveal itself on the "weakest" links. In other words, this problem will only affect specific people on the same ISP's that happen to have their routing through this particular over-utilized network element.
I investigated realtime and historically and there are currently no peering issues between any of the major US tier 1 provider handoffs. That's good news. But the bad news is I've dealt with problems like this before(on the corporate level) and they can be very difficult to resolve without putting someone on permanent escalation duty with Cryptic's provider of choice. I know you don't have control of that connectivity outside your network, but neither do the clients ISP's, and that's the biggest problem that this issue exists somewhere in the middle.
How to Help:
1) if you're a gamer that's experiencing these issues, it cant hurt to open a ticket with your ISP, but be prepared, they will have no idea what you are talking about, and will likely just tell you to reset your router. You will need to ask directly to escalate to at least tier II or tier III support, these are the folks that deal with ISP interconnectivity. You ISP will probably deny your escalation request unless you just get lucky. You can also ask them to provide you with a tier II or III email address, which they may, or may not provide. If you happen to get any Tier II or III contact, give them your traceroute to the cryptic server and tell them you are experiencing high packet loss on this connection...after that, its up to them. If they don't see the problem, they will close the ticket. If they see the problem, they will open a ticket with the middleman, and it might get addressed.
2) If you are Cryptic. Its a similar scenario in that your facility provider can only open a ticket with the suspect tier 1 ISP. However, unless you really put the pressure on your facility provider, they will likely not put the pressure on the tier 1 provider to find a suitable solution. Most likely they will claim that their connectivity is operating within agreed QOS terms, so it wont get the attention that it needs unless you continue to escalate, and escalate, and escalate the issue. Note on the attached picture the 13% packet loss...this level of packet loss definitely would not be in compliance with any QOS.
What to Do Short Term as a Gamer:
Unfortunately, if this problem persists, you only have 2 choices. Either play earlier or later when less people are online, or get a new ISP and hope it doesn't follow the same exact network route to Cryptic's servers. If you are on a laptop, you could try a friends house with a different ISP and see if the problem exists there. Also it might be helpful if people experiencing the problem state their ISP in the comments, then we can make a short list of culprit ISP's.
EDIT: The next time I see the problem getting bad, I'll locate the exact hop where the problem exists and post a screenshot of it. Right now its just mild latency spikes which probably aren't noticeable on the client side.
I do not work for Savvis or Century Link. If it fails again, just contact century link and have them go through the steps yet again.
But since it appears to be resolved... I'll see you all in game.
great work tommy6pack !
I cant test through the savvis network, but the cogent network is looking a little better tonight too @ 2%-4% packet loss, compared to 13% packet loss previously. I'll try to check a couple hours sooner tomorrow to see if peak time is occurring earlier.
if anyone going through the savvis network can do a: ping -t 208.95.185.41
and let it run for a minute, then you can see how stable/steady the ping is now. (note: hit ctrl-c to exit and see ping summary)
You are going through savvis network if you see savvis.net a lot in the latter 1/2 of your trace route: tracert 208.95.185.41
I uninstalled and reinstalled the game... now the first login page when opening says i need to enable the javascript?
Any ideas???
This is once again only seen on the PatchRedirectServer , not the PatchServer which looked fine. Is it possible some prep work you're doing for the server maintenance might be causing this?
Pinging 208.95.185.41 with 32 bytes of data:
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=67ms TTL=52
Reply from 208.95.185.41: bytes=32 time=66ms TTL=52
Reply from 208.95.185.41: bytes=32 time=64ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Reply from 208.95.185.41: bytes=32 time=63ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=63ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Reply from 208.95.185.41: bytes=32 time=62ms TTL=52
Reply from 208.95.185.41: bytes=32 time=61ms TTL=52
Ping statistics for 208.95.185.41:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 61ms, Maximum = 67ms, Average = 62ms
tommy6pack, which network do you go through on your tracert, savvis or cogentco?
Also please note I am pinging 208.95.184.176.
The IP you pinged, 208.95.185.41, also looks good for me now.
I have contacted my ISP and Cogentco... both of them say's that their networks are perfectly fine!!!
But the game is unplayable for me on EU peak hours... I'm getting tired of this now!
Wow! That's terrible. You have 1700MS of jitter plus some packet loss and overall high latency.
Here's how you can tell where the bad part of the network is for you:
When your are in peak hours and have the peak problem, use your tracroute (tracert) to get the list of numbers you will need to ping, each for about 30 seconds or until you establish a problem exists.
1) ping the last ip address to verify the issue exists: ping -t 208.95.185.41
2) now work down the list 1 hop at a time and see what you have: ping -t 38.11.40.114
3) continue down the list until you no longer see the issue: ping -t 68.48.4.22
4) once you find where the bad ping stops, you will know where on your IP route that the problem lives.
Most likely you will only need to go backwards 1 or 2 hops. Once you find the culprit it helps you to know exactly who to contact with the problem, and you also have some proof to show them, because 90% of the time they will say they have no problems, because they only look at a very high level. If you post your findings here, it may also prove helpful to the Cryptic team to isolate a bad carrier.
Please HALP! RANDOM DCs at times...
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 7 ms 7 ms 6 ms L100.DLLSTX-VFTTP-72.verizon-gni.net [173.71.10.
1]
3 13 ms 8 ms 6 ms G0-14-0-2.DLLSTX-LCR-22.verizon-gni.net [130.81.
219.106]
4 8 ms 8 ms 6 ms so-5-0-0-0.DFW9-BB-RTR2.verizon-gni.net [130.81.
199.36]
5 60 ms 59 ms 59 ms 0.so-7-1-0.XL4.BOS4.ALTER.NET [152.63.0.221]
6 60 ms 59 ms 59 ms 0.ge-9-0-0.GW15.BOS4.ALTER.NET [152.63.24.42]
7 153 ms 151 ms 152 ms internap-gw.customer.alter.net [152.179.134.214]
8 60 ms 59 ms 59 ms border11.te8-1-bbnet2.bsn.pnap.net [63.251.128.1
04]
9 * 138 ms * perfectworldent-4.border11.bsn.pnap.net [216.52.
61.78]
10 112 ms 107 ms 111 ms 208.95.185.41
Trace complete.
I guess at least rift is free to play now :P
Did a quick test on another MMO and had no problems.
Edit: So far seems to only be a problem in dungeons.
Edit: Ok take that back it is happening everywhere....
I am very disappointed, I Was looking forward to trying out the new NERFS to all my characters too......
Maybe I am missing it but I do not see a problem here?
Tracing route to patchserver.crypticstudios.com [208.95.185.41]
over a maximum of 30 hops:
1 4 ms 2 ms 9 ms 192.168.1.1
2 41 ms 15 ms 36 ms 74-130-48-1.dhcp.insightbb.com [74.130.48.1]
3 13 ms 12 ms 13 ms 74-128-16-241.dhcp.insightbb.com [74.128.16.241]
4 14 ms 15 ms 16 ms tge0-14-0-3.lsvmkyzo-ccr01.mwrtn.rr.com [65.29.25.190]
5 26 ms 31 ms 21 ms network-065-189-140-162.mwrtn.rr.com [65.189.140.162]
6 43 ms 39 ms 38 ms ae-4-0.cr0.chi30.tbone.rr.com [66.109.6.68]
7 31 ms 35 ms 34 ms ae-0-0.cr0.chi10.tbone.rr.com [66.109.6.20]
8 33 ms 31 ms 30 ms ae0.pr1.chi10.tbone.rr.com [107.14.17.192]
9 69 ms 77 ms 73 ms te0-3-0-27.ccr21.ord03.atlas.cogentco.com [154.54.11.145]
10 82 ms 81 ms 78 ms be2004.mpd22.ord01.atlas.cogentco.com [154.54.5.9]
11 106 ms 67 ms * te0-7-0-32.ccr21.bos01.atlas.cogentco.com [154.54.45.218]
12 116 ms 206 ms 277 ms te4-2.ccr01.bos06.atlas.cogentco.com [66.28.4.254]
13 72 ms 72 ms 73 ms 38.111.40.114
14 89 ms * 90 ms 208.95.185.41
Trace complete.
ping 208.95.185.41
Pinging 208.95.185.41 with 32 bytes of data:
Reply from 208.95.185.41: bytes=32 time=67ms TTL=47
Reply from 208.95.185.41: bytes=32 time=78ms TTL=47
Reply from 208.95.185.41: bytes=32 time=67ms TTL=47
Reply from 208.95.185.41: bytes=32 time=75ms TTL=47
Ping statistics for 208.95.185.41:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 67ms, Maximum = 78ms, Average = 71ms
I wish I had your packet loss. I'm at about 50% right now, with a slightly better 86mS ping.
Tracing route to 208.95.184.176 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 7 ms 6 ms 6 ms L100.BSTNMA-VFTTP-109.verizon-gni.net [96.252.62.1]
3 10 ms 9 ms 9 ms G0-15-2-1.BSTNMA-LCR-22.verizon-gni.net [130.81.32.98]
4 20 ms 9 ms 9 ms so-5-0-0-0.BOS-BB-RTR2.verizon-gni.net [130.81.151.220]
5 12 ms 12 ms 11 ms 0.ge-1-0-0.XL4.BOS4.ALTER.NET [152.63.19.245]
6 10 ms 11 ms 10 ms 0.xe-11-2-0.GW15.BOS4.ALTER.NET [152.63.21.162]
7 98 ms 97 ms 97 ms internap-gw.customer.alter.net [152.179.134.214]
8 18 ms 17 ms 16 ms border11.te7-1-bbnet1.bsn.pnap.net [63.251.128.41]
9 75 ms 69 ms 72 ms perfectworldent-4.border11.bsn.pnap.net [216.52.61.78]
10 86 ms * * 208.95.184.176
11 87 ms * 84 ms 208.95.184.176
Prior to today the game has always been playable with the notable execption of foundry quests. For whatever reason when I first load a new zone it takes at least 5+ minutes to get the patches with minutes sitting idle where the game reports the patch is downloading between 0-5kB/s. (Speed test always shows at least 30Mb/s download so the problem isn't my connection to my ISP)