test content
What is the Arc Client?
Install Arc

What is being done about the lag and rubber banding? DEV RESPONSE Page 3

135

Comments

  • This content has been removed.
  • gulberatgulberat Member Posts: 5,505 Arc User
    edited August 2015
    I'll be putting out a more official update.

    Thank you, this will be greatly appreciated as you no doubt gather having read this thread.

    May I persist and suggest that we receive regular updates, I personally would suggest fortnightly. Don't be afraid to put on the updates that you tried X and Y and it didn't work, at least it's something to cross off the list. Then instead of self entitled players like me shouting and generally making a fuss, I'll be right behind you guys as we discuss this at fleet level and on the forum, your company's forum.

    This is a good idea, to just hold to a schedule and update on that day even if the update is that some particular attempt to fix things didn't work out as planned. I second this motion. :)

    Christian Gaming Community Fleets--Faith, Fun, and Fellowship! See the website and PM for more. :-)
    Proudly F2P.  Signature image by gulberat. Avatar image by balsavor.deviantart.com.
  • earwigvr6earwigvr6 Member Posts: 293 Arc User
    valoreah wrote: »
    I'll be putting out a more official update.

    Thank you. What can we do from our end to help alleviate some of these issues?
    ​​


    Basically the main thing everyone can do is to NOT use all the nice ships/traits and powers you have bought as it's those that create the lag.

    Put your command ship in the bin. Remove all the traits you use from the ships you have bought. Don't use APB, Kemocite etc.... etc....
  • pwlaughingtrendypwlaughingtrendy Member Posts: 2,966 Arc User
    edited August 2015
    earwigvr6 wrote: »
    valoreah wrote: »
    I'll be putting out a more official update.

    Thank you. What can we do from our end to help alleviate some of these issues?
    ​​


    Basically the main thing everyone can do is to NOT use all the nice ships/traits and powers you have bought as it's those that create the lag.

    Put your command ship in the bin. Remove all the traits you use from the ships you have bought. Don't use APB, Kemocite etc.... etc....
    Uh, no. That's not it. It's a little more complicated than that.
  • pwlaughingtrendypwlaughingtrendy Member Posts: 2,966 Arc User
    There are a lot of working elements that are being changed with the powers affecting lag. Yesterday I went down to Cryptic and spoke with a couple people who really were able to put it into a better perspective for me.

    The team is changing a lot of the way that these powers function with regards to the server. The major issue is their interaction with said server. Don't worry, your powers aren't going to be changing in a way that you can see (your pew pew and deeps deeps will largely go unaffected as I understand).

    The team has a combined effort of different things. For one, they're fixing the powers as they can find them. There's been a large list of powers that they've already worked on to reduce the stress put on any server-side processes. The other effort is that they've included monitoring processes on live to get a better analysis of how different powers are hitting the server.

    Let's say that mathematically, you have an interaction that creates a fair amount of server load. You get a few hundred people on Tribble using this at at time, and you might not see anything really substantial as far as server load goes.

    But what if that same thing interacts with thousands of people on Holodeck? What if it's exponentially heavier on the server rather than simply additive? What if that power interacts with other powers creating an exponential recursive effect? It hits hard. But the team is working on it. Hell, they're bloody well tearing into the problem.

    For a while, I have been putting out company-wide reports. I talk about this being an issue and in turn, Cryptic has been combing through the power interactions to change them so that you, the players, don't have this problem any more. They've showed me that they're working on it.


    First thing: Game Crashes

    The team has been working on a graphics related crash that's been dropping people. This is believed to be the major cause of crashes and it seems to be usually happening at or around startup. We have a test going through that will have potentially a major fix for it, but that's still in testing. We're getting more testing data on that.

    We're still analyzing crash data to look in terms of any specific area related to it. There isn't a specific classification data pool for it (brand of graphics card, O/S, ram size, etc.) so the cause is still a little nebulous. But we're still narrowing in.

    Second thing: Lag (Server Stalls)

    There's a specific distinction between server stalling and client stalling. We have our data to tell us that there's data stalls happening on the server, so that's where a lot of our attention has been. We've been spending a lot of time doing analysis on the combat (powers) systems. This meant a lot of retooling of previously expensive computational designs. We're constantly retooling these powers so that they work better in terms of overall server load. This has been a case/case/case basis.

    For some people, they see these in more intensive maps related to power usage. That being said, I'm happy with some of the changes I've been seeing. We're seeing another big batch of changes come into testing right now that we'll be able to analyze.

    ~LaughingTrendy
  • pwlaughingtrendypwlaughingtrendy Member Posts: 2,966 Arc User
    I'll be putting out a more official update.

    Thank you, this will be greatly appreciated as you no doubt gather having read this thread.

    May I persist and suggest that we receive regular updates, I personally would suggest fortnightly. Don't be afraid to put on the updates that you tried X and Y and it didn't work, at least it's something to cross off the list. Then instead of self entitled players like me shouting and generally making a fuss, I'll be right behind you guys as we discuss this at fleet level and on the forum, your company's forum.


    Without giving you a flat out yes, my full intention is to try and be more transparent. I'm making more regular trips to Cryptic now (I work at PWE) in order to get a better understanding so that I can communicate with ya'll. I don't want to say a regular X amount of days, but my full intention is to keep you informed as I learn more. Always.
  • earwigvr6earwigvr6 Member Posts: 293 Arc User
    It's actually very refreshing to at least get some form of update on the matter.

    I for one appreciate the time taken to post this, even though I still feel it's an issue that is not going to go away anytime soon. I'm just not sure it's high enough up on the priority list.
  • electrumleopardelectrumleopard Member Posts: 88 Arc User
    Without giving you a flat out yes, my full intention is to try and be more transparent. I'm making more regular trips to Cryptic now (I work at PWE) in order to get a better understanding so that I can communicate with ya'll. I don't want to say a regular X amount of days, but my full intention is to keep you informed as I learn more. Always.

    Do you think a Twitch Community Chat once a week could be a thing? Kinda like your impromptu twitch announcement. Occasionally drag down a Victim-of-the-week from Cryptic to join you. Or reddit if twitch is too chaotic.
  • borticuscrypticborticuscryptic Member Posts: 2,478 Cryptic Developer
    To piggyback on Trendy's statements, I'd like to let everyone know that we're already discussing the option of doing a "Performance Review" Tribble event, once we have a number of our more prominent powers tweaks in place. Basically inviting players to come play a few specific queues, with Devs playing along, to see if we can see any measurable improvements with power performance.

    So, keep your eyes open if you'd like to be a part of such an event. We'll make an official announcement, once everything's situated.
    Jeremy Randall
    Cryptic - Lead Systems Designer
    "Play smart!"
  • This content has been removed.
  • sqwishedsqwished Member Posts: 1,475 Bug Hunter
    To piggyback on Trendy's statements, I'd like to let everyone know that we're already discussing the option of doing a "Performance Review" Tribble event, once we have a number of our more prominent powers tweaks in place. Basically inviting players to come play a few specific queues, with Devs playing along, to see if we can see any measurable improvements with power performance.

    So, keep your eyes open if you'd like to be a part of such an event. We'll make an official announcement, once everything's situated.

    Now this^^ is some of the best news I've seen in a while, count me if you need help.
    Oh, it's not broken? We can soon fix that!

  • tarastheslayertarastheslayer Member Posts: 1,541 Bug Hunter
    sqwished wrote: »
    To piggyback on Trendy's statements, I'd like to let everyone know that we're already discussing the option of doing a "Performance Review" Tribble event, once we have a number of our more prominent powers tweaks in place. Basically inviting players to come play a few specific queues, with Devs playing along, to see if we can see any measurable improvements with power performance.

    So, keep your eyes open if you'd like to be a part of such an event. We'll make an official announcement, once everything's situated.

    Now this^^ is some of the best news I've seen in a while, count me if you need help.

    Same here, that'd be fun :D
    Ten soldiers wisely led will beat a hundred without a head. - Euripides
    I no longer do any Bug Hunting work for Cryptic. I may resume if a serious attempt to fix the game is made.
  • primar13primar13 Member Posts: 1,896 Bug Hunter
    To piggyback on Trendy's statements, I'd like to let everyone know that we're already discussing the option of doing a "Performance Review" Tribble event, once we have a number of our more prominent powers tweaks in place. Basically inviting players to come play a few specific queues, with Devs playing along, to see if we can see any measurable improvements with power performance.

    So, keep your eyes open if you'd like to be a part of such an event. We'll make an official announcement, once everything's situated.

    Borticus I would very much love that to be a thing... With a LOT of People on at one time!

    PLEASE if this happens give us 2-3 days notice... So that we can get the word out to as many fleet mates as possible and allow them to get tribble patched up. For many of them who are not on tribble that often, it could take a while.
  • borticuscrypticborticuscryptic Member Posts: 2,478 Cryptic Developer
    The conversation on this subject going on over on our sub-reddit has even more details on some of the power changes we've recently been working on. I'll go ahead and copy that information over here, for those that'd rather stick with the Official Forums...

    Borg Badness:

    All Borg Ships have an ability called "Assimilate Ship" which players will recognize as the big "red borders on the screen" ability that mucks with power recharges and such. This ability was built in such a way that each tick was as expensive as creating an entire copy of your ship, complete with all ~300 powers a player ship possesses.. And each Borg Ship could do this as frequently as once every 6 seconds.

    "Assimilate Ship" was recently rebuilt so that it no longer ... screws everything up. The only player-facing difference will be that if you destroy the ship using this on you, the effect will not linger for more than 6 seconds, whereas it used to run its full duration (up to 60sec) even if the caster was killed.

    Crystalline Entity:

    Y'know how certain abilities can trip that Re-Crystallize Buff off the CE, but only has that effect on this specific Map? That's because we were granting you an additional power effect in order to do that. And we were doing so every 1 sec. And it was being added to all "foes" of the CE on the entire map. And each time it did so, it was again just as expensive as creating an entire copy of your ship, complete with all ~300 powers a player ship possesses. Every second.

    After a recent rework, it now only does this once every 5 seconds (1/5th the server load!), and only applies to Players thus reducing the impact by not attempting to add the effect to Torpedoes, Mines, Shuttles, Hangar Pets, etc. It also only refreshes every 5 seconds, instead of removing and re-applying, further reducing the server-side performance hit.

    This does mean that Hangar Pets that use Tachyon Beam and such, will no longer strip Re-Crystallize buffs from the CE. We decided the trade-off is worth it.

    Attack Pattern Beta:

    Again, as in the previous two examples, one aspect of APB was being re-applied to players that used this power on a 1x/sec basis, which - with every tick - was that massively expensive transaction where it had to reload every single one of your powers.

    This one was really tricky, and we actually had to create an entirely new type of Power just to facilitate reducing its server impact without altering its player-facing functionality. Suffice it to say, this ability can be used safely without making the servers weep. And, according to the internal testing we've been doing, there is absolutely no difference in the end-result performance of the ability, after the rebuild.
    Jeremy Randall
    Cryptic - Lead Systems Designer
    "Play smart!"
  • kyrrokkyrrok Member Posts: 1,352 Arc User
    edited August 2015
    An honorable attempt at being more transparent is much appreciated. Even more so would be solutions to the problems.

    Unfortunately my support tickets get responses none better than automated acknowledgement of the problem which is not acknowledgement at all. And when there is a response that is not automated, it says that the case hase been closed and considered solved without even the intention of solving it. This may be the inappropriate avenue for this, but I have tried ALL appropriate avenues with absolutely no success.

    Once again your discussion of the situation and solutions are appreciated.
  • This content has been removed.
  • flyingtargflyingtarg Member Posts: 105 Cryptic Developer
    edited August 2015
    I think it's important to add that a lot of the changes we're making are based off of the profiling we've done both on internal developer and test servers, and on Holodeck. When we do this profiling, we get a good overview of what operations are running and how much CPU they're taking up.

    This lets us see that some things are taking way longer or happening way more often than expected, such as the full reload of powers with Attack Pattern Beta, Assimilate Ship and the Crystalline Entity buff-stripping power. We chose to target things that perform these operations, and find means with which we can change them to not do this, as Borticus said.

    The thought in the back our minds while we make these changes, though, is that while these changes will certainly help eliminate these full power reloads, in the end, all it might do is change what our profiling results look like, and it might not have any real impact on the lag and rubberbanding you guys are seeing.

    That's still useful information for us if that happens though. If things don't improve, then at the very least we're removing some of the "noise" in our profiling, and it lets us know that we need to focus our efforts on other, less significant data points.

    So just be prepared, this might be a longer fight than just one patch or one set of changes. But we're determined to take every necessary step to make a better play experience.
    Daniel "FlyingTarg" Razza
    Star Trek Online Lead Programmer
  • captainpugwash1captainpugwash1 Member Posts: 908 Arc User
    Well it comes down to they broke their own game by releasing content again & again that was not properly tested due to deadlines.
    Now its in such a mess they dont no how to fix it.

    6 months & counting.

  • This content has been removed.
  • lucho80lucho80 Member Posts: 6,600 Bug Hunter
    The conversation on this subject going on over on our sub-reddit has even more details on some of the power changes we've recently been working on. I'll go ahead and copy that information over here, for those that'd rather stick with the Official Forums...

    Borg Badness:

    All Borg Ships have an ability called "Assimilate Ship" which players will recognize as the big "red borders on the screen" ability that mucks with power recharges and such. This ability was built in such a way that each tick was as expensive as creating an entire copy of your ship, complete with all ~300 powers a player ship possesses.. And each Borg Ship could do this as frequently as once every 6 seconds.

    "Assimilate Ship" was recently rebuilt so that it no longer ... screws everything up. The only player-facing difference will be that if you destroy the ship using this on you, the effect will not linger for more than 6 seconds, whereas it used to run its full duration (up to 60sec) even if the caster was killed.

    Crystalline Entity:

    Y'know how certain abilities can trip that Re-Crystallize Buff off the CE, but only has that effect on this specific Map? That's because we were granting you an additional power effect in order to do that. And we were doing so every 1 sec. And it was being added to all "foes" of the CE on the entire map. And each time it did so, it was again just as expensive as creating an entire copy of your ship, complete with all ~300 powers a player ship possesses. Every second.

    After a recent rework, it now only does this once every 5 seconds (1/5th the server load!), and only applies to Players thus reducing the impact by not attempting to add the effect to Torpedoes, Mines, Shuttles, Hangar Pets, etc. It also only refreshes every 5 seconds, instead of removing and re-applying, further reducing the server-side performance hit.

    This does mean that Hangar Pets that use Tachyon Beam and such, will no longer strip Re-Crystallize buffs from the CE. We decided the trade-off is worth it.

    Attack Pattern Beta:

    Again, as in the previous two examples, one aspect of APB was being re-applied to players that used this power on a 1x/sec basis, which - with every tick - was that massively expensive transaction where it had to reload every single one of your powers.

    This one was really tricky, and we actually had to create an entirely new type of Power just to facilitate reducing its server impact without altering its player-facing functionality. Suffice it to say, this ability can be used safely without making the servers weep. And, according to the internal testing we've been doing, there is absolutely no difference in the end-result performance of the ability, after the rebuild.

    So following your examples and picking stuff that affects player and target, I'm assuming stuff like Attack Pattern Delta, the Voth diplomat doff, Energy Siphon, and the DSD deflector, could fall into these expensive powers category? Not sure about Sensory Analysis since it seems like either a self buff or target debuff (the shield drain/heal buff was reported broken).
  • doyouwdoyouw Member Posts: 200 Arc User
    I'm just wondering what was that +12h downtime we had back 3 weeks ago, supposed to improve lag etc. From what I've read it was hardware trouble but solved nothing at all. It's even worst since and geting worse each weekly patch for a lot of us.
    For example since last patch I barely play STO because I get DC +50 times / day, SNR each 10-15sec and increasing as soon as it is US prime time.
    It remind me what I've read/heard about Neverwinter launching on Xbox, "be prepare to get massive lag on STO as they probably wont add new hardware but just reduce each game like STO bandwith so Xbox new players will be able to connect"....
    looks like this wasnt oax since the result is the same predicted anyway.
    Dark Side (KDF)
    HOUSE OF BORG
    "I am FLATULUS of Borg, Resistance if Futile! Prepare to pull my finger!"

    cube.jpg
  • This content has been removed.
  • entnx01entnx01 Member Posts: 549 Arc User
    To piggyback on Trendy's statements, I'd like to let everyone know that we're already discussing the option of doing a "Performance Review" Tribble event, once we have a number of our more prominent powers tweaks in place. Basically inviting players to come play a few specific queues, with Devs playing along, to see if we can see any measurable improvements with power performance.

    So, keep your eyes open if you'd like to be a part of such an event. We'll make an official announcement, once everything's situated.

    As long as you'll provide Tribble Testing Manuals or some such similar item to test certain powers (instead of assuming we have them), I'd be interested in testing like this.

    If there are certain specific setups you need, let us know that too, perhaps provide materials for it in a Tester's Box?

  • xapocalypseponyxxapocalypseponyx Member Posts: 577 Arc User
    I'll go ahead and copy that information over here, for those that'd rather stick with the Official Forums...

    Maybe the Arcgames STO page should link to Reddit?
    flyingtarg wrote: »
    So just be prepared, this might be a longer fight than just one patch or one set of changes. But we're determined to take every necessary step to make a better play experience.

    While the very lengthy period that this has been going on has been, trying, the lack of communication has been just as frustrating. This is not some nuisance bug that would be nice to have fixed. It's down-right game breaking at times and to have such long periods of silence on such a serious issue, it's left me wondering more than once if you were not talking because for whatever reason, there was no more action being taken/to take. That's not a state you should want to leave customers in.

    The information you've given today is great. What you are describing would explain much of what I've been seeing and shows you have a grasp of the issue and there is hope. Thank you for that and please do keep us updated.

    I can prepare myself for a bit more of a wait, but keep in mind, this has been dragging on for a long time. Longer than some realize. It's been... irritating.
  • thlaylierahthlaylierah Member Posts: 2,987 Arc User
    A major problem I don't see being addressed is the use of Cogentco, the source of all bandwidth backups.

    This will require that they spend more money on a reliable service instead of this bargain basement one.

    Also I remember something about a company purposefully throttling bandwidth to force another company to comply with their wishes, was Cogentco in there anywhere?
  • theanothernametheanothername Member Posts: 1,511 Arc User
    edited August 2015
    To all Cryptic / PWE personel here: Thats a very appreciated blast from the past open communication. Thanks for not just explaining but keeping up "blogging" about this. And since the STO forums are the first source I check out for STO related stuff as STO player thanks for posting it here.
    A major problem I don't see being addressed is the use of Cogentco, the source of all bandwidth backups.

    This will require that they spend more money on a reliable service instead of this bargain basement one.

    Also I remember something about a company purposefully throttling bandwidth to force another company to comply with their wishes, was Cogentco in there anywhere?

    While that may totally be true (noob alert) maybe the reworked powers also reduce the amount of communication being pressed trough the Cogentcos C64s and allow a better gaming? Its not all the time bad lag & rubber banding; its usually just the prime times. Mid week noon is usually great. Or at least was on my last holidays.
  • plaztikman64plaztikman64 Member Posts: 725 Arc User
    gulberat wrote: »
    urmuz1 wrote: »
    primar13 wrote: »
    The Devs Commented on this Repeatedly today! Please read the many Reddit Posts!

    https://www.reddit.com/r/sto/comments/3fkrtd/lagits_been_beaten_to_deathbut_shouldnt_it_be/ctqkfea

    I don't play Reddit Online, so i don't read Reddit Posts. Devs answering in an unofficial STO forum is an unofficial answer

    Unfortunately, this is how I feel--whether it's STO or Duolingo where devs answer things on Reddit instead of on their official site. Unless it's on the official site, it isn't official.

    What is also sad and annoying is that someone in that reddit blames us, the forumgoers, for being the reason that we don't receive official updates. While I sometimes call the business decisions like they are, I hope I demonstrate that it is perfectly possible to do so without a bunch of personal invective.

    I don't believe that there are only the people who read these forums, and those who only go to reddit. How else would he know about the trolls here in the first place? Trolls are everywhere, that's a fact, and excluding reddit from it is...foolish, obviously (just gave you an example :) ) .
  • plaztikman64plaztikman64 Member Posts: 725 Arc User
    edited August 2015
    flyingtarg wrote: »
    I think it's important to add that a lot of the changes we're making are based off of the profiling we've done both on internal developer and test servers, and on Holodeck. When we do this profiling, we get a good overview of what operations are running and how much CPU they're taking up.

    This lets us see that some things are taking way longer or happening way more often than expected, such as the full reload of powers with Attack Pattern Beta, Assimilate Ship and the Crystalline Entity buff-stripping power. We chose to target things that perform these operations, and find means with which we can change them to not do this, as Borticus said.

    The thought in the back our minds while we make these changes, though, is that while these changes will certainly help eliminate these full power reloads, in the end, all it might do is change what our profiling results look like, and it might not have any real impact on the lag and rubberbanding you guys are seeing.

    That's still useful information for us if that happens though. If things don't improve, then at the very least we're removing some of the "noise" in our profiling, and it lets us know that we need to focus our efforts on other, less significant data points.

    So just be prepared, this might be a longer fight than just one patch or one set of changes. But we're determined to take every necessary step to make a better play experience.

    Speaking about a better play experience, I wonder - would it be possible to remove visual/sfx effects of traits, set bonuses, you name it (individually or alltogether, just like it is possible with the set parts (space & ground) / borg console)

    and

    is it (could it be) possible that said effects are also responsible for some lag and/or fps drops, especially at high load areas like the battlezones (~20 players)?


    Anyway, thanks for all the informations from you, trendy & bort. It sure helps putting things into a better perspective for most of the playerbase.
  • belidosbelidos Member Posts: 452 Arc User
    edited August 2015
    beameddown wrote: »
    can we get an official timetable/estimate of when the lag will be gone?

    This is something that annoys me a lot in my job, people asking when something will be fixed, I feel like yelling "I'm not <expletive> psychic" at them.

    It's almost impossible to put a time on it, they've said they have some ideas and they are trying them, if it works then fine it works all fixed, if it doesn't work then they have to start over and look at something else.

    Fixing old code in games like this doesn't come with a time line, it could take and hour it could take years there's no real way to tell because it's all about trial and error.

    The best they can offer is something along the lines of "we have a patch that we're working on that may work, we'll patch it in next week and see how it goes".
  • nyxadrillnyxadrill Member Posts: 1,242 Arc User
    edited August 2015
    Can I just pause the thread here and say thanks to Mattjohnsonva for raising this thread, as its generated just the feedback we were desperate for from developers.

    I'm under no illusions that there is a quick fix to this. Its a complex animal (putting it mildly!) and I understand the processes involved in diagnostics.

    Borticus, Flyingtarg and Trendy. Thank you. I appreciate greatly the time taken to step out of your routines and reply to the thread. And to reply with such excellent detail. Faith restored :)

    Please continue to post on the forum. Personally its the place I go to and I know many other do to.

    EDIT: Oh and Trendy...you not allowed to leave PWE ... ever :) Joking aside I think you are the best community admin we've had.
    server_hamster6.png
Sign In or Register to comment.