test content
What is the Arc Client?
Install Arc
Options

Priority One Podcast - Episode 76 - Charles and the Ferengi Factory

12346»

Comments

  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Tumerboy wrote: »
    You guys are free to believe what you choose.

    It's the facts that matter. People who won't believe in the facts are just fooling themselves.

    Just like I choose to believe that Mexican food doesn't give me heartburn.

    I have no idea why I'm taking Prilosec on a daily basis. ;)
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Cryptic devs in the past have revealed that they can't go backwards. So the map thing is probably just an extension of that overall problem.

    Heck, they probably have to start with champions elements or generic cryptic engine elements and then go from there.

    Some other quips:
    And they really need to cut down on the redundency of QA approval and other nonsense he named off that holds up the process.

    Two years of watching the Cryptic development process work and I think most of us here know that QA is minimal. It might hold up the process but it's not actually doing much assurance of quality. I think Creed at Dunder Mifflin would do better work in that same timespan. Harsh criticism, but criticism well earned through the patches turned out in both Champions and STO.
    It is inconceivable that a professional development team could exist like that and be productive.

    And that's pretty much the heart of the criticism. They do exist. But they are not productive.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    superchum wrote: »
    Two years of watching the Cryptic development process work and I think most of us here know that QA is minimal. It might hold up the process but it's not actually doing much assurance of quality. I think Creed at Dunder Mifflin would do better work in that same timespan. Harsh criticism, but criticism well earned through the patches turned out in both Champions and STO.

    I would much prefer and Apple or a Blizzard approach. Some companies have the feng shui of the service experience down.

    Back when Cryptic had public phone support, they used the Champions title screen music as their hold theme. It works for the Champions title screen but I'd rather not have phone support than listen to that on a loop for 45 minutes.

    There's a lot that could be done to streamline the service experience.

    I'd honestly prefer that they shift the ticket system to look like part of the website for the game you're using. I think there's a strategic focus that could be shifted more heavily towards buyer satisfaction. I would look at what Blizzard does and shift the C-Stores to a separate site, a Cryptic gift shop. It works wonders for WoW and other F2P games like DCUO avoid publicizing their cash shop out of game. I think the front page could serve as a better community portal. I think they could showcase fan art and fan screenshots.

    The whole flow is wonky. It's just not what Cryptic does.

    I'd love to sit down for coffee with Branflakes and doodle ideas on a notepad but that ain't happening.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Nearly every other game developer I know is actually significantly slower in it's processes than we are. One of the reasons we have our own engine is to allow in house flexibility, and the ability to walk over to the guy responsible for the tool and tell him what's wrong. Licensing our engine from someone else would not speed anything up necessarily, and in the short term, would dramatically slow things down. Adding more people might help, but hiring and team distribution are not things I have any control over. I'm sure there are inefficiencies in the system, and I'm all for working those out.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    In all honesty, Tumerboy, I believe you. I've been reading back over what has been said, and I believe that you are shooting straight with us. And I want to thank you for your clarity. The fault is not with you, but rather how the development process is laid out. The problem is that the process seems to take an "all or nothing" approach. Either an update somehoe involves every aspect of development or it doesn't happen.

    If it doesn't have new maps from scratch, new art assets, new AI mechanics, a new feature, new voice tracking, new EVERYTHING, it doesn't seem to warrant the development time.

    You do realize that you can take a galaxy class bridge bathed in normal light colors for one map, and another bathed in dark blue lighting colors, it takes on a whole different mood? All you've done is change the lighting color, and suddenly you are on the bridge of a galaxy class starship that has lost power... No art team involvement.

    This is also the impression I get. I've said it before: there's no need to make every mission or episode with new stuff, or voiceovers, or cutscenes. Solid story, using existing assets, is what should be focused on. Sure, put out Featured Episodes, with all the fancy stuff, but remember: they're featured for a reason. They're supposed to be special. All the other missions and such that ought to be released in between FEs can be normal, without all the fancy stuff.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Tumerboy wrote: »
    Nearly every other game developer I know is actually significantly slower in it's processes than we are. One of the reasons we have our own engine is to allow in house flexibility, and the ability to walk over to the guy responsible for the tool and tell him what's wrong. Licensing our engine from someone else would not speed anything up necessarily, and in the short term, would dramatically slow things down. Adding more people might help, but hiring and team distribution are not things I have any control over. I'm sure there are inefficiencies in the system, and I'm all for working those out.

    When we criticize the QA process here's what is being criticized ...

    Season 2 patch goes up on Test Server for testing.

    Players test it and note, in multiple bug reports and multiple threads on test and a lot of chatter in the test channel in-game that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    The patch goes live. This bug goes live with it.

    So then players play the new patch, and in multiple bug reports and multiple threads on General and a lot of chatter in game everywhere, players state that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    It takes weeks to fix this bug. Weeks. An MMORPG puts live a patch that breaks up the ability to team. And then takes weeks to fix said bug. A massively multiplayer online role playing game breaks teaming/grouping. The bug was allowed to go live. A bug that broke the ability to grop. Not even Sony Online Entertainment would let something like that go live.

    Excuses commonly given to bugs that revolve around the Exploration Maps and the Genesis Exploration System your company uses tend to say in one form or another that to fix these things, the devs have to go into each map and tweak them by hand.

    Oh, and here's another fun mutation of the Cryptic QA process ... FIXED BUGS LEAK BACK INTO THE GAME.

    So we've been told time and again that the devs can't go back to a previous build ... but we also see previous bugs creeping right back into the code.

    Yeah, QA ... I've not witnessed quality QA from a Cryptic game. Not in STO. Not in Champions (more on that in a sec) and not in City of Heroes.

    Other gaming companies actually strive to fix bugs like the one I cited before the patch goes live. And have a QA process set up to allow that to happen. Cryptic's QA process essentially has test server set up to stress test and to move the patch along, customer-facing and showstopping bugs be damned.

    Oh and speaking of Champions ...

    My favorite example of Cryptic's horrible ability to fix bugs ...

    Zombies in Canada.

    Took forever to fix.

    Bug was easy to find. Easy to identify. They were always there. In the same spot. Invulnerable as ever. Could replicate the bug simply by logging in. And unlike the Genesis maps, there wasn't thousands of maps that needed fixing.

    Yeah ... I'm absolutely sure that QA slows up the creation process. But I've also witnessed years of QA actually not assuring any noticeable measure of quality in the content that does get released.

    The company's needed to improve its testing procedures for years. It's no joke when players say they don't even bother to report bugs anymore because none of it gets fixed until after the content goes live.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Tumerboy wrote: »
    Nearly every other game developer I know is actually significantly slower in it's processes than we are.

    Well, then... If you guys are faster than every other game developer you know, then maybe you guys could play it full out, look at what holds things up unnecessarily and cut it out. Then you can leave everyone sniffing your dust and get a reputation as a MMO company that has proven that content can be delivered on a regular basis...
    One of the reasons we have our own engine is to allow in house flexibility, and the ability to walk over to the guy responsible for the tool and tell him what's wrong. Licensing our engine from someone else would not speed anything up necessarily, and in the short term, would dramatically slow things down.

    I can accept that. When working with third party engines and tools, you've got to file your support tickets and wait for them to have time to address it. Better to have in-house development of everything so you cut out the middle-man.

    [qyote]Adding more people might help, but hiring and team distribution are not things I have any control over.[/quote]

    So talk to those who do have control over it. All you need is one person working full time with resources and systems that already exist, even using the Foundry, and content can be churned out on a daily, or at least weekly basis to keep people coming back and to generate enough buzz that STO is a game where there is always something new to do on a regular basis. Contract them. Less headache on PWE's part.

    Pass it along.
    I'm sure there are inefficiencies in the system, and I'm all for working those out.

    Pass that along to the suits who make the decisions. Because they are the only ones who can work it out.

    Thank you for reading...
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    superchum wrote: »
    When we criticize the QA process here's what is being criticized ...

    Season 2 patch goes up on Test Server for testing.

    Players test it and note, in multiple bug reports and multiple threads on test and a lot of chatter in the test channel in-game that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    The patch goes live. This bug goes live with it.

    So then players play the new patch, and in multiple bug reports and multiple threads on General and a lot of chatter in game everywhere, players state that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    It takes weeks to fix this bug. Weeks. An MMORPG puts live a patch that breaks up the ability to team. And then takes weeks to fix said bug. A massively multiplayer online role playing game breaks teaming/grouping. The bug was allowed to go live. A bug that broke the ability to grop. Not even Sony Online Entertainment would let something like that go live.

    Excuses commonly given to bugs that revolve around the Exploration Maps and the Genesis Exploration System your company uses tend to say in one form or another that to fix these things, the devs have to go into each map and tweak them by hand.

    Oh, and here's another fun mutation of the Cryptic QA process ... FIXED BUGS LEAK BACK INTO THE GAME.

    So we've been told time and again that the devs can't go back to a previous build ... but we also see previous bugs creeping right back into the code.

    Yeah, QA ... I've not witnessed quality QA from a Cryptic game. Not in STO. Not in Champions (more on that in a sec) and not in City of Heroes.

    Other gaming companies actually strive to fix bugs like the one I cited before the patch goes live. And have a QA process set up to allow that to happen. Cryptic's QA process essentially has test server set up to stress test and to move the patch along, customer-facing and showstopping bugs be damned.

    Oh and speaking of Champions ...

    My favorite example of Cryptic's horrible ability to fix bugs ...

    Zombies in Canada.

    Took forever to fix.

    Bug was easy to find. Easy to identify. They were always there. In the same spot. Invulnerable as ever. Could replicate the bug simply by logging in. And unlike the Genesis maps, there wasn't thousands of maps that needed fixing.

    Yeah ... I'm absolutely sure that QA slows up the creation process. But I've also witnessed years of QA actually not assuring any noticeable measure of quality in the content that does get released.

    The company's needed to improve its testing procedures for years. It's no joke when players say they don't even bother to report bugs anymore because none of it gets fixed until after the content goes live.

    I really have to agree with all of the above. I work in Quality Control myself for a company. If we had a product that we put through testing and the testing showed a failure or an issue that would cause our product not to function correctly, we do not release the product until the failure can be removed or managed. That is the function of QA.

    I bet your QA is a quality assurance plan, that doesn't have any actual quality control person or people to check things out and put stuff on hold when un-foreseen issues arise. Does Cryptic have any Quality Control people? Or is it just a QA plan that has procedures people are to follow etc.?
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Tumerboy wrote: »
    Nearly every other game developer I know is actually significantly slower in it's processes than we are. One of the reasons we have our own engine is to allow in house flexibility, and the ability to walk over to the guy responsible for the tool and tell him what's wrong. Licensing our engine from someone else would not speed anything up necessarily, and in the short term, would dramatically slow things down. Adding more people might help, but hiring and team distribution are not things I have any control over. I'm sure there are inefficiencies in the system, and I'm all for working those out.

    I am absolutely 100% pleased to see you here and willing to interact with us as you are.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    superchum wrote: »
    When we criticize the QA process here's what is being criticized ...

    Season 2 patch goes up on Test Server for testing.

    Players test it and note, in multiple bug reports and multiple threads on test and a lot of chatter in the test channel in-game that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    The patch goes live. This bug goes live with it.

    So then players play the new patch, and in multiple bug reports and multiple threads on General and a lot of chatter in game everywhere, players state that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    It takes weeks to fix this bug. Weeks. An MMORPG puts live a patch that breaks up the ability to team. And then takes weeks to fix said bug. A massively multiplayer online role playing game breaks teaming/grouping. The bug was allowed to go live. A bug that broke the ability to grop. Not even Sony Online Entertainment would let something like that go live.

    Excuses commonly given to bugs that revolve around the Exploration Maps and the Genesis Exploration System your company uses tend to say in one form or another that to fix these things, the devs have to go into each map and tweak them by hand.

    Oh, and here's another fun mutation of the Cryptic QA process ... FIXED BUGS LEAK BACK INTO THE GAME.

    Which other MMOs have you played? That kind of thing happens with other MMO's all the time, and some bugs take years to fix, and others are never truely fixed, recurring exactly as you describe.

    Turnaround time in getting something basic out the door might be questionable, but anything more complex is another matter. That goes for player developed content where applicable too, if you look at player produced Neverwinter, Fallout, Oblivion, or Morrowind content.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited April 2012
    Tumerboy wrote: »
    Nearly every other game developer I know is actually significantly slower in it's processes than we are. One of the reasons we have our own engine is to allow in house flexibility, and the ability to walk over to the guy responsible for the tool and tell him what's wrong. Licensing our engine from someone else would not speed anything up necessarily, and in the short term, would dramatically slow things down. Adding more people might help, but hiring and team distribution are not things I have any control over. I'm sure there are inefficiencies in the system, and I'm all for working those out.

    I think it's (and this is a subjective thing) that I and some of the posters here don't actively value the flexibility it brings. Flexibility in one area diminishes flexibility in others.

    With a more static engine, the tech advances might be 1/8th or less what they are now but many things would be considerably more stable and folks like the content team would be focused on making do with volume over prepping for code branch merges and extra time spend on bug fixes.

    I think your approach can be seen as breakneck speed or a snail's pace depending on what the user values more...

    ... And I'm honestly a fan of a more limited engine with a focus on asset and content development as opposed to a flexible and versatile engine where the team is constantly bracing for the next collision with a load of new features.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited May 2012
    Tumerboy wrote: »
    There are some talented Foundry authors out there, no doubt.
    I still question the idea that they are making their top shelf missions in mere hours.

    However, take any of them, and throw them into a company. They now have bosses to report to. They have QA to go through. They have PR and marketing to obey.

    An individual can make every decision unilaterally. That is, they have no one else they have to answer to, and can make every decision on their own. That speeds things up a ton.

    An individual working on a team, will be getting opinions and ideas from that team. They will be collaborating. They will be improving their work based off of the feedback they receive from their fellow team members. A Designer, on a team such as ours, will have dependencies. The mission he creates, may need new art, new audio, or various other departments to be involved. That designer doesn't just need art time if they are making a whole new map. Even if they are reusing an old map, they will likely want to change up at least a few props, to make it feel slightly different. All of those dependencies take time, as they all have to be scheduled. Everyone has to go through production, to keep the schedules, and make sure everyone is on task to get the mission completed on time. That team then needs to test the mission itself, and make sure it functions, make sure it makes sense, make sure it's fun. It also has to go through QA, to further test the mission, and make sure it isn't broken in some way that the designers hadn't noticed.
    It then needs to be included to a build, and published for you all to play. And that's ignoring any Tribble time.

    Yes, coworkers and bureaucracy adds a good deal of time to the mix. That is not necessarily a bad thing. While that mission, made by one dude, might be amazing, it's also got a fair chance at not being amazing, or being broken in some way that the author did not notice. There are benefits and drawbacks to working on a team.

    So, lets take a kill 10 targs mission, and see how that might break down.
    1) Mission has been decided, kill 10 targs. Sounds like a blast!
    2) Locate an existing map that we want to use for the kill task.
    3) Save that map off, to a new name, so that it doesn't interfere with whatever is going on there normally. That's probably an hour or two, to ensure the new copy is functional, and check it in for others to use.
    4) Blow away all of the existing population, so the map can be filled with targs, probably a few hours.
    5) Repopulate with targs, and nothing else. Again, a few hours, maybe up to a day depending on how large the map is, and what you're doing with said targs. This is assuming there is nothing but plain targs to fight. No bosses, no interesting mechanics, nothing but targs.
    6) Who's going to give you this mission? Have to hook the mission dialog up to someone. I really don't know how long it takes Kestrel to write something. I'm going to assume that the text for this is pretty simple, and it's just some dude who wants you to kill targs for no particular reason. 3 hours to write, and hook up to random dude
    7) How do you get to the map? Have to hook up a door somewhere. Finding a location, and hooking up that door to point at this new copy of the map is probably a couple hours.
    8) Oh, I'm sure there were some crashes, and runaround time in there, trying to coordinate all of that between a few different people. +3 hours
    9) we need to playtest the new mission, to make sure it's functional. That's going to take a while to find some time where everyone is free for an hour or so. So while the playtest itself may only be 30 minutes, the first slot open on everyone's schedule who needs to be there, is 2 days from now. +2 days
    10) What? The playtest went perfectly? There are no bugs? That's the first time in the history of video games! Amazing!
    11) We should still send it over to QA for a bit, to make sure they didn't find anything. +2 days
    12) Wow! They didn't find anything either? This is the most perfect mission ever!
    13) Ok, it's a perfect mission, with no bugs, we'll send it over to get included in the next build. That build isn't being made until Friday though, so that's another couple days. And it won't go out until next Thursday, which is almost a week past that. + 1 week or more.

    All in all, I'd guess that the fastest a new mission could be completed, and pushed out, is a two weeks.

    If the dev tools are that clunky I'm amazed anything gets done. The City of Heroes Mission Architect system for UGT is about a thousand times better than what your describing and from what I understand, the Devs version of those same tools is even better.

    Maybe Cryptic needs to look to their own past as a guide to the future.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited May 2012
    DKeith2011 wrote:
    If the dev tools are that clunky I'm amazed anything gets done. The City of Heroes Mission Architect system for UGT is about a thousand times better than what your describing and from what I understand, the Devs version of those same tools is even better.

    Maybe Cryptic needs to look to their own past as a guide to the future.

    Really?

    Clunky?

    What Tacohead just went through was probably the proverbial "101" of any design process in multimedia. Granted, it might be bureaucratic, but there is no getting around that, the steps (however simply he might have put it) are universal in any production.

    If you have a magical development tool that has a checkbox labelled "Ignore need for supervisor to sign off my work, push to publication", I'd love to hear about it, because my job would get a hell of a lot easier. :p
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited May 2012
    superchum wrote: »
    When we criticize the QA process here's what is being criticized ...

    Season 2 patch goes up on Test Server for testing.

    Players test it and note, in multiple bug reports and multiple threads on test and a lot of chatter in the test channel in-game that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    The patch goes live. This bug goes live with it.

    So then players play the new patch, and in multiple bug reports and multiple threads on General and a lot of chatter in game everywhere, players state that Something is Terribly Wrong with the patch because it keeps putting you in separate maps in an exploration cluster when you are in a team.

    It takes weeks to fix this bug. Weeks. An MMORPG puts live a patch that breaks up the ability to team. And then takes weeks to fix said bug. A massively multiplayer online role playing game breaks teaming/grouping. The bug was allowed to go live. A bug that broke the ability to grop. Not even Sony Online Entertainment would let something like that go live.

    Excuses commonly given to bugs that revolve around the Exploration Maps and the Genesis Exploration System your company uses tend to say in one form or another that to fix these things, the devs have to go into each map and tweak them by hand.

    Oh, and here's another fun mutation of the Cryptic QA process ... FIXED BUGS LEAK BACK INTO THE GAME.

    So we've been told time and again that the devs can't go back to a previous build ... but we also see previous bugs creeping right back into the code.

    Yeah, QA ... I've not witnessed quality QA from a Cryptic game. Not in STO. Not in Champions (more on that in a sec) and not in City of Heroes.

    Other gaming companies actually strive to fix bugs like the one I cited before the patch goes live. And have a QA process set up to allow that to happen. Cryptic's QA process essentially has test server set up to stress test and to move the patch along, customer-facing and showstopping bugs be damned.

    Oh and speaking of Champions ...

    My favorite example of Cryptic's horrible ability to fix bugs ...

    Zombies in Canada.

    Took forever to fix.

    Bug was easy to find. Easy to identify. They were always there. In the same spot. Invulnerable as ever. Could replicate the bug simply by logging in. And unlike the Genesis maps, there wasn't thousands of maps that needed fixing.

    Yeah ... I'm absolutely sure that QA slows up the creation process. But I've also witnessed years of QA actually not assuring any noticeable measure of quality in the content that does get released.

    The company's needed to improve its testing procedures for years. It's no joke when players say they don't even bother to report bugs anymore because none of it gets fixed until after the content goes live.

    Agreed, Cryptic's QA blows and is FAIL, beyond any other level of fail I have ever seen from any other company, ever.
  • Options
    Archived PostArchived Post Member Posts: 2,264,498 Arc User
    edited May 2012
    Koppenflak wrote:
    Really?

    Clunky?

    What Tacohead just went through was probably the proverbial "101" of any design process in multimedia. Granted, it might be bureaucratic, but there is no getting around that, the steps (however simply he might have put it) are universal in any production.

    If you have a magical development tool that has a checkbox labelled "Ignore need for supervisor to sign off my work, push to publication", I'd love to hear about it, because my job would get a hell of a lot easier. :p

    Oversight is one thing, but taking hours or more to simply clear and replace the mobs on an existing map is insane. If the games assets are so unstable that simply copying a file and renaming it breaks that file then Cryptic has really set a new standard for poor programming.

    Mission Architect allows you to pick and map in the game, populate it with any enemy in the game (or one you create yourself), set goals and create mission text in minutes. Almost every asset in the game is available to the players and has produced some truly remarkable missions.

    As was mentioned before, Cryptic created the Mission Architect system so the Devs could create missions quickly and easily. This removed a huge time sink and streamlined the production cycle, Devs could create missions then put them up for review and QA in a fraction of the time.

    They did it for CoH, why can't they do it for STO?
Sign In or Register to comment.