I think there is room in the Foundry to provide quality content, but it all starts from a clear statement of intention by Cryptic about what is acceptable in Foundry content. If Cryptic specifically states that the creation of exploit missions in the foundry can result in suspension of foundry design privileges that is a great start.
After that, the ability to publish foundry missions should remain a privilege that is paid for via cash transaction. That will prevent a player who has been banned from publishing from simply making another free account to circumvent the ban.
Then Cryptic should provide the users with an exploit reporting flag so that the players themselves can indicate which content is exploitative. Then once enough players flag the content Cryptic can review it and remove it if it is deemed inappropriate. If the content is removed the player who published it should get a warning. After three violations their foundry publishing ability is revoked.
We also need a way to prevent what we used to call the "zero star mafia" in City of Heroes. This was a group of players that would monitor the new submissions on the CoH equivalent of the Foundry. They would immediately rate any new content that was not submitted by their group with zero stars. Of course, they would also rank their group's content with high rankings. This caused their content to consistently bubble to the top regardless of actual play value.
I have a couple of ideas on how to hamstring the "zero star mafia" in Neverwinter. First, you should not be able to rank content unless you reached a certain level or status in the game that takes a real effort to reach. This will avoid free play accounts from being used to game ratings since they would require a significant time investment on the part of the player to be able to rate Foundry content.
The game should also track the average rankings of content by each player. If they are consistently ranking content published by people on their friends list or guild very highly, and also ranking people that are not friends or guildmates very low, then their content rating ability should be revoked. All accounts that are accessing the game from the same IP addresses should also have their content rating ability suspended as well, pending further review.
I believe that if these two suggestions are implemented they would help maintain the quality and integrity of the Foundry. They are not perfect solutions and if somebody else has more foolproof ideas then I humbly bow to their suggestions.
After that, the ability to publish foundry missions should remain a privilege that is paid for via cash transaction. That will prevent a player who has been banned from publishing from simply making another free account to circumvent the ban.
If it fails again... The next version of the Foundry should be controlled by a limited number of members the company allows to use, not everyone. It's a tool and feature that you do have to put limits on. How things go some times. Be easier finding quality to.
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
You can skip ahead to the "How do you monetize UGC authors" if you don't want to watch the whole thing it is long, but a very good talk in my opinion.
I'll even go ahead and say that the Astral Diamond tip system has the potential to backfire as well. It creates a rich-get-richer feedback loop that separates the few authors in the spotlight (first few pages of the browse UI) from everyone else.
The moment you bring anything that looks like real money (and AD is, sorta) as a reward into the system, things have the potential to get ugly. Like xavrath said, there were voting cliques in COH, and they had no real incentive to mess with the ratings other than griefing and maybe removing a rival arc from the spotlight (so theirs could get there instead). When there's an actual tangible reward from having that level of exposure.... yep. Don't forget that since NW is a f2p game, anyone will be able to create their own army of voter accounts, too (which was not the case in COH - not sure about STO)... so this'll definitely need to be dealt with
edit: xavrath, good to see another COH MA-er in here! I posted some of my initial concerns about the NW foundry in light of my COH experiences here. I've been collecting some other thoughts as well, but I'm just itching for the Foundry forum to open so I can unleash it on the unsuspecting populace.
I'll even go ahead and say that the Astral Diamond tip system has the potential to backfire as well. It creates a rich-get-richer feedback loop that separates the few authors in the spotlight (first few pages of the browse UI) from everyone else.
The moment you bring anything that looks like real money (and AD is, sorta) as a reward into the system, things have the potential to get ugly. Like xavrath said, there were voting cliques in COH, and they had no real incentive to mess with the ratings other than griefing and maybe removing a rival arc from the spotlight (so theirs could get there instead). When there's an actual tangible reward from having that level of exposure.... yep. Don't forget that since NW is a f2p game, anyone will be able to create their own army of voter accounts, too (which was not the case in COH - not sure about STO)... so this'll definitely need to be dealt with
edit: xavrath, good to see another COH MA-er in here! I posted some of my initial concerns about the NW foundry in light of my COH experiences here. I've been collecting some other thoughts as well, but I'm just itching for the Foundry forum to open so I can unleash it on the unsuspecting populace.
It won't really matter as much because the "featured" UGC that will be shown on the window that pops up when you enter the game (can't remember what they called it) is decided by Cryptic not the community. So the adventures with the most exposure (the featured ones) will not be decided by voting cliques.
Seriously? You feel we should have to pay to create content?
I'll have some of whatever you are smoking.
Yeah count me in as thinking it's a bad idea too. Mostly because it punishes publication, which for role players is very destructive. We will be taking down old mission and putting up new missions at least once a week, maybe more. We're building a dynamic, interactive campaign, serialized adventures where "what happens next" is based on what choices the characters in our guild made.
So we'll be constantly pushing content down to make way for new, updated content.
Our methodology is a lot of fun, but would be crucified under any scheme that required cash every time you wanted to publish something.
It sounds elitist, but we're not making content to get 100 "A" ratings and a few thousand AD in tips. We're making content to augment a RP campaign. We're catering to specific players and story lines that we make and enhance based on your choices. Our stuff is quality (I like to think so anyway), but it is catered to a very specific audience.
SHADOW - A secret cabal for those who thirst for wealth and power. Check out SHADOW on YouTube!
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
It won't really matter as much because the "featured" UGC that will be shown on the window that pops up when you enter the game (can't remember what they called it) is decided by Cryptic not the community. So the adventures with the most exposure (the featured ones) will not be decided by voting cliques.
I know -- they mentioned this in the video too.
My problem with this is, again, scaling. A small live team selecting a handful of content ever so often is a drop in the bucket with a (probably) fairly narrow view on what is "good" UGC -- not to mention it disincentivizes people from searching for non-highlighted content, which reduces exposure for all the non-highlighted people even further. Why bother looking beyond the first few pages when I'm getting 10 recommendations from the live team every week? You could possibly make an argument that it also decreases the value of unofficial (community) reviews, since the typical player is going to trust the paid live team a lot more than a bunch of amateurs, no matter how spirited they may be. The typical player also doesn't read forums and 3rd-party sites to begin with, but that's a topic for another day...
Against exploiters, sure, but I will say that my rating of UGC is very likely to be strongly correlated with the XP and loot efficiency of the quest, simply because I'm here for the combat, and that's the primary mechanism for getting XP/loot from a quest, by my understanding.
Things that I find tedious, like deep dialog trees and long-distance journeys, will get a low rating from me. Unless, of course, such things are called out in the quest description, in which case I'd know to steer clear.
And THIS is exactly why the current system of rating Foundry content completely and utterly SUCKS, and your post should be a huge red flag to Cryptic that it needs an overhaul.
This works both ways, players looking for a deep dialogue tree and story with less combat will likely give the combat oriented quests, even if they are well designed, a low score because it's not what they like.
The simple Star rating is far to ambiguous, as you never know from where the reviewer is coming from.
And unfortunately, most players are just looking for that fast and easy XP and loot quest, even through the Foundry was really designed to offer far more. And so, the quests that utilize the full potential of the Foundry get bad ratings because they are not "XP and loot efficiency" which frankly is NOT what the Foundry is for at all.
Don't know where the huge disconnect happened where players went from demanding more content to their MMO's, and really just want faster XP and loot gains with less story (and of course complain that the game was "too short")
I am not very optimistic unlike people here - exploiters outnumber us one to many.
Current exploiting system which needs a fix is defanging of monsters. In STO they make half walls to stop attacks of enemy ships so you kill them without facing any danger of getting hit.
Thus by destroying enemies or "defanged" enemies, you get free XP.
In NW it can be used as using melee enemies and making them at unreachable position and then kill them with ranged attacks.
This I think is a simple fix. While yes trying to limit the ability to exploiters to even do this, I think the real thing that needs to be done is actual disciplinary actions for both the Authors and repeat offenders.
Personally, if anyone goes into a quest, see that the mobs are all melee and can't attack you, KNOWS it is not right. They can BS all they want about how they did not realize it was broken, but unless your a total and legitimate idiot, you know they know better.
I think if a quest get brought up to the GM's as a possible exploiter quest, they should, upon verifying that it is, not only give the author a temp(or permanent ban for repeat offenses) from using the Foundry, but also check to see what players made repeated play throughs of the quest. It is one thing to play it once, think it's bugged, go back to verify. But to run it 3 or more times in rapid succession tells me that they knew EXACTLY what they were doing. In these cases they need to also get a warning then a temp ban from playing Foundry content.
The reason there are so many exploiters, is because frankly, most companies these days lack the will to actually hold players ACCOUNTABLE for their actions. People see that there is rarely any reason NOT to exploit, so they do.
I figured I'd post this^ since its relevant to this discussion and there's a lot of speculation going on.
Its kind of a long presentation about UGS in general, but it explains a lot about what kinds of measures the devs have taken into account to determine UGC rewards in NWO. The actual UGC Rewards part starts around 22 mins in (you can jump right into that section by clicking from the list on the side), but they also cover a lot of background details at the begining, reviewing UGC systems leading up to NWO, which may give a hint about the dev's thought process going into this.
He neglected that a lot of those "only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone.
I would really only worry so much about this, if Cyptic has an issue with people producing lame grind missions then I'm all for leaving it up to them to work it out. Some folks see the only point in playing an MMO as grinding gear and being "better", for them some will make the "kill 10 rats in the cages, collect gold and xp" quests. For those who don't want this content, the other stuff will be created. So long as there's a personal sorting system with a search function that understands whether or not you want the "free xp" missions, I'm fine. I just hope the foundry ends up a legitimate way for players to design stories and that Cryptic doesn't make the gear ladder endless in an effort to appease the other group.
Generally the worst thing about MMOs is always the player base.
0
sebukaiMember, Neverwinter Beta Users, Neverwinter Guardian UsersPosts: 15Arc User
edited March 2013
I agree with OP. I don't want people to game the system, as contradictory as that may seem.
He neglected that a lot of those "only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone.
Umm he did not say those 200 missions where the highest rated. He said 200 mission that where as good as missions Cryptic made. Rating never came into the statement it was about the quality of the work.
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
Umm he did not say those 200 missions where the highest rated. He said 200 mission that where as good as missions Cryptic made. Rating never came into the statement it was about the quality of the work.
The "... the good content" slide says that "only about 200 received 'the highest rating'". He does not define what "highest rating" is (if you really wanted to stretch it, it could be interpreted to mean something other than the 5-star system).... but immediately after that he segues into talking about the rating system.
The "Identify the Good Content" slide basically says explicitly that the 5-star rating system is what identifies the good content, and it works "really really well to get good content to filter up" (at 34:30 or so). He does talk about the highlighting specific arcs in "Feature the Good content" right after that, but that's a different beast. I don't want to watch the entire talk again, but in the QA section the speaker explicitly said that good missions "bubble to the top" in the context of the rating system at one point, too. Also, there's a guy who asks him how many people use the search function, and he replies that not many do at all, and most just look at whatever is highlighted in front.
BTW, assuming that the 200 really refers to '200 missions the community and our Live Team highlighted as the best content of the bunch', how do they know that the 200 are really the best? Have they played every single mission? Like I posted on the previous page, they seem to think hand-picking a miniscule amount of content (according to the tastes of the live team) is going to solve the problem of getting stories played (and more importantly getting the content to the players according to the needs/tastes of the players). It won't.
The "... the good content" slide says that "only about 200 received 'the highest rating'". He does not define what "highest rating" is (if you really wanted to stretch it, it could be interpreted to mean something other than the 5-star system).... but immediately after that he segues into talking about the rating system.
The "Identify the Good Content" slide basically says explicitly that the 5-star rating system is what identifies the good content.
ignore what the slide says, listen to 33:48 - 33:58 where he says 200 as good as the content his dev team created. He didn't say "with a 5 star rating".
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
ignore what the slide says, listen to 33:48 - 33:58 where he says 200 as good as the content his dev team created. He didn't say "with a 5 star rating".
Why ignore the slide?
Especially since on the next slide he goes into a spiel about how rating is everything (both on the slide and in his talk) in the context of "how do you find this good content"... right after the part you mentioned, at 34:07 he segues into the next slide saying "how do you find that good content - how do you get them to your players? This is really simple, very straightforward: you let your community identify that content. You put in a rating system. COH, STO, NW all have a community-generated rating system, a 5-star system...".
Anyway, summary: relying on a 5-star rating system doesn't work. Relying on a small number (200 is small) of hand-picked missions as the divine word of cthulhu for classifying content as "wheat" or "chaff" doesn't work, for different reasons.
lanessar13Member, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 8Arc User
edited March 2013
I know it's fashionable to hate on Foundry-created content for STO, and say that it's filled with garbage. I just started playing STO last month for something to do before NWO, and also to understand why the "pay to win" hate comes from this game on an in-depth level. I have to say, there is some validity to the claim, but it's not 100% valid - the best stuff can be gotten without paying. I digress, back to STO foundry content - the rewards for Foundry are not as good as episodic content, but they are not terrible, either. Comparing to the NWO Foundry quests, I'd say they improved and made it more rewarding to do the more fleshed-out foundry content.
However, of the "featured content" Foundry missions (everything I run into just digging about when playing STO), everything I've played has about an hour's length, does in fact have a decent story. While it is hit or miss as being "as good as developer's content", even the "worst" was not that bad. To get to the "terrible" missions, you have to sort of dig into the lower-starred content. Seems the players are pretty good at rating things well, and balancing out the hater ratings. Even mediocre missions had a lot of four star ratings, so it wasn't like the haters (even on an RP-centric mission) weighed the vote much - about 110 "haters" with 59,000 rating the mission 4 stars or more.
Basically, depending on how many people play your content, it tends to balance out. I have a feeling that (since STO is missing this), the tag system that they talk about will come in far more handy for identifying content some people will want to play - for example, if I like dungeon delves, tagging the content with that will help me locate that.
I think the speaker saying the search function isn't used is not viewing things in the correct context. The search feature for STO content does not categorize by "tag", so no one is able to use it. Searching "romulan", for example will return junk results, making the search feature useless, therefore the tool isn't used. I foresee that with the tag system, NWO search will get far more use than STO. So, flawed premise (apples and oranges, honestly). As an analogy, it's like googling "refrigerator" when you want to buy a new fridge, and getting sporting results returned because one of the teams have a guy named "the Fridge" on it. If that happened in the first page of results, no one would be using google much, either. with this flaw, you could state "no one wants to have a search engine", until it is done right.
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
I know it's fashionable to hate on Foundry-created content for STO, and say that it's filled with garbage. I just started playing STO last month for something to do before NWO, and also to understand why the "pay to win" hate comes from this game on an in-depth level. I have to say, there is some validity to the claim, but it's not 100% valid - the best stuff can be gotten without paying. I digress, back to STO foundry content - the rewards for Foundry are not as good as episodic content, but they are not terrible, either. Comparing to the NWO Foundry quests, I'd say they improved and made it more rewarding to do the more fleshed-out foundry content.
(snip)
I actually didn't play STO at all (I have some friends who did/do, though), but I have a lot of experience (3+ years) with the City of Heroes Mission Architect, which was the prototype for the Foundry. Both of those systems have similar rating systems (STO is a bit improved by having comments be visible to everyone instead of just the author). The problem isn't actually haters / voter cliques as much as the entire rating system basically serving as momentum -- content created near the start of the game will be played by almost everyone, while new authors will struggle to even get one person to play their creations (even if you have a 5-star average and 2 plays, you'll still be on page 40+). This only gets worse with time...
Hand-picking 'highlighted' content is a bit better in concept than COH's dev choice system (which basically kept your story arc on page 1/2 permanently), but suffers from the same problems of scale and subjectivity. How is the team selecting the highlighted stories - based on suggestions from the forums, randomly searching for stories, or something else? How do they decide whether the content they play sinks or swims? Some players think RA Salvatore is the best writer ever, others think he's a hack. Some players like story content, others (a LOT of 'others') like farming mobs. Some people like to read a lot of text, others like it when the action is fast and furious. These are all valid playstyles.
The problem with thinking rating systems or hand-picked spotlights are the end-all solution is that they really aren't.
NB: the reason I'm posting so much about this is that this was a really serious problem in COH, and probably the single most discussed 'meta-issue' in the MA forums for years. There are known problems with browsing/searching/rating in MMO UGC with a large amount of content (say, the 300 thousand story arcs in COH by month 4), and proposed solutions to those problems. Ultimately, I want NW to at least try to address those problems instead of pretending they don't exist... for this particular problem I believe the only satisfying answer is "improve the tools so that players can find content THEY personally would like". And yes, the tag system is absolutely crucial for this, but it does not solve problems just by existing. COH had a fairly in-depth tag system and it didn't help -- because it was not possible to tag story arcs with what people were actually looking for ('farm', 'story', etc).
btw, I do have a txt file with about 38k of text on this subject sitting on my HD just waiting for the Foundry forums to open. This is probably the only thing wrt Neverwinter that I feel really passionate about, after all.
apocrs1980Member, Neverwinter Beta Users, SilverstarsPosts: 0Arc User
edited March 2013
I've cut my finger and dripped blood to swear upon creating the best content for everyone's enjoyment. I will also seek and destroy any exploitative content I find while playing.
[SIGPIC][/SIGPIC]
The Cragsteep Crypt - BETA Ravenloft Look for@Apocrs1980 or visit the main page here or Ravenloft here
0
lanessar13Member, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 8Arc User
...The problem isn't actually haters / voter cliques as much as the entire rating system basically serving as momentum -- content created near the start of the game will be played by almost everyone, while new authors will struggle to even get one person to play their creations ....
Hand-picking 'highlighted' content is a bit better in concept than COH's dev choice system (which basically kept your story arc on page 1/2 permanently), but suffers from the same problems of scale and subjectivity. How is the team selecting the highlighted stories - based on suggestions from the forums, randomly searching for stories, or something else?
Honestly, I'm not sure it's as much of a problem with STO - featured content as you'd think. I'll have to play more foundry content missions to see how this plays out, and see if I can find newer stuff. There are also better systems for the Foundry content in NWO. You can get Foundry missions by talking to local bartenders (if they start near there), Harper agents, a "Job board", etc.
Obviously, to get more people to play your mission, simply have it start near a starter/central hub area, then have an area trans at the gate - I remember several that were hard to get to because of confusing quest start areas in Beta. I'm sure authors can be bright and leverage that system to get "quick recruits", or feedback will change the system I observed.
Until a dev actually comments on the logic behind it, it's hard to recommend changes to the logic - we have some good suggestions here, but it should be tempered with the existing system. There are a number of people going onto the beta for the Foundry "soon", so I'm hoping those Foundry people are also examining the spotlight parameters.
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
... There are also better systems for the Foundry content in NWO. You can get Foundry missions by talking to local bartenders (if they start near there), Harper agents, a "Job board", etc.
Obviously, to get more people to play your mission, simply have it start near a starter/central hub area, then have an area trans at the gate - I remember several that were hard to get to because of confusing quest start areas in Beta. I'm sure authors can be bright and leverage that system to get "quick recruits", or feedback will change the system I observed.
...
Yes, those systems can work to improve the author's chances... but IMO everyone will just put their quests in the biggest population center they can find for more exposure, and then the player is looking at a 10000-item quest list when clicking on the job board / harper agent / etc. If anything, it'll just help the "flavor" of the missions, which is of course a good thing -- but doesn't solve the problem.
BTW, I agree that whatever improvements are proposed, they must remain within the framework of the current system. Saying something like "oh yeah, just implement an expert system to give unerring content suggestions to all players" or "implement a smart ranking system on par with google pagerank" is out of the question. I did post a few plausible suggestions here (and I don't claim credit for them -- they are all based on ideas that were discussed and debated in the COH MA community).
BTW, I agree that whatever improvements are proposed, they must remain within the framework of the current system. Saying something like "oh yeah, just implement an expert system to give unerring content suggestions to all players" or "implement a smart ranking system on par with google pagerank" is out of the question. I did post a few plausible suggestions ...
That's a good idea. I'll post over to that thread as well, but honestly, I think learning from the mistakes of yesteryear help. With a star rating system, we should probably contact Maximus over at the NWVault. Maybe it's a "trade secret", but they did several modifications to the ranking system which discounted low votes if n=Y were over a certain rating.
*Sighs* Again, it's hard to make suggestions not knowing the current system. I hope the devs make this a bit more transparent, not so that exploiters can figure out how to downvote something, but that the testers can suggest better ways to prevent downvoting from taking a toll.
As I recall, even stuff on the NW Vault got spotlit if it had few votes, but the general ratings were very high. This actually had me trying a LOT of "modules" in NWN/NWN2 which I would have otherwise overlooked.
Especially since on the next slide he goes into a spiel about how rating is everything (both on the slide and in his talk) in the context of "how do you find this good content"... right after the part you mentioned, at 34:07 he segues into the next slide saying "how do you find that good content - how do you get them to your players? This is really simple, very straightforward: you let your community identify that content. You put in a rating system. COH, STO, NW all have a community-generated rating system, a 5-star system...".
Anyway, summary: relying on a 5-star rating system doesn't work. Relying on a small number (200 is small) of hand-picked missions as the divine word of cthulhu for classifying content as "wheat" or "chaff" doesn't work, for different reasons.
It doesn't matter. I am not disputing that there might be work needed on the rating system and how to find content. I am disputing your statement "He neglected that a lot of those "only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone." which is clearly not true. You can keep throwing up diversions to the statement and trying to cover the previous seemingly blatant misdirection of the truth with other statements, but it does not change the blatant misrepresentation of the information, when he clearly states "200 quests, that is 200 hours not created by my dev team that is as good as the stuff they created" So your statement of "He neglected that a lot of those only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone." is clearly bull****. Weather you meant to try to mislead people or just didn't listen and understand the information being imparted I can't say. However regardless of which is true, your statement is a crock.
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
It doesn't matter. I am not disputing that there might be work needed on the rating system and how to find content. I am disputing your statement "He neglected that a lot of those "only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone." which is clearly not true. You can keep throwing up diversions to the statement and trying to cover the previous seemingly blatant misdirection of the truth with other statements, but it does not change the blatant misrepresentation of the information, when he clearly states "200 quests, that is 200 hours not created by my dev team that is as good as the stuff they created" So your statement of "He neglected that a lot of those only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone." is clearly bull****. Weather you meant to try to mislead people or just didn't listen and understand the information being imparted I can't say. However regardless of which is true, your statement is a crock.
Yeah, except I didn't post that at all; that was another poster. Go back and look.
Thanks for the completely unnecessary attack, though -- instead of actually trying to discuss the points in my posts, this being a discussion thread and all. :rolleyes:
Yeah, except I didn't post that at all; that was another poster. Go back and look.
Thanks for the completely unnecessary attack, though -- instead of actually trying to discuss the points in my posts, this being a discussion thread and all. :rolleyes:
Man I shouldn't post when I am working Sorry. I don't disagree with the other things in your post. Well most of them, but I can't offer any real opinion til I see it in action and actually have experience with it.
0
zaphtasticMember, Neverwinter Beta Users, Neverwinter Guardian Users, SilverstarsPosts: 0Arc User
Man I shouldn't post when I am working Sorry. I don't disagree with the other things in your post. Well most of them, but I can't offer any real opinion til I see it in action and actually have experience with it.
'sok, we all make mistakes ... well, I know I do, anyway.
----
That's a good idea. I'll post over to that thread as well, but honestly, I think learning from the mistakes of yesteryear help. With a star rating system, we should probably contact Maximus over at the NWVault. Maybe it's a "trade secret", but they did several modifications to the ranking system which discounted low votes if n=Y were over a certain rating.
*Sighs* Again, it's hard to make suggestions not knowing the current system. I hope the devs make this a bit more transparent, not so that exploiters can figure out how to downvote something, but that the testers can suggest better ways to prevent downvoting from taking a toll.
As I recall, even stuff on the NW Vault got spotlit if it had few votes, but the general ratings were very high. This actually had me trying a LOT of "modules" in NWN/NWN2 which I would have otherwise overlooked.
I agree that taking cues from other communities is a good idea. However, keep in mind that one of the main underlying issues here is the symmetric/asymmetric nature of content in UGC.
NWN and other mod communities are asymmetric -- there is a huge pool of players, and a really small pool of content creators. There's a high barrier-of-entry, making a new adventure takes a lot of effort, and while a lot of people may make mini-adventures to mess around with the editor, most don't release it. As I understand it, content is also released at a fairly slow pace that reviewers can keep up with, so it is really good for reviewer-based quality control. (out of curiosity, how many NWN2 adventures are released in a day, and how many have been released to this date?)
MMO UGC is more symmetric, especially for games where customization is king (COH) and - I speculate - where the Foundry is touted as a main defining feature of the game (NW). Toolkits are accessible, and making new content is easy. At least in COH there was an absolute deluge of content created by people messing around, or trying out how the MA works, or making their own private areas --none of which are really enjoyable for others. In COH, there were over 300000 story arcs (a story arc was a set of 1-5 single missions) by month 4, near the end of the MA's Golden Age. By that point, there were a total of 521 story arc reviews in the ENTIRE community consisting of multiple forums and fansites (I have numbers to support this, I just copy/pasted this from that 38k txt file I mentioned ), which is 0.17% of the total available content.
My point is basically that while reviews and spotlights are great for the author community (and for players too, somewhat), the only solution that can keep up with the volume of content being created is to drive the system by the only scalable part of the equation, which is the playerbase... and the rating system is a big part of it. It is just that the rating system, as it is now, has quite a few problems on its own when it comes to "letting good content bubble up to the top".
starkaosMember, Neverwinter Beta Users, Neverwinter Hero Users, Neverwinter Knight of the Feywild UsersPosts: 0Arc User
edited March 2013
Players are going to exploit if the system allows it. As long as the Neverwinter devs learn from the mistakes of Star Trek Online such as amount of astral diamonds rewarded based on average time players take to complete mission, then it is not a concern.
0
kerlaaMember, Neverwinter Beta Users, Neverwinter Guardian UsersPosts: 0Arc User
Can we make a pact as a community to put a stop to this nonsense and swear a blood oath to not let exploiters and the folks looking to game the system an avenue. I know these type of Foundry missions will be there but because we can vote on it lets not give them any more incentive to ruin what is arguably going to be the greatest new MMO feature in years.
I long for the day when the majority of the Foundry missions are designed with the fan in mind to give meaningful content. I just hope there are plenty of DM's who are willing to design and implment some of the best content the genre has ever seen. Hoping theres a strong presence of the NWN content designers here to help push this game to its full potential.
I agree to this wholeheartedly! Even though my quests and campaigns will likely be average at best I will try my best to give the players non-exploited content.
There is a rumor floating around that I am working on a new foundry quest. It was started by me.
Comments
After that, the ability to publish foundry missions should remain a privilege that is paid for via cash transaction. That will prevent a player who has been banned from publishing from simply making another free account to circumvent the ban.
Then Cryptic should provide the users with an exploit reporting flag so that the players themselves can indicate which content is exploitative. Then once enough players flag the content Cryptic can review it and remove it if it is deemed inappropriate. If the content is removed the player who published it should get a warning. After three violations their foundry publishing ability is revoked.
We also need a way to prevent what we used to call the "zero star mafia" in City of Heroes. This was a group of players that would monitor the new submissions on the CoH equivalent of the Foundry. They would immediately rate any new content that was not submitted by their group with zero stars. Of course, they would also rank their group's content with high rankings. This caused their content to consistently bubble to the top regardless of actual play value.
I have a couple of ideas on how to hamstring the "zero star mafia" in Neverwinter. First, you should not be able to rank content unless you reached a certain level or status in the game that takes a real effort to reach. This will avoid free play accounts from being used to game ratings since they would require a significant time investment on the part of the player to be able to rate Foundry content.
The game should also track the average rankings of content by each player. If they are consistently ranking content published by people on their friends list or guild very highly, and also ranking people that are not friends or guildmates very low, then their content rating ability should be revoked. All accounts that are accessing the game from the same IP addresses should also have their content rating ability suspended as well, pending further review.
I believe that if these two suggestions are implemented they would help maintain the quality and integrity of the Foundry. They are not perfect solutions and if somebody else has more foolproof ideas then I humbly bow to their suggestions.
Not going to happen. Watch this video for why.
http://www.gdcvault.com/play/1016471/User-Generated-Content-In-MMOs
You can skip ahead to the "How do you monetize UGC authors" if you don't want to watch the whole thing it is long, but a very good talk in my opinion.
If it fails again... The next version of the Foundry should be controlled by a limited number of members the company allows to use, not everyone. It's a tool and feature that you do have to put limits on. How things go some times. Be easier finding quality to.
The moment you bring anything that looks like real money (and AD is, sorta) as a reward into the system, things have the potential to get ugly. Like xavrath said, there were voting cliques in COH, and they had no real incentive to mess with the ratings other than griefing and maybe removing a rival arc from the spotlight (so theirs could get there instead). When there's an actual tangible reward from having that level of exposure.... yep. Don't forget that since NW is a f2p game, anyone will be able to create their own army of voter accounts, too (which was not the case in COH - not sure about STO)... so this'll definitely need to be dealt with
edit: xavrath, good to see another COH MA-er in here! I posted some of my initial concerns about the NW foundry in light of my COH experiences here. I've been collecting some other thoughts as well, but I'm just itching for the Foundry forum to open so I can unleash it on the unsuspecting populace.
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
Seriously? You feel we should have to pay to create content?
I'll have some of whatever you are smoking.
It won't really matter as much because the "featured" UGC that will be shown on the window that pops up when you enter the game (can't remember what they called it) is decided by Cryptic not the community. So the adventures with the most exposure (the featured ones) will not be decided by voting cliques.
Yeah count me in as thinking it's a bad idea too. Mostly because it punishes publication, which for role players is very destructive. We will be taking down old mission and putting up new missions at least once a week, maybe more. We're building a dynamic, interactive campaign, serialized adventures where "what happens next" is based on what choices the characters in our guild made.
So we'll be constantly pushing content down to make way for new, updated content.
Our methodology is a lot of fun, but would be crucified under any scheme that required cash every time you wanted to publish something.
It sounds elitist, but we're not making content to get 100 "A" ratings and a few thousand AD in tips. We're making content to augment a RP campaign. We're catering to specific players and story lines that we make and enhance based on your choices. Our stuff is quality (I like to think so anyway), but it is catered to a very specific audience.
Check out SHADOW on YouTube!
My problem with this is, again, scaling. A small live team selecting a handful of content ever so often is a drop in the bucket with a (probably) fairly narrow view on what is "good" UGC -- not to mention it disincentivizes people from searching for non-highlighted content, which reduces exposure for all the non-highlighted people even further. Why bother looking beyond the first few pages when I'm getting 10 recommendations from the live team every week? You could possibly make an argument that it also decreases the value of unofficial (community) reviews, since the typical player is going to trust the paid live team a lot more than a bunch of amateurs, no matter how spirited they may be. The typical player also doesn't read forums and 3rd-party sites to begin with, but that's a topic for another day...
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
And THIS is exactly why the current system of rating Foundry content completely and utterly SUCKS, and your post should be a huge red flag to Cryptic that it needs an overhaul.
This works both ways, players looking for a deep dialogue tree and story with less combat will likely give the combat oriented quests, even if they are well designed, a low score because it's not what they like.
The simple Star rating is far to ambiguous, as you never know from where the reviewer is coming from.
And unfortunately, most players are just looking for that fast and easy XP and loot quest, even through the Foundry was really designed to offer far more. And so, the quests that utilize the full potential of the Foundry get bad ratings because they are not "XP and loot efficiency" which frankly is NOT what the Foundry is for at all.
Don't know where the huge disconnect happened where players went from demanding more content to their MMO's, and really just want faster XP and loot gains with less story (and of course complain that the game was "too short")
This I think is a simple fix. While yes trying to limit the ability to exploiters to even do this, I think the real thing that needs to be done is actual disciplinary actions for both the Authors and repeat offenders.
Personally, if anyone goes into a quest, see that the mobs are all melee and can't attack you, KNOWS it is not right. They can BS all they want about how they did not realize it was broken, but unless your a total and legitimate idiot, you know they know better.
I think if a quest get brought up to the GM's as a possible exploiter quest, they should, upon verifying that it is, not only give the author a temp(or permanent ban for repeat offenses) from using the Foundry, but also check to see what players made repeated play throughs of the quest. It is one thing to play it once, think it's bugged, go back to verify. But to run it 3 or more times in rapid succession tells me that they knew EXACTLY what they were doing. In these cases they need to also get a warning then a temp ban from playing Foundry content.
The reason there are so many exploiters, is because frankly, most companies these days lack the will to actually hold players ACCOUNTABLE for their actions. People see that there is rarely any reason NOT to exploit, so they do.
He neglected that a lot of those "only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone.
Generally the worst thing about MMOs is always the player base.
Umm he did not say those 200 missions where the highest rated. He said 200 mission that where as good as missions Cryptic made. Rating never came into the statement it was about the quality of the work.
The "Identify the Good Content" slide basically says explicitly that the 5-star rating system is what identifies the good content, and it works "really really well to get good content to filter up" (at 34:30 or so). He does talk about the highlighting specific arcs in "Feature the Good content" right after that, but that's a different beast. I don't want to watch the entire talk again, but in the QA section the speaker explicitly said that good missions "bubble to the top" in the context of the rating system at one point, too. Also, there's a guy who asks him how many people use the search function, and he replies that not many do at all, and most just look at whatever is highlighted in front.
BTW, assuming that the 200 really refers to '200 missions the community and our Live Team highlighted as the best content of the bunch', how do they know that the 200 are really the best? Have they played every single mission? Like I posted on the previous page, they seem to think hand-picking a miniscule amount of content (according to the tastes of the live team) is going to solve the problem of getting stories played (and more importantly getting the content to the players according to the needs/tastes of the players). It won't.
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
ignore what the slide says, listen to 33:48 - 33:58 where he says 200 as good as the content his dev team created. He didn't say "with a 5 star rating".
Especially since on the next slide he goes into a spiel about how rating is everything (both on the slide and in his talk) in the context of "how do you find this good content"... right after the part you mentioned, at 34:07 he segues into the next slide saying "how do you find that good content - how do you get them to your players? This is really simple, very straightforward: you let your community identify that content. You put in a rating system. COH, STO, NW all have a community-generated rating system, a 5-star system...".
Anyway, summary: relying on a 5-star rating system doesn't work. Relying on a small number (200 is small) of hand-picked missions as the divine word of cthulhu for classifying content as "wheat" or "chaff" doesn't work, for different reasons.
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
However, of the "featured content" Foundry missions (everything I run into just digging about when playing STO), everything I've played has about an hour's length, does in fact have a decent story. While it is hit or miss as being "as good as developer's content", even the "worst" was not that bad. To get to the "terrible" missions, you have to sort of dig into the lower-starred content. Seems the players are pretty good at rating things well, and balancing out the hater ratings. Even mediocre missions had a lot of four star ratings, so it wasn't like the haters (even on an RP-centric mission) weighed the vote much - about 110 "haters" with 59,000 rating the mission 4 stars or more.
Basically, depending on how many people play your content, it tends to balance out. I have a feeling that (since STO is missing this), the tag system that they talk about will come in far more handy for identifying content some people will want to play - for example, if I like dungeon delves, tagging the content with that will help me locate that.
I think the speaker saying the search function isn't used is not viewing things in the correct context. The search feature for STO content does not categorize by "tag", so no one is able to use it. Searching "romulan", for example will return junk results, making the search feature useless, therefore the tool isn't used. I foresee that with the tag system, NWO search will get far more use than STO. So, flawed premise (apples and oranges, honestly). As an analogy, it's like googling "refrigerator" when you want to buy a new fridge, and getting sporting results returned because one of the teams have a guy named "the Fridge" on it. If that happened in the first page of results, no one would be using google much, either. with this flaw, you could state "no one wants to have a search engine", until it is done right.
Hand-picking 'highlighted' content is a bit better in concept than COH's dev choice system (which basically kept your story arc on page 1/2 permanently), but suffers from the same problems of scale and subjectivity. How is the team selecting the highlighted stories - based on suggestions from the forums, randomly searching for stories, or something else? How do they decide whether the content they play sinks or swims? Some players think RA Salvatore is the best writer ever, others think he's a hack. Some players like story content, others (a LOT of 'others') like farming mobs. Some people like to read a lot of text, others like it when the action is fast and furious. These are all valid playstyles.
The problem with thinking rating systems or hand-picked spotlights are the end-all solution is that they really aren't.
NB: the reason I'm posting so much about this is that this was a really serious problem in COH, and probably the single most discussed 'meta-issue' in the MA forums for years. There are known problems with browsing/searching/rating in MMO UGC with a large amount of content (say, the 300 thousand story arcs in COH by month 4), and proposed solutions to those problems. Ultimately, I want NW to at least try to address those problems instead of pretending they don't exist... for this particular problem I believe the only satisfying answer is "improve the tools so that players can find content THEY personally would like". And yes, the tag system is absolutely crucial for this, but it does not solve problems just by existing. COH had a fairly in-depth tag system and it didn't help -- because it was not possible to tag story arcs with what people were actually looking for ('farm', 'story', etc).
btw, I do have a txt file with about 38k of text on this subject sitting on my HD just waiting for the Foundry forums to open. This is probably the only thing wrt Neverwinter that I feel really passionate about, after all.
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
Ravenloft
Look for@Apocrs1980 or visit the main page here or Ravenloft here
Honestly, I'm not sure it's as much of a problem with STO - featured content as you'd think. I'll have to play more foundry content missions to see how this plays out, and see if I can find newer stuff. There are also better systems for the Foundry content in NWO. You can get Foundry missions by talking to local bartenders (if they start near there), Harper agents, a "Job board", etc.
Obviously, to get more people to play your mission, simply have it start near a starter/central hub area, then have an area trans at the gate - I remember several that were hard to get to because of confusing quest start areas in Beta. I'm sure authors can be bright and leverage that system to get "quick recruits", or feedback will change the system I observed.
Until a dev actually comments on the logic behind it, it's hard to recommend changes to the logic - we have some good suggestions here, but it should be tempered with the existing system. There are a number of people going onto the beta for the Foundry "soon", so I'm hoping those Foundry people are also examining the spotlight parameters.
BTW, I agree that whatever improvements are proposed, they must remain within the framework of the current system. Saying something like "oh yeah, just implement an expert system to give unerring content suggestions to all players" or "implement a smart ranking system on par with google pagerank" is out of the question. I did post a few plausible suggestions here (and I don't claim credit for them -- they are all based on ideas that were discussed and debated in the COH MA community).
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
That's a good idea. I'll post over to that thread as well, but honestly, I think learning from the mistakes of yesteryear help. With a star rating system, we should probably contact Maximus over at the NWVault. Maybe it's a "trade secret", but they did several modifications to the ranking system which discounted low votes if n=Y were over a certain rating.
*Sighs* Again, it's hard to make suggestions not knowing the current system. I hope the devs make this a bit more transparent, not so that exploiters can figure out how to downvote something, but that the testers can suggest better ways to prevent downvoting from taking a toll.
As I recall, even stuff on the NW Vault got spotlit if it had few votes, but the general ratings were very high. This actually had me trying a LOT of "modules" in NWN/NWN2 which I would have otherwise overlooked.
It doesn't matter. I am not disputing that there might be work needed on the rating system and how to find content. I am disputing your statement "He neglected that a lot of those "only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone." which is clearly not true. You can keep throwing up diversions to the statement and trying to cover the previous seemingly blatant misdirection of the truth with other statements, but it does not change the blatant misrepresentation of the information, when he clearly states "200 quests, that is 200 hours not created by my dev team that is as good as the stuff they created" So your statement of "He neglected that a lot of those only 200 missions with highest ratings" are really just farmable quests with a skeleton story and are far INFERIOR to the developer created content in everyone." is clearly bull****. Weather you meant to try to mislead people or just didn't listen and understand the information being imparted I can't say. However regardless of which is true, your statement is a crock.
Thanks for the completely unnecessary attack, though -- instead of actually trying to discuss the points in my posts, this being a discussion thread and all. :rolleyes:
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
Man I shouldn't post when I am working
---- I agree that taking cues from other communities is a good idea. However, keep in mind that one of the main underlying issues here is the symmetric/asymmetric nature of content in UGC.
NWN and other mod communities are asymmetric -- there is a huge pool of players, and a really small pool of content creators. There's a high barrier-of-entry, making a new adventure takes a lot of effort, and while a lot of people may make mini-adventures to mess around with the editor, most don't release it. As I understand it, content is also released at a fairly slow pace that reviewers can keep up with, so it is really good for reviewer-based quality control. (out of curiosity, how many NWN2 adventures are released in a day, and how many have been released to this date?)
MMO UGC is more symmetric, especially for games where customization is king (COH) and - I speculate - where the Foundry is touted as a main defining feature of the game (NW). Toolkits are accessible, and making new content is easy. At least in COH there was an absolute deluge of content created by people messing around, or trying out how the MA works, or making their own private areas --none of which are really enjoyable for others. In COH, there were over 300000 story arcs (a story arc was a set of 1-5 single missions) by month 4, near the end of the MA's Golden Age. By that point, there were a total of 521 story arc reviews in the ENTIRE community consisting of multiple forums and fansites (I have numbers to support this, I just copy/pasted this from that 38k txt file I mentioned
My point is basically that while reviews and spotlights are great for the author community (and for players too, somewhat), the only solution that can keep up with the volume of content being created is to drive the system by the only scalable part of the equation, which is the playerbase... and the rating system is a big part of it. It is just that the rating system, as it is now, has quite a few problems on its own when it comes to "letting good content bubble up to the top".
Getting good content to players in the Foundry - challenges and solutions
Handle: @zaphtastic
On my Dwarven forefathers I so swear
I agree to this wholeheartedly! Even though my quests and campaigns will likely be average at best I will try my best to give the players non-exploited content.