I'm not sure if it's just because I have close to as many plays as I have ratings, but it doesn't really feel like plays are accounted for in the adjusted rating even if the Wiki says so.
Rescue in Rainwall, ID: NW-DRQK3HKVV Chef's Challenge, ID: NW-DGTKIBVF3 The Dreamfall Campaign, ID: NWS-DEB7Z9IJC(3 Quests) Ogre: For the win!, ID: NW-DR5O3PD63 Cults and Culture, ID: NW-DCLSFYSQ2
I'm not sure if it's just because I have close to as many plays as I have ratings, but it doesn't really feel like plays are accounted for in the adjusted rating even if the Wiki says so.
Plays have absolutely no effect on the adjusted rating, or else Campaigns & Kobolds would be on the Best list. It has over a thousand plays, and only 400 something reviews. Even if a play only gave 0.001 adjusted rating points, a tenth of what I get from a five star review, it would easily have made the best list because it would gain at least 0.6 adjusted rating points from the plays alone, putting me at a 4.4 adjusted rating. By the way, I'm republishing my quests now.
Plays have absolutely no effect on the adjusted rating, or else Campaigns & Kobolds would be on the Best list. It has over a thousand plays, and only 400 something reviews. Even if a play only gave 0.001 adjusted rating points, a tenth of what I get from a five star review, it would easily have made the best list because it would gain at least 0.6 adjusted rating points from the plays alone, putting me at a 4.4 adjusted rating. By the way, I'm republishing my quests now.
That's not quite how the plays works. They don't give points per se, but bring the adjusted rating closer to the average rating asymptotically (see my post in the sticky thread, where I link to someone who calculated the % for a number of play values).
You can't compare. Amazon customers can only rate if a comment is left and those comments give clarity and context to the rating. If this is an aspect that I think won't bother me, then yes I am more than happy to buy. Heck, my most recent purchase was a 500 pounds sterling (it didn't like the pound sign :mad:) mixer than had been down-rated because the dough hooks kept breaking. Newsflash: they haven't built a mixer yet that can handle dough, that's why we have bread-making machines.
Amazon also moderate reviews and removes troll ratings. The Foundry has to rely on the adjusted ratings system to balance trolls at both ends of the scale and as far as I know, that is all the check it gets.
Well in that case if comments aren't required then low ratings should have even LESS impact on adjusted rating, because no explanation is required when expressing dissatisfaction. How many poor reviews have you seen on quests complaining about the loot? Even the ones that specifically mention at the top of the description that loot is outside the author's control?
Personally I think you should either be required to leave a comment if you provide a rating, or ratings without comments (regardless of the number of stars) should have a lower impact on the adjusted rating.
[SIGPIC][/SIGPIC]
Felling the Forgemaster: NW-DOHCJ5VE3(Elligible for Foundry Daily) Fleshrend's Big Adventure: NW-DBWJJYFDK (Elligible for Foundry Daily)
That's not quite how the plays works. They don't give points per se, but bring the adjusted rating closer to the average rating asymptotically (see my post in the sticky thread, where I link to someone who calculated the % for a number of play values).
My average rating for both my quests is 4.1, so lets say a hundred plays boost the adjusted rating by one percent of the average rating. One percent of 4.1 is 0.041, if I'm not mistaken. Two three star reviews would wipe the progress from a hundred plays in this case, and I'm willing to bet that if you do in fact gain adjusted rating points from plays, it's even less than 1% of your average rating per 100 plays.
Well in that case if comments aren't required then low ratings should have even LESS impact on adjusted rating, because no explanation is required when expressing dissatisfaction. How many poor reviews have you seen on quests complaining about the loot? Even the ones that specifically mention at the top of the description that loot is outside the author's control?
Personally I think you should either be required to leave a comment if you provide a rating, or ratings without comments (regardless of the number of stars) should have a lower impact on the adjusted rating.
I honestly think the devs should require a user to comment if they are rating a quest, and that the comment should be approved before the review is allowed to be posted. They could also display a disclaimer about Foundry limitations (Foundry authors have no control over loot, etc.) and start revoking the rating privileges of users who attempt to leave obvious troll reviews and get reported for it, plus remove said reviews. They'd never do this though, because that would require either paying somebody, or trusting a volunteer player, and the devs seem incapable of or unwilling to do these things.
Also, in regards to getting adjusted rating points from plays according to a percentage of your average rating, why do we see Foundry authors with adjusted ratings that seem higher than their average ratings? I have noticed quests with adjusted ratings of 4.6-4.7 that only showed four stars in their banner, whereas a lot of quests with adjusted ratings of less than 4.4 show four and a half stars in their banner. If the adjusted rating can climb higher than the average rating as I suspect, wouldn't that disprove this whole theory about plays adding a percentage of your average rating to your adjusted rating?
My average rating for both my quests is 4.1, so lets say a hundred plays boost the adjusted rating by one percent of the average rating. One percent of 4.1 is 0.041, if I'm not mistaken. Two three star reviews would wipe the progress from a hundred plays in this case, and I'm willing to bet that if you do in fact gain adjusted rating points from plays, it's even less than 1% of your average rating per 100 plays.
More plays drive the adjusted rating up, closer to the average (increasing towards the average asymptotically, not linearly), but the example 2 three stars would bring the average down. This means that it's hard to determine exactly what effect 2 plays with three star reviews would do to a 4.1 rating, as one factor increases, and one decreases, without doing more math than I'm willing to do for this game. When one of my quests had only two ratings (a 4 and a 5), my adjusted rating was 1.89. In that case a 3 star rating would likely increase my adjusted rating, despite lowering the average rating.
Plays without reviews will increase the adjusted average (asymptotically, not linearly, so more plays on something like Bills Tavern would have basically zero effect), ratings will still affect popular quests.
I honestly think the devs should require a user to comment if they are rating a quest, and that the comment should be approved before the review is allowed to be posted. They could also display a disclaimer about Foundry limitations (Foundry authors have no control over loot, etc.) and start revoking the rating privileges of users who attempt to leave obvious troll reviews and get reported for it, plus remove said reviews. They'd never do this though, because that would require either paying somebody, or trusting a volunteer player, and the devs seem incapable of or unwilling to do these things.
Do you mean the comment should be approved by the author or by Cryptic? I'm guessing you mean by Cryptic, though I don't see how they'd ever have the manpower to pull that off. If you meant approval by the author, that provides way too much potential for abuse. I think giving the author the ability to nullify one or two ratings, or maybe one rating per week, wouldn't necessarily be a bad thing. Not the comment, mind you - that should always stay if someone leaves one. Just say "this obviously punitive rating shouldn't be counted toward the adjusted total".
As for revoking rating privileges, I think even if they paid someone (or several people) to monitor reviews it would be impossible to punish all of the actual troll reviewers.
I think a better, more automated response would be to maybe lower the impact of a particular review based on the reviewer's past behavior. For example, if someone leaves a 1 star review, and they almost always leave a one star review, that person is either hypercritical compared to the general player base or has a habit of being intentionally punitive. The flip side would be to do the same for reviewers who almost always rate quests at 5 stars. They're maybe either not critical enough or always rate 5 stars simply out of support.
All of the above presumes that the current system isn't providing an accurate gauge of the quality of most quests, though. Since all quests operate under the same set of rules, if the current system provides an effective gauge of relative quality between quests then any changes to the rating calculation would be moot.
[SIGPIC][/SIGPIC]
Felling the Forgemaster: NW-DOHCJ5VE3(Elligible for Foundry Daily) Fleshrend's Big Adventure: NW-DBWJJYFDK (Elligible for Foundry Daily)
By my calculations, it's 86 percent.. Campaigns & Kobolds has an average rating of 4.1, its adjusted rating is 3.89, which is 94 percent of the adjusted rating. So over 1000 plays has increased it's adjusted rating by eight percent of the average rating. At that rate, it takes 1750 plays to reach 1000% of your adjusted rating, unless it starts to give you less of a percentage based increase as you get more plays. Basically, instead of trying to get a thousand plays, I should have kept republishing until I got 30 good reviews in a row.
I personally do not want to be rated a 5 simply because I traded a review with someone. It is a disservice to both the players who are now can't accuratly determine what rating a quest is anymore, and to the authors, who think everything is hunky doory because they have a bunch of 5 star reviews, even though their quest is barely 3-star quality.
Likewise I will always rate based on a combination of quality AND what I percieve as the intended focus of the quest. If it shows that the author was trying for a very combat, grindy quest, then I treat the quest as such, and then look for flaws in that style. I will not downrate a quest just because I think it has too much combat, as long as it was known the intention for the quest was a combat quest, but I will downrate of the implementation of the quest is really bad.
As much as the trolls ruin the rating system, we as authors do NOTHING to help the issue by being just the opposite. Might as well as remove ratings all together as they will not be accurate at all for the average, non-foundry player looking for something to play.
There are far more high rated quests out there that are of 2 to 3 quality then there are 3-5 quality, but these lower quality quests are stil lsitting at 4 stars simply because of authors think of them selves as doing a good thing and giving high ratings regardless of actual quality just because they are "supporting the community"...but the players are just as much a part of the community as the authors, and they suffer the most when this happens.
thelazar1 is not imagining things; there seems to be a point cap that triggers anonymous 1 and 2 star bombs. This happens at any time of day so it really seems like a bot.
The real question is, "why is it possible to make anonymous reviews". The tin-foil is telling me there is only one possible reason...
thelazar1 is not imagining things; there seems to be a point cap that triggers anonymous 1 and 2 star bombs. This happens at any time of day so it really seems like a bot.
The real question is, "why is it possible to make anonymous reviews". The tin-foil is telling me there is only one possible reason...
I experienced this with Campaigns & Kobolds, and to a greater extent Zigby's First Strike. Campaigns & Kobold got hit every time the adjusted rating hit 3.9. Zigby's First Strike got hit every time it got above 3.7. Now, it could be that the tin-foil is just a little paranoid, but I still suspect a conspiracy...
Anyway, Devs, can we get a positive only rating system, please? It would eliminate a lot of the paranoia, arguing, and bitterness in the Foundry community.
By my calculations, it's 86 percent.. Campaigns & Kobolds has an average rating of 4.1, its adjusted rating is 3.89, which is 94 percent of the adjusted rating. So over 1000 plays has increased it's adjusted rating by eight percent of the average rating. At that rate, it takes 1750 plays to reach 1000% of your adjusted rating, unless it starts to give you less of a percentage based increase as you get more plays. Basically, instead of trying to get a thousand plays, I should have kept republishing until I got 30 good reviews in a row.
And yes, the more plays you have, the less each additional play can raise the adjusted score. The first 60 plays get your adjusted rating around 87% of the way to the average. The next 1000 plays only gets you another 10% (to 97%).
0
lolsorhandMember, Neverwinter Beta Users, Neverwinter Hero Users, Neverwinter Knight of the Feywild UsersPosts: 981Bounty Hunter
edited August 2013
Wu, Perhaps you should simply look at your own work. And improve it enough so that people might enjoy it, instead of pointing towards my quest - Are you angry? I had 30 reviews, and 28 5s. 2 4s. I'm sorry if that offends you.
Wu, Perhaps you should simply look at your own work. And improve it enough so that people might enjoy it, instead of pointing towards my quest - Are you angry? I had 30 reviews, and 28 5s. 2 4s. I'm sorry if that offends you.
I agree with the sentiment - one shouldn't immediately assume that a poor rating is someone trolling. But if people leave a bad rating with no comment, how do you know what to improve?
[SIGPIC][/SIGPIC]
Felling the Forgemaster: NW-DOHCJ5VE3(Elligible for Foundry Daily) Fleshrend's Big Adventure: NW-DBWJJYFDK (Elligible for Foundry Daily)
0
lolsorhandMember, Neverwinter Beta Users, Neverwinter Hero Users, Neverwinter Knight of the Feywild UsersPosts: 981Bounty Hunter
I agree with the sentiment - one shouldn't immediately assume that a poor rating is someone trolling. But if people leave a bad rating with no comment, how do you know what to improve?
Complaining about it doesn't make it any better - This is more of a childish fixation. You shouldn't make quests if all you care about is the rating. Have fun, and start at that - An improved rating system is coming, so have patience. As for above, this is far from the first time it has happened. As in calling out someone. It's quite funny I got a funny review that I shared with my friends shortly after the post was made, from an alt account
Have fun, that's what truly matters. I love making maps, and that's why I'm doing it. Why are you doing it?
Have fun, that's what truly matters. I love making maps, and that's why I'm doing it. Why are you doing it?
I make them for the same reason - I like the process. I sure as heck wouldn't be doing it if all I wanted was to be recognized for my awesomeness /s There are much easier ways to achieve validation than tossing a Neverwinter Foundry Quest out into the void, after all.
And I'm also not necessarily complaining. I don't start with the assumption that my maps are some gift to the world and get upset if someone doesn't like them. I also know there are lots of flaws...I've only made two maps at this point. I started this post primarily because I honestly didn't know you could leave a rating without a comment. so I was confused by what I saw in the UI.
All that said, it doesn't hurt to point out where the current system could be better if that's where the conversation turns. If someone is motivated by recognition of their work, even if you look down on such motivation you have to admit the current system makes it pretty difficult. I expect things will be a lot different with the UI updates in the coming patch, though.
[SIGPIC][/SIGPIC]
Felling the Forgemaster: NW-DOHCJ5VE3(Elligible for Foundry Daily) Fleshrend's Big Adventure: NW-DBWJJYFDK (Elligible for Foundry Daily)
Complaining about it doesn't make it any better - This is more of a childish fixation. You shouldn't make quests if all you care about is the rating. Have fun, and start at that - An improved rating system is coming, so have patience. As for above, this is far from the first time it has happened. As in calling out someone. It's quite funny I got a funny review that I shared with my friends shortly after the post was made, from an alt account
Have fun, that's what truly matters. I love making maps, and that's why I'm doing it. Why are you doing it?
I agree. Have fun. Don't let any review get to you. (Of course you have to get reviews first.)
I do it solely for the enjoyment of making maps. And I try both ways... I use pre-made maps (which some authors look down at) and I make my own.
Sweet Water and Light Laughter Till Next We Meet.
Narayan
0
lolsorhandMember, Neverwinter Beta Users, Neverwinter Hero Users, Neverwinter Knight of the Feywild UsersPosts: 981Bounty Hunter
I make them for the same reason - I like the process. I sure as heck wouldn't be doing it if all I wanted was to be recognized for my awesomeness /s There are much easier ways to achieve validation than tossing a Neverwinter Foundry Quest out into the void, after all.
And I'm also not necessarily complaining. I don't start with the assumption that my maps are some gift to the world and get upset if someone doesn't like them. I also know there are lots of flaws...I've only made two maps at this point. I started this post primarily because I honestly didn't know you could leave a rating without a comment. so I was confused by what I saw in the UI.
All that said, it doesn't hurt to point out where the current system could be better if that's where the conversation turns. If someone is motivated by recognition of their work, even if you look down on such motivation you have to admit the current system makes it pretty difficult. I expect things will be a lot different with the UI updates in the coming patch, though.
You know what? I'll give your maps a play through!
Which brings me to another point I have made here in the past...if fellow authors trade 5* to help each other get on the list and those stars are not a fair reflection of the quality of the quest, then the expectations of the players who follow are not going to be met. Unfortunately, if their expectations are not met they tend to not be generous - even stupid people know when they have been conned.
A couple of authors have said to me that 3* is not a 'Good' rating and to that I say...well, what I'd say wouldn't get around the filter. Three is a good rating; if it has been diminished, then blame authors who have been less than honest when rating each other's quests. You reap what you sow.
100 times ^ this.
As an Author I value a genuine 3*, or even a genuine 1* over a false 5*.
For me 3* is "it works, there's nothing structurally (game breaking) wrong with the maps, the quest makes sense, and there's maybe some typo's, and a few missed default labels etc".
4* is "Its a very polished, well put together, and clever use of Foundry assets, with an above average storyline, good attention to detail, and minimal typo's / labelling issues".
5* is "excellent / exceptional use of Foundry assets, excellent story, impressive attention to detail, no labelling errors, but I'll still let a few typos slide".
So I don't see a 3* rating as a bad thing either.
As a community we simply can't beat the dishonesty of the exploit Authors and Players by being equally dishonest in how we rate one another.
Furthermore, such dishonest will, as Karitr pointed out, eventually turn around and bite us. Players going in to a quest that has tons of 5* ratings but is actually only 3* content are not going to be happy. And players can read folks, they'll soon figure it out if a bunch of Authors are deliberately up-rating one another.
We need to start thinking about the Player - Author relationship in the same way we think about the Author - Cryptic relationship.
In the latter we are the consumer, and we need to have trust in Cryptic that they will do the right and honest thing as regards what we consume - that is they'll help us deal with the trolls, the exploiters etc and they'll give us a product in working order and that is "as advertised".
But in the former relationship we are the purveyor, and the Players are the consumers. We need to be honest with them, in the way we expect Crytpic to be honest with us. And we can't do that by falsely up-rating one another's content just to try and mitigate against the, perceived, errors of the system.
Hmmm....something just occurred to me. As part of my job from time to time I get performance reviews from customers measured on the same scale - 1 to 5. Except it's not one overall review of the performance - the reviewer actually provides ratings for various aspects of their experience.
Most people actually rate just about everything as a 5, which is of course what we strive for. Some people, though (myself included) can't just give out a 5 to be nice, or avoid hurting feelings. Something only deserves a max rating if it earned that rating.
If any of our categories are rated a 3 or less, I actually start getting emails asking me why. That is...unless most or all of the other categories are also rated about the same. That indicates the reviewer sees a 3 as maybe average, and not bad. We get so many 5s, though, that we reflexively see a 3 as bad.
Anyway, back to the point. What if reviewers were able to rate 3 or 4 aspects of a quest? Major categories like what ronin described - Story, detail, difficulty, overall quality...take your pick. That would serve a few of purposes:
1. As an author, if all ratings were not 5s but about the same, it's likely the reviewer is just more critical than average, and it may not point to an issue with your quest that needs attention
2. As an author, if one rating was lower than the others, then even if the reviewer didn't comment you might at least have some idea what they didn't like so you could consider addressing it
3. As a player, you could see aggregate ratings for each aspect of the quest, which could help you pick out what you'd like from the existing HUGE pool. If a quest had a terrible story rating but a top-notch combat rating, and you're just looking for hack and slash, you wouldn't care that there was no story.
We need to start thinking about the Player...
I cut your quote short there (even though that wasn't actually your point) because I think those words are important. Think about the players, if you're not already. I think many of us spend so much time working on quests and playing other author's quests for testing and kickstarting them that we forget what it's like to be a random player that never visits a forum. You have this gigantic list of stuff to pick from...how do you choose something that's going to be worth your time? Right now I'd argue the UI is way worse for a player than the authors.
[SIGPIC][/SIGPIC]
Felling the Forgemaster: NW-DOHCJ5VE3(Elligible for Foundry Daily) Fleshrend's Big Adventure: NW-DBWJJYFDK (Elligible for Foundry Daily)
0
cipher9nemoMember, Neverwinter Beta UsersPosts: 0Arc User
As an Author I value a genuine 3*, or even a genuine 1* over a false 5*.
I understand the principles behind your post, and agree with many of them, but this statement sort of rubs me the wrong way.
If the feedback and review is the same why would you prefer a "genuine" 3 over a "false" 5 rating? Can you honestly tell me that you'd rather get 3 star ratings on your quest rather than 5 stars? Has your quest already made it 'big time' yet? Or do you think everything you'll make now and into the future deserves a "genuine" 5 star rating? I haven't played your quest yet, so you'll have to tell me.
The first problem is that every author will have a biased opinion of their own quest.
The second problem is that many authors worry too much about whether other quests receive accurate ratings or not.
If I review an author's quest either from the official forums here, from Scribe's Enclave, or within my guild, I will always give them a 5 star rating if they're not a featured/spotlighted quest and not near the top of the best tab. It's not because I think their quest is truly a 5 star piece of work. It's because the rating system is broken and I'm doing all I can to support the underdogs here to get their quest attention. Once their quest 'makes it big', then the real reviews take place where people can be brutally honest. That second phase--with ratings from the mass audiences of Neverwinter--determines what the quest will eventually receive. It will overpower any "false" 5 star ratings we as a smaller community could ever give a quest.
It Cryptic didn't have this adjusted rating HAMSTER that penalizes low play count quests I would agree with you, redneckronin. But since it's there it rewards 1.) older quests, even worse for ones that have been around since beta, and 2.) quests that get featured/spotlighted. The masses of Neverwinter's audience plays mostly those quests that show up at the top of the tabs. The play count is proof of that. That's simply not fair to newer authors and authors with UGC that doesn't 'play' the system like so many before them. That's why I give them all five stars to help move that adjusted rating and get them some attention for the masses.
The rating system is broken, and I hope you agree. If you do, then why on earth would you support feeding that broken system by claiming "genuine" ratings? If not, then I hope you get a chance to reflect on that again in the future.
Cipher, you are over-estimating the amount of plays and reviews needed to get on a list. Not to mention missing ronin's point by a country mile.
If a player picks up a relatively new quest which has only been reviewed by authors, friends of, family and guildmates and so has 100% five star ratings, he or she - unaware these ratings were awarded not for the quality of the quest, but to try and circumvent the ratings system - will expect an exciting adventure ahead of them. Upon playing the quest and finding it far from 5* worthy, they are likely to undervalue the content as opposed to rating it fairly.
You guys have been rating out of kindness for as long as I have been coming here and still the system is deemed 'broken' by many, you included.
This shows you're circumventing nothing. The actual system in use is (according to wiki, I'm a not a statistician), one that provides the most accurate results and by trying to circumvent it, you become part of the problem which results in the necessity for a system like this to be implemented.
We need to start thinking about the Player - Author relationship in the same way we think about the Author - Cryptic relationship.
In the latter we are the consumer, and we need to have trust in Cryptic that they will do the right and honest thing as regards what we consume - that is they'll help us deal with the trolls, the exploiters etc and they'll give us a product in working order and that is "as advertised".
But in the former relationship we are the purveyor, and the Players are the consumers. We need to be honest with them, in the way we expect Crytpic to be honest with us. And we can't do that by falsely up-rating one another's content just to try and mitigate against the, perceived, errors of the system.
I wish all author's felt this way. I know many do, but thank you ronin for saying it
The first time my quest got a 5-star review from a random stranger, i was very happy that i made it. I had a lot of fun creating the story, and was really happy that someone else liked it too. I never advertised my quest until it fell off of the "New" tab, and it's never been submitted to Cryptic. It's doing well enough on it's own, better than i hoped it would, and i'm having just as much fun working on my second quest.
Every 5-star review makes me feel really good, and i have 37 of them (at last count). Every 4-star makes me happy, and i have 39 of those (at last count). Every 3-star review i get makes me feel i could have done something better, and i have 15 of those (at last count).
1-star reviews don't bother me, i really just ignore them, all 8 of them (at last count). What hurts the most, for me, are the 2-star reviews, both of them (at last count).
A few times, i've contacted people who left the 3-star reviews, to see what disappointed them and what would have made it better for them. I've made changes to appease them, and a couple of these people have gone back later and increased their rating based on these changes. However, it seems not everyone is interested in seeing the changes i made based on their feedback, but that's fine too. Maybe those changes will get me a better rating from someone else later on.
As for the author community, i feel a bit out-of-the-loop at times. I've seen authors accused of revenge 1-starring, and i've seen authors admit to doing it, and i just don't get that. Why do they get so mad at each other? Even before i started making my first quest, i was like a child in an amusement park playing foundry quests, i thought it was so cool to see what other players created. And especially after i started tinkering with the editor myself, i found so much appreciation and admiration for what authors come up with. To the best of my recollection, i've never given a 1- or 2-star rating, and not too many 3-stars. I'm easily entertained, and give a lot of 5-stars because the quest makes me feel so giddy, and i give a lot of 4-stars because i appreciate the work that went into it, even if it didn't make me all giddy like some of them do. There was a long thread about this before, and i remember explaining that the reason the current system kind of works, is because everyone has their own way of rating. Not everyone votes fairly, players (and i don't mean other authors, i just don't get that) give 1-star ratings for illogical reasons, i.e. loot, spite, just because they can. So, giving a logical fair rating of 3-stars doesn't give it a 3-star average, because someone else trolled the system. It is for that reason that i am more willing to give 5- and 4-star reviews, to do my little part in off-setting the players that are trolling the system. My 5-star rating doesn't give the quest a 5-star average, it just increases the average it has from all the other ratings. And also, i guess the quests players create just make me happier than some other people.
I'm anxious to see how the new rating system works out for everyone.
and by trying to circumvent it, you become part of the problem which results in the necessity for a system like this to be implemented.
It's a 'chicken and the egg' problem. I wasn't here rating and reviewing quests at the time, so I'm definitely not "part of the problem". The problem existed far before I ever participated.
So how can you possibly blame reviewers for being a part of the problem when the problem already existed? Until Cryptic/PWE finds and implements a rating system that works, you can't fault people for being kind to the community or reviewers here.
Just like the stock market, when a publicly traded name takes a nose dive you can't blame everyone else for bailing after that fact.
As for the author community, i feel a bit out-of-the-loop at times. I've seen authors accused of revenge 1-starring, and i've seen authors admit to doing it, and i just don't get that. Why do they get so mad at each other?
Comments
According to Neverwinter Wiki: (http://neverwinter.gamepedia.com/Guide_to_the_Foundry)
"Cryptic uses the Lower Bound of Wilson Score Interval to determine the rating of quests."
And according to Wikipedia, that would be:
http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Wilson_score_interval
I'm not sure if it's just because I have close to as many plays as I have ratings, but it doesn't really feel like plays are accounted for in the adjusted rating even if the Wiki says so.
Chef's Challenge, ID: NW-DGTKIBVF3
The Dreamfall Campaign, ID: NWS-DEB7Z9IJC (3 Quests)
Ogre: For the win!, ID: NW-DR5O3PD63
Cults and Culture, ID: NW-DCLSFYSQ2
Plays have absolutely no effect on the adjusted rating, or else Campaigns & Kobolds would be on the Best list. It has over a thousand plays, and only 400 something reviews. Even if a play only gave 0.001 adjusted rating points, a tenth of what I get from a five star review, it would easily have made the best list because it would gain at least 0.6 adjusted rating points from the plays alone, putting me at a 4.4 adjusted rating. By the way, I'm republishing my quests now.
Well in that case if comments aren't required then low ratings should have even LESS impact on adjusted rating, because no explanation is required when expressing dissatisfaction. How many poor reviews have you seen on quests complaining about the loot? Even the ones that specifically mention at the top of the description that loot is outside the author's control?
Personally I think you should either be required to leave a comment if you provide a rating, or ratings without comments (regardless of the number of stars) should have a lower impact on the adjusted rating.
Felling the Forgemaster: NW-DOHCJ5VE3 (Elligible for Foundry Daily)
Fleshrend's Big Adventure: NW-DBWJJYFDK
(Elligible for Foundry Daily)
My average rating for both my quests is 4.1, so lets say a hundred plays boost the adjusted rating by one percent of the average rating. One percent of 4.1 is 0.041, if I'm not mistaken. Two three star reviews would wipe the progress from a hundred plays in this case, and I'm willing to bet that if you do in fact gain adjusted rating points from plays, it's even less than 1% of your average rating per 100 plays.
I honestly think the devs should require a user to comment if they are rating a quest, and that the comment should be approved before the review is allowed to be posted. They could also display a disclaimer about Foundry limitations (Foundry authors have no control over loot, etc.) and start revoking the rating privileges of users who attempt to leave obvious troll reviews and get reported for it, plus remove said reviews. They'd never do this though, because that would require either paying somebody, or trusting a volunteer player, and the devs seem incapable of or unwilling to do these things.
Plays without reviews will increase the adjusted average (asymptotically, not linearly, so more plays on something like Bills Tavern would have basically zero effect), ratings will still affect popular quests.
Do you mean the comment should be approved by the author or by Cryptic? I'm guessing you mean by Cryptic, though I don't see how they'd ever have the manpower to pull that off. If you meant approval by the author, that provides way too much potential for abuse. I think giving the author the ability to nullify one or two ratings, or maybe one rating per week, wouldn't necessarily be a bad thing. Not the comment, mind you - that should always stay if someone leaves one. Just say "this obviously punitive rating shouldn't be counted toward the adjusted total".
As for revoking rating privileges, I think even if they paid someone (or several people) to monitor reviews it would be impossible to punish all of the actual troll reviewers.
I think a better, more automated response would be to maybe lower the impact of a particular review based on the reviewer's past behavior. For example, if someone leaves a 1 star review, and they almost always leave a one star review, that person is either hypercritical compared to the general player base or has a habit of being intentionally punitive. The flip side would be to do the same for reviewers who almost always rate quests at 5 stars. They're maybe either not critical enough or always rate 5 stars simply out of support.
All of the above presumes that the current system isn't providing an accurate gauge of the quality of most quests, though. Since all quests operate under the same set of rules, if the current system provides an effective gauge of relative quality between quests then any changes to the rating calculation would be moot.
Felling the Forgemaster: NW-DOHCJ5VE3 (Elligible for Foundry Daily)
Fleshrend's Big Adventure: NW-DBWJJYFDK
(Elligible for Foundry Daily)
Likewise I will always rate based on a combination of quality AND what I percieve as the intended focus of the quest. If it shows that the author was trying for a very combat, grindy quest, then I treat the quest as such, and then look for flaws in that style. I will not downrate a quest just because I think it has too much combat, as long as it was known the intention for the quest was a combat quest, but I will downrate of the implementation of the quest is really bad.
As much as the trolls ruin the rating system, we as authors do NOTHING to help the issue by being just the opposite. Might as well as remove ratings all together as they will not be accurate at all for the average, non-foundry player looking for something to play.
There are far more high rated quests out there that are of 2 to 3 quality then there are 3-5 quality, but these lower quality quests are stil lsitting at 4 stars simply because of authors think of them selves as doing a good thing and giving high ratings regardless of actual quality just because they are "supporting the community"...but the players are just as much a part of the community as the authors, and they suffer the most when this happens.
The real question is, "why is it possible to make anonymous reviews". The tin-foil is telling me there is only one possible reason...
I experienced this with Campaigns & Kobolds, and to a greater extent Zigby's First Strike. Campaigns & Kobold got hit every time the adjusted rating hit 3.9. Zigby's First Strike got hit every time it got above 3.7. Now, it could be that the tin-foil is just a little paranoid, but I still suspect a conspiracy...
Anyway, Devs, can we get a positive only rating system, please? It would eliminate a lot of the paranoia, arguing, and bitterness in the Foundry community.
http://nw-forum.perfectworld.com/showthread.php?153441-Please-explain-what-is-adjusted-ratings&p=2207451&viewfull=1#post2207451
And yes, the more plays you have, the less each additional play can raise the adjusted score. The first 60 plays get your adjusted rating around 87% of the way to the average. The next 1000 plays only gets you another 10% (to 97%).
Brethren of the Five, Campaign. - Story focused
The Dwarven Tale - Hack 'N Slash
I agree with the sentiment - one shouldn't immediately assume that a poor rating is someone trolling. But if people leave a bad rating with no comment, how do you know what to improve?
Felling the Forgemaster: NW-DOHCJ5VE3 (Elligible for Foundry Daily)
Fleshrend's Big Adventure: NW-DBWJJYFDK
(Elligible for Foundry Daily)
Complaining about it doesn't make it any better - This is more of a childish fixation. You shouldn't make quests if all you care about is the rating. Have fun, and start at that - An improved rating system is coming, so have patience. As for above, this is far from the first time it has happened. As in calling out someone. It's quite funny I got a funny review that I shared with my friends shortly after the post was made, from an alt account
Have fun, that's what truly matters. I love making maps, and that's why I'm doing it. Why are you doing it?
Brethren of the Five, Campaign. - Story focused
The Dwarven Tale - Hack 'N Slash
To find creative ways to incorporate cheese into serious dramas.
Threads: Part 1: Rising of the Dragons (NW-DNGIC6AJC) | Part 2: Abyssal Pursuit (NW-DESQ9HQAZ)
Indeed lol
Brethren of the Five, Campaign. - Story focused
The Dwarven Tale - Hack 'N Slash
I make them for the same reason - I like the process. I sure as heck wouldn't be doing it if all I wanted was to be recognized for my awesomeness /s
And I'm also not necessarily complaining. I don't start with the assumption that my maps are some gift to the world and get upset if someone doesn't like them. I also know there are lots of flaws...I've only made two maps at this point. I started this post primarily because I honestly didn't know you could leave a rating without a comment. so I was confused by what I saw in the UI.
All that said, it doesn't hurt to point out where the current system could be better if that's where the conversation turns. If someone is motivated by recognition of their work, even if you look down on such motivation you have to admit the current system makes it pretty difficult. I expect things will be a lot different with the UI updates in the coming patch, though.
Felling the Forgemaster: NW-DOHCJ5VE3 (Elligible for Foundry Daily)
Fleshrend's Big Adventure: NW-DBWJJYFDK
(Elligible for Foundry Daily)
I agree. Have fun. Don't let any review get to you. (Of course you have to get reviews first.)
I do it solely for the enjoyment of making maps. And I try both ways... I use pre-made maps (which some authors look down at) and I make my own.
Narayan
You know what?
Brethren of the Five, Campaign. - Story focused
The Dwarven Tale - Hack 'N Slash
As long as I'm having fun playing them, I don't normally care - I'll send feedback though. But it depends, I'm not always handing out fives
Edit: - Should also add, I've never rebublished anything. And The Trials of Five had the same ratings going in as this one had.
Brethren of the Five, Campaign. - Story focused
The Dwarven Tale - Hack 'N Slash
Thanks
Felling the Forgemaster: NW-DOHCJ5VE3 (Elligible for Foundry Daily)
Fleshrend's Big Adventure: NW-DBWJJYFDK
(Elligible for Foundry Daily)
100 times ^ this.
As an Author I value a genuine 3*, or even a genuine 1* over a false 5*.
For me 3* is "it works, there's nothing structurally (game breaking) wrong with the maps, the quest makes sense, and there's maybe some typo's, and a few missed default labels etc".
4* is "Its a very polished, well put together, and clever use of Foundry assets, with an above average storyline, good attention to detail, and minimal typo's / labelling issues".
5* is "excellent / exceptional use of Foundry assets, excellent story, impressive attention to detail, no labelling errors, but I'll still let a few typos slide".
So I don't see a 3* rating as a bad thing either.
As a community we simply can't beat the dishonesty of the exploit Authors and Players by being equally dishonest in how we rate one another.
Furthermore, such dishonest will, as Karitr pointed out, eventually turn around and bite us. Players going in to a quest that has tons of 5* ratings but is actually only 3* content are not going to be happy. And players can read folks, they'll soon figure it out if a bunch of Authors are deliberately up-rating one another.
We need to start thinking about the Player - Author relationship in the same way we think about the Author - Cryptic relationship.
In the latter we are the consumer, and we need to have trust in Cryptic that they will do the right and honest thing as regards what we consume - that is they'll help us deal with the trolls, the exploiters etc and they'll give us a product in working order and that is "as advertised".
But in the former relationship we are the purveyor, and the Players are the consumers. We need to be honest with them, in the way we expect Crytpic to be honest with us. And we can't do that by falsely up-rating one another's content just to try and mitigate against the, perceived, errors of the system.
All The Best
Looking For Reviews For Your Foundry Quest?
Drop By Scribe's Enclave & Meet Up With Volunteer Reviewers.
Most people actually rate just about everything as a 5, which is of course what we strive for. Some people, though (myself included) can't just give out a 5 to be nice, or avoid hurting feelings. Something only deserves a max rating if it earned that rating.
If any of our categories are rated a 3 or less, I actually start getting emails asking me why. That is...unless most or all of the other categories are also rated about the same. That indicates the reviewer sees a 3 as maybe average, and not bad. We get so many 5s, though, that we reflexively see a 3 as bad.
Anyway, back to the point. What if reviewers were able to rate 3 or 4 aspects of a quest? Major categories like what ronin described - Story, detail, difficulty, overall quality...take your pick. That would serve a few of purposes:
1. As an author, if all ratings were not 5s but about the same, it's likely the reviewer is just more critical than average, and it may not point to an issue with your quest that needs attention
2. As an author, if one rating was lower than the others, then even if the reviewer didn't comment you might at least have some idea what they didn't like so you could consider addressing it
3. As a player, you could see aggregate ratings for each aspect of the quest, which could help you pick out what you'd like from the existing HUGE pool. If a quest had a terrible story rating but a top-notch combat rating, and you're just looking for hack and slash, you wouldn't care that there was no story.
I cut your quote short there (even though that wasn't actually your point) because I think those words are important. Think about the players, if you're not already. I think many of us spend so much time working on quests and playing other author's quests for testing and kickstarting them that we forget what it's like to be a random player that never visits a forum. You have this gigantic list of stuff to pick from...how do you choose something that's going to be worth your time? Right now I'd argue the UI is way worse for a player than the authors.
Felling the Forgemaster: NW-DOHCJ5VE3 (Elligible for Foundry Daily)
Fleshrend's Big Adventure: NW-DBWJJYFDK
(Elligible for Foundry Daily)
I understand the principles behind your post, and agree with many of them, but this statement sort of rubs me the wrong way.
If the feedback and review is the same why would you prefer a "genuine" 3 over a "false" 5 rating? Can you honestly tell me that you'd rather get 3 star ratings on your quest rather than 5 stars? Has your quest already made it 'big time' yet? Or do you think everything you'll make now and into the future deserves a "genuine" 5 star rating? I haven't played your quest yet, so you'll have to tell me.
The first problem is that every author will have a biased opinion of their own quest.
The second problem is that many authors worry too much about whether other quests receive accurate ratings or not.
If I review an author's quest either from the official forums here, from Scribe's Enclave, or within my guild, I will always give them a 5 star rating if they're not a featured/spotlighted quest and not near the top of the best tab. It's not because I think their quest is truly a 5 star piece of work. It's because the rating system is broken and I'm doing all I can to support the underdogs here to get their quest attention. Once their quest 'makes it big', then the real reviews take place where people can be brutally honest. That second phase--with ratings from the mass audiences of Neverwinter--determines what the quest will eventually receive. It will overpower any "false" 5 star ratings we as a smaller community could ever give a quest.
It Cryptic didn't have this adjusted rating HAMSTER that penalizes low play count quests I would agree with you, redneckronin. But since it's there it rewards 1.) older quests, even worse for ones that have been around since beta, and 2.) quests that get featured/spotlighted. The masses of Neverwinter's audience plays mostly those quests that show up at the top of the tabs. The play count is proof of that. That's simply not fair to newer authors and authors with UGC that doesn't 'play' the system like so many before them. That's why I give them all five stars to help move that adjusted rating and get them some attention for the masses.
The rating system is broken, and I hope you agree. If you do, then why on earth would you support feeding that broken system by claiming "genuine" ratings? If not, then I hope you get a chance to reflect on that again in the future.
Hammerfist Clan. Jump into the Night: NW-DMXWRYTAD
If a player picks up a relatively new quest which has only been reviewed by authors, friends of, family and guildmates and so has 100% five star ratings, he or she - unaware these ratings were awarded not for the quality of the quest, but to try and circumvent the ratings system - will expect an exciting adventure ahead of them. Upon playing the quest and finding it far from 5* worthy, they are likely to undervalue the content as opposed to rating it fairly.
You guys have been rating out of kindness for as long as I have been coming here and still the system is deemed 'broken' by many, you included.
This shows you're circumventing nothing. The actual system in use is (according to wiki, I'm a not a statistician), one that provides the most accurate results and by trying to circumvent it, you become part of the problem which results in the necessity for a system like this to be implemented.
I wish all author's felt this way. I know many do, but thank you ronin for saying it
Every 5-star review makes me feel really good, and i have 37 of them (at last count). Every 4-star makes me happy, and i have 39 of those (at last count). Every 3-star review i get makes me feel i could have done something better, and i have 15 of those (at last count).
1-star reviews don't bother me, i really just ignore them, all 8 of them (at last count). What hurts the most, for me, are the 2-star reviews, both of them (at last count).
A few times, i've contacted people who left the 3-star reviews, to see what disappointed them and what would have made it better for them. I've made changes to appease them, and a couple of these people have gone back later and increased their rating based on these changes. However, it seems not everyone is interested in seeing the changes i made based on their feedback, but that's fine too. Maybe those changes will get me a better rating from someone else later on.
As for the author community, i feel a bit out-of-the-loop at times. I've seen authors accused of revenge 1-starring, and i've seen authors admit to doing it, and i just don't get that. Why do they get so mad at each other? Even before i started making my first quest, i was like a child in an amusement park playing foundry quests, i thought it was so cool to see what other players created. And especially after i started tinkering with the editor myself, i found so much appreciation and admiration for what authors come up with. To the best of my recollection, i've never given a 1- or 2-star rating, and not too many 3-stars. I'm easily entertained, and give a lot of 5-stars because the quest makes me feel so giddy, and i give a lot of 4-stars because i appreciate the work that went into it, even if it didn't make me all giddy like some of them do. There was a long thread about this before, and i remember explaining that the reason the current system kind of works, is because everyone has their own way of rating. Not everyone votes fairly, players (and i don't mean other authors, i just don't get that) give 1-star ratings for illogical reasons, i.e. loot, spite, just because they can. So, giving a logical fair rating of 3-stars doesn't give it a 3-star average, because someone else trolled the system. It is for that reason that i am more willing to give 5- and 4-star reviews, to do my little part in off-setting the players that are trolling the system. My 5-star rating doesn't give the quest a 5-star average, it just increases the average it has from all the other ratings. And also, i guess the quests players create just make me happier than some other people.
I'm anxious to see how the new rating system works out for everyone.
[UGC] Kolde Acres (Discontinued)
It's a 'chicken and the egg' problem. I wasn't here rating and reviewing quests at the time, so I'm definitely not "part of the problem". The problem existed far before I ever participated.
So how can you possibly blame reviewers for being a part of the problem when the problem already existed? Until Cryptic/PWE finds and implements a rating system that works, you can't fault people for being kind to the community or reviewers here.
Just like the stock market, when a publicly traded name takes a nose dive you can't blame everyone else for bailing after that fact.
QFT.
Hammerfist Clan. Jump into the Night: NW-DMXWRYTAD