Question about Reviews

rogu3ish · August 2013

I saw two new reviews show up on one of my quests to day - one at 2 stars and one at 3 stars. When I look at the list of reviews, however (both when editing the quest and in-game) I don't see any 2 or 3 star reviews. Is it possible to leave a review without providing any feedback? And if so, does it then not show up in the list of reviews?

xpurpleinsanityx · August 2013

That would be correct. If they do not add a comment, the review is still counted, but it is anonymous.

rogu3ish · August 2013

Thanks. That's too bad, really. Everyone who has left a comment has rated the quest 5 stars, with one exception at 4. It would be nice to know what the players who rated the 2 and 3 didn't like about it

thalazar1 · August 2013

those of us* with tin foil hats like to think it was because our rating was getting too high.

*talking about myself not poking at any one else. tin foil may cause blindness, sudden sadness, in ability to enjoy anything, random desire for ice cream, night terrors, or in extreme cases death. not recommend for use by any one, see your GM if tin foil is right for you.

groshie · August 2013

thalazar1 wrote: »

those of us* with tin foil hats like to think it was because our rating was getting too high.

The people who apparently do that sort of thing would leave 1-stars though, right? For maximum impact?

thalazar1 · August 2013

they could or they could leave 2 and 3 stars. my quest hovers around 4 ish. when it gets up to 4.08ish it seems to get enough 2 and 3 stars to drop it to 3.90 then it climbs back up to 4.02ish and now i just got two 1 stars dropping it back down to 3.95.

I'm find it funny that ppl feel the need to regulate quests like this to prevent them from being at the top of the new tab or even making it to the best tab (I know that sounds a bit conceited but i have more 5 stars then 4, 3, 2, and 1s combined, and only seem to get 1, 2, 3s when i climb too high in adjusted rating)
another week or two and the quest will fall into the foundry abyss and it wont matter anymore.

xpurpleinsanityx · August 2013

thalazar1 wrote: »

they could or they could leave 2 and 3 stars. my quest hovers around 4 ish. when it gets up to 4.08ish it seems to get enough 2 and 3 stars to drop it to 3.90 then it climbs back up to 4.02ish and now i just got two 1 stars dropping it back down to 3.95.

I'm find it funny that ppl feel the need to regulate quests like this to prevent them from being at the top of the new tab or even making it to the best tab (I know that sounds a bit conceited but i have more 5 stars then 4, 3, 2, and 1s combined, and only seem to get 1, 2, 3s when i climb too high in adjusted rating)
another week or two and the quest will fall into the foundry abyss and it wont matter anymore.

o.o That's sooooo weird. That was happening to me when I was in that zone of adjusted rating and I thought I was going a lil crazy with it. It stopped mostly once I broke past that area of rating...

Edit: It was -all- silent 2's and 3's

groshie · August 2013

thalazar1 wrote: »

they could or they could leave 2 and 3 stars. my quest hovers around 4 ish. when it gets up to 4.08ish it seems to get enough 2 and 3 stars to drop it to 3.90 then it climbs back up to 4.02ish and now i just got two 1 stars dropping it back down to 3.95.

xpurpleinsanityx wrote: »

o.o That's sooooo weird. That was happening to me when I was in that zone of adjusted rating and I thought I was going a lil crazy with it. It stopped mostly once I broke past that area of rating...

Edit: It was -all- silent 2's and 3's

Sorry to hear that.

You're probably a bit too attached to the game when you want to set up a fund for troll victims, but I do.

wuhsin · August 2013

I bet they were from Foundry authors who have quests sitting near the bottom of the list with a 2.1 average, numerous typos throughout the quest, and NPC's with numbers still in their names. Almost all of my low/mediocre ratings came from Foundry authors matching that description, I know because I searched any handle that left me a review. Sure, a few are from regular players, but typically players are easier to impress than Foundry authors, and have less incentive to give a bad rating.

antonkyle · August 2013

wuhsin wrote: »

I bet they were from Foundry authors who have quests sitting near the bottom of the list with a 2.1 average, numerous typos throughout the quest, and NPC's with numbers still in their names. Almost all of my low/mediocre ratings came from Foundry authors matching that description, I know because I searched any handle that left me a review. Sure, a few are from regular players, but typically players are easier to impress than Foundry authors, and have less incentive to give a bad rating.

I hate to think that is true. You would think that authors would understand the effect low ratings can have. I have never gave less than a four star myself and I never will. It's possible that I may have before I became an author but even then I don't think so.

Anyway, have you put your quest's back up? Was looking forward to playing the 2nd, I heard good things about it.

cipher9nemo · August 2013

Sorry to hear OP. Don't let it get you down, as I think a lot of authors get those sort of anonymous one-stars as well.

I have left a few one-star reviews but I always leave a comment to let the author know why. The last one-star review I pretty much chewed the author out for wasting my time, lol. The quest was that bad, everything, like objects just thrown around and a story that could have passed for a rug rat's writing. I didn't feel bad about that one-star because I found out the author also made an exploit map. So the only two quests they made: exploit and horrible, time-wasting map.

groshie · August 2013

I'm a bit torn when it comes to leaving one-stars.

One the one hand, if I feel a quest is wasting my time I'd much rather "ragequit" and do something better with my time and maybe send a PM to the author instead about what I felt could be improved if it's not obvious that they don't care...
And yet on the other hand, reviews also serve as a warning to other players so they won't have to waste their time on an obvious exploit/troll map.

Reviews are also extremely subjective as it is so when I think a quest is "downright unplayable" someone else might think was the best quest ever, leaving a low-star review with a nasty comment might prevent some from playing a quest they would actually like as well.
Then again, if I only left 4 or 5-star reviews then all quests would be tied on the "Best"-tab.
Just... Meh.

thalazar1 · August 2013

I don't see a need to give 1 or 2 stars. If I feel the quest was that bad i will not rate, and give a large list of things to be fixed as well as suggestions. if i really hate the quest i just drop it and don't even finish. more then enough lower stars will come from the public, with reviews like "Good quest! *****" and a few that just seem weird like "It was ok I guess *****"

My favorite is "xk *****" still not sure what that is but i have taken it under advisement and am working hard to fix it to improve my quests quality

rogu3ish · August 2013

Well, the quest only had like 11 reviews and 20 plays when those two ratings were given, so it was on the New tab but WAY down. I really doubt anyone browsed to it, so it probably wasn't due to random plays.

Meaning whoever left the reviews were playing it because they saw it in my sig, in my thread on Scribe's Enclave, or saw my Foundry Spotlight thread. Come to think of it, I believe the ratings came in shortly after I created the Spotlight thread. Of course I know not everyone will like the same things, and it's possible the ratings are legit, but it's still a bit suspicious. Oh well, not exactly something to be too bothered about.

karitr · August 2013

I read a review for a quest I played yesterday that said "I don't normally leave a comment, but this was so good I had to". The point being, that a lot of people don't bother to leave feedback and there is nothing sinister about it at all.

You say your quest was actually showing in a tab, but then conclude it wouldn't have been browsed to. Hmm, one has to wonder then why authors go to the trouble of getting on a tab at all.

Which brings me to another point I have made here in the past...if fellow authors trade 5* to help each other get on the list and those stars are not a fair reflection of the quality of the quest, then the expectations of the players who follow are not going to be met. Unfortunately, if their expectations are not met they tend to not be generous - even stupid people know when they have been conned.

A couple of authors have said to me that 3* is not a 'Good' rating and to that I say...well, what I'd say wouldn't get around the filter. Three is a good rating; if it has been diminished, then blame authors who have been less than honest when rating each other's quests. You reap what you sow.

warlordsfist · August 2013

I have never rated a Foundry mission below four stars - as an aspiring author, I appreciate all the work that goes into making an adventure.

rogu3ish · August 2013

karitr wrote: »

You say your quest was actually showing in a tab, but then conclude it wouldn't have been browsed to. Hmm, one has to wonder then why authors go to the trouble of getting on a tab at all.

There are currently two tabs where your quest might appear that a random player may find it: Best and New. The Best tab shows around 40 entries - most quests will never see that tab, and they probably shouldn't.

The New tab shows hundreds of entries, and every quest that gets beyond a few ratings and plays will eventually appear there. I conclude it hadn't been browsed to because there were at least a hundred, maybe more quests above it in the list. Anyone browsing to find a quest to play almost certainly wouldn't look that far down unless they just drag the slider to a random location.

Which brings me to another point I have made here in the past...if fellow authors trade 5* to help each other get on the list and those stars are not a fair reflection of the quality of the quest, then the expectations of the players who follow are not going to be met. Unfortunately, if their expectations are not met they tend to not be generous - even stupid people know when they have been conned.

A couple of authors have said to me that 3* is not a 'Good' rating and to that I say...well, what I'd say wouldn't get around the filter. Three is a good rating; if it has been diminished, then blame authors who have been less than honest when rating each other's quests. You reap what you sow.

I agree that simply rating 5 stars on a quest to 'support' fellow authors can be a disservice. I don't operate that way. But I wouldn't argue in the current adjusted rating system that 3 is a "good" rating. Three star (and below) ratings appear to have a stronger impact on adjusted rating than 5 star ratings. Wushin has gone on at length about this in other threads. I actually don't think it matters that much, because every quest is subjected to the same system.

This also goes back to the original point of the likelihood that a quest will be browsed to, though. I'm sure the average player doesn't scroll very far down the New list to find something interesting to play, and there are a very limited number of spots on the Best tab. Both lists are sorted solely by adjusted rating right now, so it's definitely possible to "punish" an author you don't like by leaving intentionally poor ratings. I'm not saying that's happened to me (I can't imagine I've impacted anyone enough in this community for them to want to punish me), but it's a legitimate concern for those who are visible and put forth a lot of effort to get their work noticed.

groshie · August 2013

karitr wrote: »

...if fellow authors trade 5* to help each other get on the list and those stars are not a fair reflection of the quality of the quest, then the expectations of the players who follow are not going to be met.

This is true, but there's also the other side of the coin - I'm talking about the troll reviews that seem to be flourishing.
It could of course be coincidence, but I find it weird that so many authors seem to recognize the problem and see a certain systematic pattern in their reviews.

karitr wrote: »

A couple of authors have said to me that 3* is not a 'Good' rating and to that I say...well, what I'd say wouldn't get around the filter. Three is a good rating; if it has been diminished, then blame authors who have been less than honest when rating each other's quests.

Hence me being a bit torn about the ratings.
Many authors do believe a 3 is a low rating (in my world it represents an "ok"), but so do many players who give them. I don't think you should place the blame only on the authors here, no one interprets the rating system the same and just as some authors (and players) may upvote a quest made by a guildie or a friend, there are most likely people who'll downrate out of spite or just for a personal laugh.

cipher9nemo · August 2013

warlordsfist wrote: »

I have never rated a Foundry mission below four stars - as an aspiring author, I appreciate all the work that goes into making an adventure.

Then you haven't seen some of the trolling, silly trash out there. If an author spends enough time to post on the forums and promote their quest, chances are it's at least worthwhile. But there are plenty who just don't care, and it shows in their UGC. Worse are the exploiters.

As for pretty much anyone here on the forums, I've never rated their quests lower than may be a 3. And if it's review trading or supporting authors asking for help I pretty much always rate them 5 even if they're not 5 quality. Because that's supporting our community. Not everyone does that.

drakesigar · August 2013

groshie wrote: »

The people who apparently do that sort of thing would leave 1-stars though, right? For maximum impact?

We just assume anything lower than a 4 round ere is a communist plot. Better safe than sorry.

ananvilhurtz · August 2013

I will give mediocre quests that would normally rate 3/5 or so a 5/5 if I see 1 star reviews that I don't think are deserved.

I'll mention that in the review, though.

rogu3ish · August 2013

cipher9nemo wrote: »

As for pretty much anyone here on the forums, I've never rated their quests lower than may be a 3. And if it's review trading or supporting authors asking for help I pretty much always rate them 5 even if they're not 5 quality. Because that's supporting our community. Not everyone does that.

Between the 'official' review thread here, individual review requests, and Scribe's Enclave, I try to play between 2 and 4 different foundries every day by new authors, and I've seen some pretty rough stuff. I think the worst I've ever given was a three, and that's really only on the spreadsheet maintained by ash4ll because they were so bad or broken I couldn't actually complete them to leave an in-game review. Looking back over my history, though, I do tend to give out mostly 4s. I prefer to reserve 5s for quests that I think are particularly good.

Personally I think a 3/5 is a bad rating, meaning there were one or more pretty negative aspects. A one or two would be for a terrible or broken quest. Based on other review sites using the same scale, I think I'm pretty average in that point of view. How many people go to Amazon, for example, and say to themselves "Oh, that thing has an average 3 star rating, I bet it'll be good".

karitr · August 2013

rogu3ish wrote: »

There are currently two tabs where your quest might appear that a random player may find it: Best and New. The Best tab shows around 40 entries - most quests will never see that tab, and they probably shouldn't.

The New tab shows hundreds of entries, and every quest that gets beyond a few ratings and plays will eventually appear there. I conclude it hadn't been browsed to because there were at least a hundred, maybe more quests above it in the list. Anyone browsing to find a quest to play almost certainly wouldn't look that far down unless they just drag the slider to a random location.

I agree that simply rating 5 stars on a quest to 'support' fellow authors can be a disservice. I don't operate that way. But I wouldn't argue in the current adjusted rating system that 3 is a "good" rating. Three star (and below) ratings appear to have a stronger impact on adjusted rating than 5 star ratings. Wushin has gone on at length about this in other threads. I actually don't think it matters that much, because every quest is subjected to the same system.

This also goes back to the original point of the likelihood that a quest will be browsed to, though. I'm sure the average player doesn't scroll very far down the New list to find something interesting to play, and there are a very limited number of spots on the Best tab. Both lists are sorted solely by adjusted rating right now, so it's definitely possible to "punish" an author you don't like by leaving intentionally poor ratings. I'm not saying that's happened to me (I can't imagine I've impacted anyone enough in this community for them to want to punish me), but it's a legitimate concern for those who are visible and put forth a lot of effort to get their work noticed.

I take your word for it that it is 100s. When looking for new quests to play from the catalogue, I often scroll immediately to the bottom and then work my way up, so I probably haven't noticed it was so many.

As for the adjusted rated system, while I am not invested enough to go into how it works too deeply, I thought it adjusted all quests to a median of three until they have achieved X number of plays. Now forgive me because, as I say I haven't looked too deeply into what this means in actuality, so could well be wrong, but if this is true then wouldn't that explain why receiving 5* ratings isn't having the impact many - including Wuhsin - thinks it should have?

It would be interesting for those who have 100s or 1000s of plays, to let us know what, if any, difference there is between their average rating and their adjusted rating. There must be some point where these two numbers meet.

groshie wrote: »

This is true, but there's also the other side of the coin - I'm talking about the troll reviews that seem to be flourishing.
It could of course be coincidence, but I find it weird that so many authors seem to recognize the problem and see a certain systematic pattern in their reviews.

Hence me being a bit torn about the ratings.
Many authors do believe a 3 is a low rating (in my world it represents an "ok"), but so do many players who give them. I don't think you should place the blame only on the authors here, no one interprets the rating system the same and just as some authors (and players) may upvote a quest made by a guildie or a friend, there are most likely people who'll downrate out of spite or just for a personal laugh.

groshie, I don't doubt there are jealous authors or outright nutcases who will 1* bomb quests, but it seems to me that by trying to counter those trolls, or beat the system, or even just be kind, a plethora of 5 stars can create the type of backlash you're trying to avoid. And of course that particularly hurts when a quest hasn't had enough plays to act as a buffer.

If we had a database that allowed us to see all quests, I would have no hesitation in trying out quests with an average of three stars. To me that is a good rating and as I prefer the longer quests, the ones that get down-rated for length, I would even expect my preferred quests to be around the 3* mark rather than pushing those upper figures.

ananvilhurtz wrote: »

I will give mediocre quests that would normally rate 3/5 or so a 5/5 if I see 1 star reviews that I don't think are deserved.

I'll mention that in the review, though.

Yeah, I have seen 5* ratings with comments such as "Here supporting a fellow author". I appreciate the honesty in those comments, but take umbrage at the term 'mediocre' for three stars

But there you go, how we rate as individuals is subjective.

groshie · August 2013

karitr wrote: »

It would be interesting for those who have 100s or 1000s of plays, to let us know what, if any, difference there is between their average rating and their adjusted rating. There must be some point where these two numbers meet.

No idea about this one actually.
All my quests (my most played quest has over 100 plays and about 85 ratings) have an average rating of around 4.5 if I check in the Foundry, but their adjusted rating is still around 3 - 3.5.

cipher9nemo · August 2013

rogu3ish wrote: »

Personally I think a 3/5 is a bad rating, meaning there were one or more pretty negative aspects. A one or two would be for a terrible or broken quest. Based on other review sites using the same scale, I think I'm pretty average in that point of view. How many people go to Amazon, for example, and say to themselves "Oh, that thing has an average 3 star rating, I bet it'll be good".

I personally rate a little differently than that. I wouldn't rate a quest lower than three just because something was broken. Instead I'd mention it in the review or a PM to the author and give them a chance to fix it. So it really depends on the overall quality for me. Even if the quest was a little amateurish, if you could at least tell they put a lot of love into it, then it gets a higher rating from. If the quest was just thrown together and lacking that love, I'd rate lower.

Some of those quest you can almost hear a kid behind their PC going "this is so cool, what else can I throw in here to mess with the player?". In contrast, at other times, you can sense that the author really cared about about player experience.

groshie · August 2013

karitr wrote: »

If we had a database that allowed us to see all quests, I would have no hesitation in trying out quests with an average of three stars. To me that is a good rating and as I prefer the longer quests, the ones that get down-rated for length, I would even expect my preferred quests to be around the 3* mark rather than pushing those upper figures.

I fully agree with that we would need some sort of new system to view and/or rate Foundry quests.
I hope Cryptic is working on *something*, it would really make a great game even more awesome in my opinion.

karitr · August 2013

odd character causing my brain to mush. See below

karitr · August 2013

rogu3ish wrote:

How many people go to Amazon, for example, and say to themselves "Oh, that thing has an average 3 star rating, I bet it'll be good".

You can't compare. Amazon customers can only rate if a comment is left and those comments give clarity and context to the rating. If this is an aspect that I think won't bother me, then yes I am more than happy to buy. Heck, my most recent purchase was a 500 pounds sterling (it didn't like the pound sign :mad:) mixer than had been down-rated because the dough hooks kept breaking. Newsflash: they haven't built a mixer yet that can handle dough, that's why we have bread-making machines.

Amazon also moderate reviews and removes troll ratings. The Foundry has to rely on the adjusted ratings system to balance trolls at both ends of the scale and as far as I know, that is all the check it gets.

karitr · August 2013

groshie wrote: »

No idea about this one actually.
All my quests (my most played quest has over 100 plays and about 85 ratings) have an average rating of around 4.5 if I check in the Foundry, but their adjusted rating is still around 3 - 3.5.

Wonder if we apply thumb screws and bright lights, would the authors pushing the 4.4s on the Best list spill the goods.

wuhsin · August 2013

antonkyle wrote: »

Anyway, have you put your quest's back up? Was looking forward to playing the 2nd, I heard good things about it.

I've been thinking about it. I need to fix the flicker on the walls in it first.

thalazar1 · August 2013

karitr wrote: »

As for the adjusted rated system, while I am not invested enough to go into how it works too deeply, I thought it adjusted all quests to a median of three until they have achieved X number of plays.

so far as i could tell the quest in editor will have an average which is a true average of the stars
3x 5*
1x 4*
1x 3*
22 total stars / number of reviews is an average of 4.4, the adjusted average seems to take into account plays and something else that i cant quite place.
so same scores but 10 plays would be 22 total stars / total plays which is 2.2 adjusted. minus one play (authors play through) would be 2.44

This isnt exactly right but its the closest i have come to figuring out the adjusted rating. though when i try this math on my own quests its off by 0.01 to 0.2 so there is something else in there that also affects it.

Question about Reviews

Comments