There's nothing you can do about that, short of demanding that everyone who signs the ELUA for the Foundry provides proof of age or something of that nature.
Cryptic will cover their collective backsides by stating that anything seen in the Foundry may not be suitable to a T rated game and using the system is at your own risk.
The funny thing about the EULA is that its meaningless for minors anyway. A minor cant enter into a legal contract, so if they DO "sign" it its automatically void and doesnt "protect" Cryptic from anything. That said, its not like their going to get sued or anything. Any minor that is playing has to have done so with their parents consent(to pay for the monthly sub at least), and its the parents responsibility to investigatge the product.
Per the Devs, the purpose of the review system is to weed out inappropriate content, not judge how "good" the mission is.
It's both actually.
A mission can go though the testing system and come out as a open to everyone mission, with a rating of 1 star, because it's not very good.
You have two different things happening by playtesters, one is completing the mission, once it's been completed X times, it's pushed to live. This allows missions to be checked for inappropriate material before it's open to the public.
The other is rating it for how good or bad the mission is, but this is also done by the playtesters before the mission is made open to the public.
Get a three flag system as Nagus suggests, make everyone sign a EULA to play UGC, regardless, and be done with it.
That's not how Cryptic wants to do it however. I also think it's a bad idea.
Why should anyone be forced to see something they'd rather not? If someone doesn't want to take the risk of seeing something inappropriate then they should have that option.
The goals of play testing is to complete the mission. If the mission is completed, the reviewer is given the chance to rate the mission 1-5 stars (and possibly add comments).
What if the mission cannot be completed? How should it be processed? As a playtester can only vote after they have completed the mission, is there a clause in the Terms of Use doc to allow flagging of missions that cannot be?
If so, are there any checks and balances in place to differentiate between impossible missions (Target objective hidden behind terrain glitches), and merely difficult missions (clicking six consoles in an order determined by mission text without spamming Zone chat for help.)?
I agree with your points about flagging. However, I don't know if people shouldn't be able to rate it based on content as well. Yes, the main purpose of play testing is to filter inappropriate stuff. However, if some of my fleetmates want to play and rate my mission, they shouldn't have to wait for 10 other, random people to play test it for appropriateness first. Sure, they could experience it, and tell me "boy that was great!", but they'd have to play it again for the rest of the community to see their 5-star rating.
I'll concede the voting issue. However, I do think there needs to be multiple flags required to remove a mission from testing, as I explained.
We will take flagging content very seriously as reporting missions in this manner can lead to a mission being pulled from the game until the author addresses the complaint.
If your mission is flagged for a genuine reason, the author is made aware of the reason for their mission getting flagged. The author can then edit their mission and remove the content which violates the EULA. If your mission is flagged for no reason that violates the EULA, then the person who flagged your mission may or may not get disciplined (TBD), depending on the circumstances.
P.S. I am happy to see that your concerns have been addressed.
you say the rating, which seems to be a simple 1 - 5 stars like / not like button doesnt determine if the mission makes it into the game?
So if say 20 Players are needed to play the mission before it gets into the game and all those 20 People say it's TRIBBLE and vote 1 star it still gets into the game?.... WHY?
i believe you can also comment on missions for people to read as well to make it more clear to people who decide to play it or not.
the reason 1 star missions will get in is because everybody has the right to have their mission published as long as its not breaking any rules. if you dont then want to play a 1 star mission that is of course your choice, you dont have to but likes and dislikes are personal preferences. someone else might play that mission and think its the best mission ever because that is their choice.
i believe you can also comment on missions for people to read as well to make it more clear to people who decide to play it or not.
the reason 1 star missions will get in is because everybody has the right to have their mission published as long as its not breaking any rules. if you dont then want to play a 1 star mission that is of course your choice, you dont have to but likes and dislikes are personal preferences. someone else might play that mission and think its the best mission ever because that is their choice.
Then how about publishing everyone's vote history? If you find yourself liking a mission that someone voted five stars, even though the average score is lower, then you might like other missions they vote for.
Then how about publishing everyone's vote history? If you find yourself liking a mission that someone voted five stars, even though the average score is lower, then you might like other missions they vote for.
im not sure if thats easy to set up or not but im all for improving the way to search missions through different criteria.
i assume you mean something like 'find missions rated highly by this player' or something?
If the Community takes to User Generated Content, then there's going to be a lot of missions to choose from. Most of them will, unfortunately, be test missions and simple single-room-full-of-hostile-NPCs maps.
So we need a way to separate the wheat from the chaff.
Others have suggested that Cryptic appoint some kind of 'Advisory Council' to judge which User Content is worthy. This is a very dangerous path to go down. As proven a few months ago, the Community doesn't like being told what it's supposed to like. If a 'first among equals' board is implemented, then the possibilities offered by UGC will be restricted by politics and favouritism, and overwhelmed by volume.
I would rather there be anarchy, with the Community creating anything they like within the confines of the UGC toolkit. I have the right to say that I like other players' content, and I have the right to say that I don't like it. That is the limit of my influence.
In the rare case where someone else thinks I have impeccable taste, they should be able to see what content I like and dislike. That way, each player can publish their recommendations to others.
This can be formalised by allowing players to group missions together under Categories, creating catalogues of content.
Categories can be both global and personal. When a mission is created, the author could place it under a Global category of 'Space Missions' or 'Ground Missions', for example. That way, another player can look through these categories and find what type of mission they want to play.
If a player feels that an author has mis-categorized their content (e.g. putting a Ground map in a Space mission), then the player can choose to block that mission, or the author, from appearing to them in the catalogue. (Only in extreme EULA-breaking cases should an author be banned from UGC altogether.)
Personal Categories are more fun. These can be created by individual players and can be named anything. The player can then place any content they wish into these personal categories, be it missions they've created themselves, or missions they've discovered in the wild.
Personal Categories can then be subscribed to by other players. So if a player creates a category like "Experimental Storytelling", others can subscribe to it and be notified when new content is added.
Another player could create a Personal Category such as "LOLCATS ROOL LULZ!!!!111!!!", and be perfectly within their rights to do so, without an Advisory Council deeming their perspective irrelevant.
This also leaves the door open to create Competition Categories, run by both Cryptic and the Community. A player could suggest a theme to create against, and offer an in-game prize to whoever 'wins'. Entries to these competitions can be added to a Category that others can subscribe to and vote on.
I feel that putting the ability to filter content into the hands of the community will make for a better experience than allowing a demagogic cabal to rule the show.
(Presented unedited and with the appreciation that the scope and purpose of UGC has changed in the interim.)
It has nothing to do with someone being an "AH" as you so maturely put it. Per the Devs, the purpose of the review system is to weed out inappropriate content, not judge how "good" the mission is. However, by combining the two, it increases the chance of people, either intentionally or simply not knowing any better, combining the two into one opinion. Its better to simply avoid the issue when its not a necessary component of the system at that point.
I see no c hance for confusion. The "flagging" thing doesn't sound like "hey, I didn't like this" but "Mission content uses likeness of Spock naming him Spuck." "Swear Words".
I have a little more trust in a reviewer to distinguish between "Vulcan should never laugh" and "Sexually explicit dialogue". The first affets the rating, the second sets a flag.
Do we know the policy on using pre-existing characters? There have been a few examples already of people using, for example, Spock or 7 of 9. Surely that's not against the rules? How could it be?
Will the content created by /ignored players be visible to those who did the /ignoring?
That is, let's say Player A has put Player B on /ignore for whatever reason -- something in the forums, something in the game, whatever. (I am given to understand the two are tied together.) Player A jumps into the game to play some UGC.
If Player A did NOT sign up to review UGC, will he see Player B's missions?
If Player A DID sign up to review UGC, will he see Player B's missions?
Will Player B show up in any filtered search or 'browsing' for UGC -- explicit or implicit -- done by Player A?
Let's turn it around. Let's say Player B is playing some UGC and plays one of Player A's missions. (Player A still has Player B on /ignore.) If Player B comments on Player A's UGC, will Player A see those comments?
Do we know the policy on using pre-existing characters? There have been a few examples already of people using, for example, Spock or 7 of 9. Surely that's not against the rules? How could it be?
CBS might have a different view on that.
What CBS's policy will be remains to be seen. At the most extreme end, it is entirely possible that players will not be allowed to make use of any planet, ship, or person mentioned in any of the TV series, movies, books, or other licensed works. It is likely not to be that strict, but I do not think that CBS will allow players to create missions with any of the major cast members or prominent ships (like Defiant, Enterprise (any letter), Excelsior, Voyager.) There will also likely be all sorts of other caveats, that any UGC becomes the property of Cryptari, may not contradict existing canon, may not alter origin stories, may not involve certain time periods or events*, may not be republished in any form, etc. etc. etc. *burp* legalaase.
In short, considering the hoops that professional Trek book writers go through to write in the Trek universe, CBS is not going to let a random person off the Internet have their merry wicked way with any major/regular Trek character.
Sure, you can complain about it. But the complaints have to go to CBS, and I doubt that the CBS 'zecks and their army of barely-domesticated Briefcase-Armed Assault Lawyers, Esq., will listen to you or I. (Heck, I would say that the more people who clamor for us to be allowed to use a particular element, the more CBS will say 'no' on the grounds of Rule 34 and oh hell no).
* - For example, considering the Abramsverse Trek films, I can see CBS saying 'BigNO!' to any mission set in TOS. Might not make sense to us, but I can see them doing that.
I would have thought that this is where we'd create/access player-made content. It makes a lot of sense and would be a great way to add some functionality to ship interiors. Hopefully they'll put it there in the future.
He already said in STO general that it should be a part of the Foundry upon release. a moment to find it:
What CBS's policy will be remains to be seen. At the most extreme end, it is entirely possible that players will not be allowed to make use of any planet, ship, or person mentioned in any of the TV series, movies, books, or other licensed works. It is likely not to be that strict, but I do not think that CBS will allow players to create missions with any of the major cast members or prominent ships (like Defiant, Enterprise (any letter), Excelsior, Voyager.) There will also likely be all sorts of other caveats, that any UGC becomes the property of Cryptari, may not contradict existing canon, may not alter origin stories, may not involve certain time periods or events*, may not be republished in any form, etc. etc. etc. *burp* legalaase.
In short, considering the hoops that professional Trek book writers go through to write in the Trek universe, CBS is not going to let a random person off the Internet have their merry wicked way with any major/regular Trek character.
Sure, you can complain about it. But the complaints have to go to CBS, and I doubt that the CBS 'zecks and their army of barely-domesticated Briefcase-Armed Assault Lawyers, Esq., will listen to you or I. (Heck, I would say that the more people who clamor for us to be allowed to use a particular element, the more CBS will say 'no' on the grounds of Rule 34 and oh hell no).
* - For example, considering the Abramsverse Trek films, I can see CBS saying 'BigNO!' to any mission set in TOS. Might not make sense to us, but I can see them doing that.
I do believe there will be an "approved use of likeness" list
I would suggest that the people who are testing the mission orginally dont even get to rate it, as that is not the purpose of the review per your comments. The purpose of the review is only to determine whether the mission contains inappropriate content or not. That being the case, what the person thinks of the mission really doesnt matter at that point. This will allow missions that make it past the review and go live to do so with a "fresh start", without any positive or negative ratings attatched to them.
A second suggestion would be that a mission not be removed from testing unless it receives 3 flags. For example, if the number of times it is required to be played before going live is 10, then it would seem reasonable that if the mission were REALLY inappropriate that it would be flagged as such at least 3 times if not more. This will prevent missions from getting removed from testing that ARENT really inappropriate, due to someone flagging them who doesnt know what their really supposed to be doing.
Other than these two things, I think the proposed system sounds pretty good and look forward to testing
I like these suggestions. From DStahl's latest I get the feeling Reviewers will have a drop-down box of reasons to flag a mission instead of an open ended "Your mission isn't canon, I'm denying it".
Making sure missions are flagged by multiple Reviewers before taking it down is also much better.
We'll have to see how it goes. At this point, I think not having enough Reviewers may be a bigger problem than having Rogue Reviewers.
I REALLY hope that CBS does not take the draconian view that you described. Let us have a little fun with it! If I want people to be able to time travel and hang out with Picard and crew, where's the harm? If I keep them all in character, and don't do anything inappropriate, no harm no foul, right?
I guess we'll just have to wait and see what the policy is... but if a dev could comment, either generally or even on my specific TNG time travel question, that would be the preferable option.
I REALLY hope that CBS does not take the draconian view that you described. Let us have a little fun with it! If I want people to be able to time travel and hang out with Picard and crew, where's the harm? If I keep them all in character, and don't do anything inappropriate, no harm no foul, right?
I guess we'll just have to wait and see what the policy is... but if a dev could comment, either generally or even on my specific TNG time travel question, that would be the preferable option.
I agree with you but it's already too late. DStahl has already said Reviewers are also responsible for flagging content for Actor Likenesses and anything else that CBS does not want:
There will be a Foundry Terms of Use Doc that outlines rules that must be adhered to when making content. This will include CBS guidelines for what can and can't be used, as well as specific things that are deemed not-allowable and cause for Flagging.
This Flagging will not be used for things such as "I didn't like this mission" or "This sucks" or "This doesn't feel Star Trek to me"... but instead for things that violate IP license rules (example: using a character or likeness that is on the not allowed list), violating standard terms of service rules (example: advertising other products or services in your mission text), or attempting to circumvent foul language filters.
I wish this were true, but I don't think it is. No system will ever be full proof enough to keep greifers and voting cartels from keeping missions at the bottom of the pile. While the search engine will return results on whatever criteria they put into it, most still won't play a low rated mission, regardless of how good it actually is.
I think you overestimate the power a block or cartel would have in a game of this size, and if they only allow a player to rate once, their voice can easily get drowned out.
I REALLY hope that CBS does not take the draconian view that you described. Let us have a little fun with it! If I want people to be able to time travel and hang out with Picard and crew, where's the harm? If I keep them all in character, and don't do anything inappropriate, no harm no foul, right?
I guess we'll just have to wait and see what the policy is... but if a dev could comment, either generally or even on my specific TNG time travel question, that would be the preferable option.
Well, I hope we find out sooner rather than later! I'd be surprised if CBS hasn't made their requirements known to Cryptic by now, and it may be helpful for us to see what those requirements are.
short and sweet of it is if your in a big fleet were you all play each others missions then your missions will get played more etc. Nice that people will need to join fleets agian
Well, I hope we find out sooner rather than later! I'd be surprised if CBS hasn't made their requirements known to Cryptic by now, and it may be helpful for us to see what those requirements are.
I agree. And I certainly hope likenesses aren't limited in some way by what rights the actors have or have not decided to give CBS. This whole thing is rather worrisome...
short and sweet of it is if your in a big fleet were you all play each others missions then your missions will get played more etc. Nice that people will need to join fleets agian
Can you explain what you mean? As all missions go public to all players, at least that is what Cryptic has said so far, it has no impact on belonging to a fleet or not. If Cryptic on the other had made it possible to publish a mission for fleet members only then there would be some truth to it.
I think what the poster means is, if I'm in a fleet of say 50 people, I can post a link to my mission on the fleet board and trust that my fleet mates will make a point of playing it. I can also in theory count on my fleet members to rate it highly... Which pushes it up the list when people look at mission ratings.
More people are likely to play a 5 star mission reviewed by 15 people then a 2 star mission reviewed by 3.
If however I'm not in a fleet, I have to count on enough random people to play it to get it promoted to 'public' status.
Of course I can post here, or some other place like the UGC wiki thingie someone is working on and trust people there will see my post and try my mission... So I don't see how being part of a fleet will help much, other then giving you another source of testers.
Well the UGC writers do have the option to blaze thier own path.
Make your own iconic NPCs...the OTHER important folks never mentioned in current star trek lore.
There is more than 1 Q.....you don't need THE Q, you can use a different Q...there are several members
of that race...all have thier own agenda.
Make up your own stuff...keep it Trek, and build on it.
Maybe on earth there is a romulan underground working with Federation Intel branch that
exchange favors for...a secret place to plot the return of the true romulan royal family....
maybe there is a special secret taskforce to track down orion pirates
YOU create the epic NPCs, secret departments, rogue scientists ect.. Then base all your adventures
with the understanding that they exist...in your version of things.
The Federation is so big, not even the Federation Pres. would know half of what goes on behind
closed doors.
Why wouldn't branch 13 handle this ? We are more focused and we are apart from them, they have
other ...less pressing matters to deal with
I have never heard of the Klingon/Vulcan timeline intervension agency......Good....we prefer it that way.
Of course I can post here, or some other place like the UGC wiki thingie someone is working on and trust people there will see my post and try my mission... So I don't see how being part of a fleet will help much, other then giving you another source of testers.
Question, which has been mentioned by players but I have not seen a dev comment on it yet:
Will UGC special-EULA-signing play testers be receiving in-game currency rewards like EC or XP? On the one hand, this would prevent people from skimming through the special EULA just to get to UGC faster. On the other hand, it would make reviewing more like an actual job, and would also force testers to replay all the missions they had reviewed to get rewards.
I am SO looking forward to this being implemented!
While there are some good ideas and suggestions put forth in these forums, I have faith that Cryptari will use wisdom in setting up the review and rating system before all is said and done.
Comments
The funny thing about the EULA is that its meaningless for minors anyway. A minor cant enter into a legal contract, so if they DO "sign" it its automatically void and doesnt "protect" Cryptic from anything. That said, its not like their going to get sued or anything. Any minor that is playing has to have done so with their parents consent(to pay for the monthly sub at least), and its the parents responsibility to investigatge the product.
It's both actually.
A mission can go though the testing system and come out as a open to everyone mission, with a rating of 1 star, because it's not very good.
You have two different things happening by playtesters, one is completing the mission, once it's been completed X times, it's pushed to live. This allows missions to be checked for inappropriate material before it's open to the public.
The other is rating it for how good or bad the mission is, but this is also done by the playtesters before the mission is made open to the public.
That's not how Cryptic wants to do it however. I also think it's a bad idea.
Why should anyone be forced to see something they'd rather not? If someone doesn't want to take the risk of seeing something inappropriate then they should have that option.
What if the mission cannot be completed? How should it be processed? As a playtester can only vote after they have completed the mission, is there a clause in the Terms of Use doc to allow flagging of missions that cannot be?
If so, are there any checks and balances in place to differentiate between impossible missions (Target objective hidden behind terrain glitches), and merely difficult missions (clicking six consoles in an order determined by mission text without spamming Zone chat for help.)?
I'll concede the voting issue. However, I do think there needs to be multiple flags required to remove a mission from testing, as I explained.
The following was stated in the OP, which suggests (but doesn't confirm) that authored missions will not be pulled after being flagged just once.
If your mission is flagged for a genuine reason, the author is made aware of the reason for their mission getting flagged. The author can then edit their mission and remove the content which violates the EULA. If your mission is flagged for no reason that violates the EULA, then the person who flagged your mission may or may not get disciplined (TBD), depending on the circumstances.
P.S. I am happy to see that your concerns have been addressed.
Is this done by the system or by some poor Cryptic employee? How long is the wait?
i believe you can also comment on missions for people to read as well to make it more clear to people who decide to play it or not.
the reason 1 star missions will get in is because everybody has the right to have their mission published as long as its not breaking any rules. if you dont then want to play a 1 star mission that is of course your choice, you dont have to but likes and dislikes are personal preferences. someone else might play that mission and think its the best mission ever because that is their choice.
Then how about publishing everyone's vote history? If you find yourself liking a mission that someone voted five stars, even though the average score is lower, then you might like other missions they vote for.
im not sure if thats easy to set up or not but im all for improving the way to search missions through different criteria.
i assume you mean something like 'find missions rated highly by this player' or something?
That would work for starters, taking into account that people have different tastes.
Or as I posted about five months ago:
(Presented unedited and with the appreciation that the scope and purpose of UGC has changed in the interim.)
I have a little more trust in a reviewer to distinguish between "Vulcan should never laugh" and "Sexually explicit dialogue". The first affets the rating, the second sets a flag.
Will the content created by /ignored players be visible to those who did the /ignoring?
That is, let's say Player A has put Player B on /ignore for whatever reason -- something in the forums, something in the game, whatever. (I am given to understand the two are tied together.) Player A jumps into the game to play some UGC.
Let's turn it around. Let's say Player B is playing some UGC and plays one of Player A's missions. (Player A still has Player B on /ignore.) If Player B comments on Player A's UGC, will Player A see those comments?
CBS might have a different view on that.
What CBS's policy will be remains to be seen. At the most extreme end, it is entirely possible that players will not be allowed to make use of any planet, ship, or person mentioned in any of the TV series, movies, books, or other licensed works. It is likely not to be that strict, but I do not think that CBS will allow players to create missions with any of the major cast members or prominent ships (like Defiant, Enterprise (any letter), Excelsior, Voyager.) There will also likely be all sorts of other caveats, that any UGC becomes the property of Cryptari, may not contradict existing canon, may not alter origin stories, may not involve certain time periods or events*, may not be republished in any form, etc. etc. etc. *burp* legalaase.
In short, considering the hoops that professional Trek book writers go through to write in the Trek universe, CBS is not going to let a random person off the Internet have their merry wicked way with any major/regular Trek character.
Sure, you can complain about it. But the complaints have to go to CBS, and I doubt that the CBS 'zecks and their army of barely-domesticated Briefcase-Armed Assault Lawyers, Esq., will listen to you or I. (Heck, I would say that the more people who clamor for us to be allowed to use a particular element, the more CBS will say 'no' on the grounds of Rule 34 and oh hell no).
* - For example, considering the Abramsverse Trek films, I can see CBS saying 'BigNO!' to any mission set in TOS. Might not make sense to us, but I can see them doing that.
He already said in STO general that it should be a part of the Foundry upon release. a moment to find it:
I do believe there will be an "approved use of likeness" list
I like these suggestions. From DStahl's latest I get the feeling Reviewers will have a drop-down box of reasons to flag a mission instead of an open ended "Your mission isn't canon, I'm denying it".
Making sure missions are flagged by multiple Reviewers before taking it down is also much better.
We'll have to see how it goes. At this point, I think not having enough Reviewers may be a bigger problem than having Rogue Reviewers.
I REALLY hope that CBS does not take the draconian view that you described. Let us have a little fun with it! If I want people to be able to time travel and hang out with Picard and crew, where's the harm? If I keep them all in character, and don't do anything inappropriate, no harm no foul, right?
I guess we'll just have to wait and see what the policy is... but if a dev could comment, either generally or even on my specific TNG time travel question, that would be the preferable option.
I agree with you but it's already too late. DStahl has already said Reviewers are also responsible for flagging content for Actor Likenesses and anything else that CBS does not want:
I think you overestimate the power a block or cartel would have in a game of this size, and if they only allow a player to rate once, their voice can easily get drowned out.
Well, I hope we find out sooner rather than later! I'd be surprised if CBS hasn't made their requirements known to Cryptic by now, and it may be helpful for us to see what those requirements are.
I agree. And I certainly hope likenesses aren't limited in some way by what rights the actors have or have not decided to give CBS. This whole thing is rather worrisome...
Can you explain what you mean? As all missions go public to all players, at least that is what Cryptic has said so far, it has no impact on belonging to a fleet or not. If Cryptic on the other had made it possible to publish a mission for fleet members only then there would be some truth to it.
I think what the poster means is, if I'm in a fleet of say 50 people, I can post a link to my mission on the fleet board and trust that my fleet mates will make a point of playing it. I can also in theory count on my fleet members to rate it highly... Which pushes it up the list when people look at mission ratings.
More people are likely to play a 5 star mission reviewed by 15 people then a 2 star mission reviewed by 3.
If however I'm not in a fleet, I have to count on enough random people to play it to get it promoted to 'public' status.
Of course I can post here, or some other place like the UGC wiki thingie someone is working on and trust people there will see my post and try my mission... So I don't see how being part of a fleet will help much, other then giving you another source of testers.
Make your own iconic NPCs...the OTHER important folks never mentioned in current star trek lore.
There is more than 1 Q.....you don't need THE Q, you can use a different Q...there are several members
of that race...all have thier own agenda.
Make up your own stuff...keep it Trek, and build on it.
Maybe on earth there is a romulan underground working with Federation Intel branch that
exchange favors for...a secret place to plot the return of the true romulan royal family....
maybe there is a special secret taskforce to track down orion pirates
YOU create the epic NPCs, secret departments, rogue scientists ect.. Then base all your adventures
with the understanding that they exist...in your version of things.
The Federation is so big, not even the Federation Pres. would know half of what goes on behind
closed doors.
Why wouldn't branch 13 handle this ? We are more focused and we are apart from them, they have
other ...less pressing matters to deal with
I have never heard of the Klingon/Vulcan timeline intervension agency......Good....we prefer it that way.
ECT...
Exactly what I was thinking.
Will UGC special-EULA-signing play testers be receiving in-game currency rewards like EC or XP? On the one hand, this would prevent people from skimming through the special EULA just to get to UGC faster. On the other hand, it would make reviewing more like an actual job, and would also force testers to replay all the missions they had reviewed to get rewards.
While there are some good ideas and suggestions put forth in these forums, I have faith that Cryptari will use wisdom in setting up the review and rating system before all is said and done.
Make it so!