Rating system needs a rethink

It's been bugging me for years that the ratings system is not quite right.

Here is a rating example from one of my own griddlers...

0.00% 25.00% 50.00% 75.00% 100.00%
1 1 1 12 16 83.06% current rating

0 1 1 12 16 85.83% rating prior to 0% award
1 1 1 12 22 85.81% 6 100%'s needed to get near to previous rating

so in the example that means that one person giving a 0% vote can only be countered by 6 people giving a 100% vote which seems a bit extreme to me.

I don't have any ideas for a better system but any suggestions would be welcomed

Jingai

I have a problem with the 0% category in general. I've never used it simply because no published puzzle is worth zero. And I rarely vote 25%.

But since you brought it up, I would like to see more granularity in the vote. There's a lot of territory between 75% and 100% for example. Some puzzles are strong 80% or 90%, but I have to compromise to vote. How about a 10% scale, without 0%? If you think it's worth zero then don't vote.

I wouldn't like too much "granularity" as mudshark proposes, but I certainly agree that eliminating 0 would curb the nasty effects on high-rated puzzles.

With a minimal change (preserving the five rating categories) the problem could be mitigated, say, rating the five boxes 1,2,3,4,5 and thus making the minimum score 1 and not 0. This is the system in place at major websites such as Netflix.com (for rating movies), where 1 = "Hated it," 2= "Didn't like it," 3 = Liked it," 4="Really liked it," 5="Loved it." Maybe "word categories" (unlike the percentages we have now) would be easier for the voters to pick.

Still, it is a general fact of life that a lowest outlier rating spoils the fun. Even with the system described above a 1 followed by a 5 is still equivalent to two 3 scores; a 1 followed by three 5 is equivalent to four 4 scores, and thus it takes more and more fives to stick to the highest (4-5) tier, though not as many as if the low damaging score were a 0.

I usually don't have faith in most puzzlers votes and when it comes to voting, the more options available the more judgmental one usually becomes. However if someone usually votes 75% when they really wanted to give an 80% or 90%, no author would argue against a 10% scale. Another positive I see would be that more people would probably vote and hopefully stabilize the votes to reflect a puzzle's true worth. I have seen far too many puzzles that are not difficult or are extremely uncreative have higher than expected scores as well as great puzzles with extremely great creativity have unfathomably low scores. As a note, many of these are older puzzles and have had a long time to have votes cast on them.

I have found personally that only 11% to 25% of my puzzles' solvers actually vote on the puzzles, but I really can't attest to any benefits or detriments if a 10% scale were to be instituted.

While I would favor a 10% scale, I truly believe that when voting on a puzzle, regardless of scale, these factors should NOT influence a puzzler's vote: a puzzle subject's ambiguity or its ubiquitousness; the puzzler's solving capabilities and if a puzzle requires them to "guess;" a single or dual use of colors that the author used to be creative/authentic rather than those that would be deemed more easily visible to color-challenged puzzlers; the time it takes to solve; the number of colors used; or the amount of "empty" space. These factors SHOULD be considered when voting: a puzzle's creativeness, picture integrity, fun, and challenge.

I find that the worst offenses that would ever warrant a 25% or even 0% from me usually have one or more of these factors: not very attractive pictures; at least two colors that are impossible to tell apart; four or more extremely close colors; multiple use of bright colors (though this could have been an author's oversight and can be easily forgiven); pure fill-ins for more than half the picture; or incredibly abstract filler that doesn't really amount to anything except an increase in challenge/points. I don't believe I've ever given a 0% and probably less than ten 25%s.

My twenty-two cents.

Ah, the naivete of honest people. Don't get me wrong, I'm was, and for the most part, still am one of them. The best phrase that could sum this up is "Welcome to the Internet."

The anonymity of the internet gives one free reigns to do things one would normally wouldn't do. This type of person can be a mild harasser or a full on "troll." Best thing to do is ban/ignore/etc. Given the warmth and positivity of this puzzling community, one would think these people would be rare. These "trolls" go to ENORMOUS lengths just to create controversy or commit in douchbaggery behavior. Trolls are more likely to vote on puzzles and bring the % down than an honest person is likely to vote at all. That honest person may have really liked the puzzle but they just don't care enough (i.e. complacency) to take 2 seconds and vote. That's why one troll can have so much influence.

Hagit posted an example in a different thread where, if memory serves, one user voted about 1200 times on 1300 solved griddlers and 1000 of those votes were zeros. Whether or not the puzzles were minimalist or not I can't recall, not that that should matter.

I know that trolls will vote 0 or 25 for longer and more complicated puzzles with great images, especially if that puzzle holds a high percentage. For example, if you were to solve strunka's "Tiger" triddler, you would see that currently, the 0 and the 25 graph bars are the same height, and roughly 2/5 the height of the 75 and 100 bars, which are the same height. There is not a single 50. If you look at Autenticarse's "Nightmare" triddler, the bar graphs increase stepwise up to 100 being the most. While Tiger has 44 solvers, Nightmare 580, as well as not as long a publication history, it's percentage is actually lower. If you were to see both images, and compare the challenges, I personally feel that Tiger is so much better than Nightmare. But I believe the prevailing factors that will drop Tiger more are the difficulty, heavy use of 1s and 2s, and a puzzler's weariness after solving, especially if done in one or two sittings.

What needs to be done is that if there is a discrepancy such as a gap between percentages (e.g. no 25s or 50s but five 0s and multiple 75s/100s) or a reverse bell curve (e.g. ten 0s, one 50, ten 100s), the system should somehow remove or balance the oddities. The parameters needed to inform the system to react in a certain manner would be probably be extremely difficult as well as scrutinized to insure fair and just voting. Now if a puzzle has multiple 0s/25s and possibly only one 100, then that 100 should be removed by this system as well since the puzzle is more than likely extremely poor.

All this is extreme wishful thinking and really only matters since so few people vote. If more people voted, I doubt there would be such contention with the influence trolls have around here.

I guess all I can say is that as an author, I don't strive to put out low quality puzzles. I take time to refine and make sure it's solvable and even those 50s can be a slap in the face.