Your replies to "curious to know" thread in the forum taught us a lot about the way you vote for the puzzles. It also led us to think that the way the popularity is ranked should be changed. In her reply in the forum Barbara wrote "Maybe eliminating so many high votes and so many low votes would be the answer to a true popularity score". She uttered our exact thought. That's the way it is also done in many sporting competitions like Gymnastics.

We made some tests upon some puzzles with many votes, with a few votes, with high popularity, with low popularity, and got some results.

We also realized that if we want to change the voting calculation (eliminate a certain amount of votes), a minimum of votes is required. This need for minimum raised other questions. Today, using the old voting system, a single vote sets the popularity ranking right away. Is that correct? Can a single solver speak for all? What is the right number of minimum votes?

On this Friday (April 23rd, 9pm GMT) we will have a team meeting. Before the meeting takes place we would like to hear your suggestions
----------------------------------------------------------------------
What is the right way to calculate the popularity?
----------------------------------------------------------------------
Is it right to eliminate votes from the edges (0s and 100s)?
How many votes should be eliminated?
Can you come up with another idea to calculate the votes?
Do we need a minimum number of votes?
What is the minimum required?

According to your suggestions we will make more tests and show you the results during the chat meeting.

I think that all votes should be count. That's democratic. As an authot I'd like to see all votes high and low. That gives me more information than censored version.
What about rating 1 to 5. And average like 3,75. Or stars maybe?
After all I like this system, that is used now.

Mari

>----------------------------------------------------------------------
>What is the right way to calculate the popularity?
>----------------------------------------------------------------------
>Is it right to eliminate votes from the edges (0s and 100s)?

I'm not sure that it would solve anything to eliminate votes, and the ones eliminated proably won't like it that much. When you have done the analysis, does eliminating votes change the popularity of most puzzles very much? If it doesn't affect the puzzle, much, there may be no point in eliminating votes - if it does, that brings up hoards of other issues.....

>How many votes should be eliminated?

Difficult, since many puzzles, even in monthly most popular only have just over 10 votes - eliminating votes could radically change the most popular list, and eliminating even 1 vote out of 10 is eliminating 10% of the results (hmm, where is a statistician when you need them!!!)

>Can you come up with another idea to calculate the votes?

I think it was Barbara who suggested voting on more than one aspect of the puzzle. It seems most votes are cast for 2 issues - the end picture, and the "solveability" of the puzzle. Perhaps we should votes seperately on these 2 issues

>Do we need a minimum number of votes?

For what? We have a min for most pop and most pop of last month, we could put a min on the sort when you rank by popularity, but if the min was 5 and a griddler had only 4 votes, where would it be ranked, at the end of the list? Another thought - this could possibly encourage more people to vote... hmmm...
>What is the minimum required?

Who knows - I have had such a long answer already I obviously have far too much time on my hands.

I'll be out of town and miss the chat on Friday, but I will be interested to see what the results are.

Seeya

Mak

Answers

>When you have done the analysis, does eliminating votes change the popularity of most puzzles very much?

The real change is viewable in extreme cases.
How many times have you asked yourself "How could this puzzle get a 0 vote?" We took some examples of definite 75-100 puzzles, eliminated the same amount of 0 and 100 votes, and the results were amazing. There is a big difference.

>I'll be out of town and miss the chat on Friday, but I will be interested to see what the results are.

The results and a summary of the meeting will be published in the news on Saturday (after the chat meeting).

Hagit

I Iike the voting system just the way it is. Please let each vote count.

messing with the extreem votes will decreas the differance between the puzzles, thereby making the votes less meaningful.

also, removing a fixed number of votes will cause puzzles with few votes to change more drastically then those with many votes Probably the more difficult ones will have a greater effect on their score, since less people are likely to vote for them, and they are likely to get higher scores then small votes.

Now, if there must be an elimination of highs and lows, then I sugest calculating the number of votes to be removed by the precent of voters. say 10% from each side of the voting. which means 80% of all votes will count, while the highest 10% and the lowest 10% will be removed. This system will take care of the question of minimun number of voters there will be no elimination until 10% of all votes equals one vote.

another thought If there will be a calculation, it might effect people's voting behaviour for example what if people decide they will vote more to the middle, to make their vote count? Or if some decide not to vote at all anymore? Any change in voting pattern is likely to effect new puzzles more strongly then old puzzles, making the comparison of their popularity less meaningful.

I rather see the real votes like it is now.

Thanks for listening
T.

This "popularity" problem is getting complicated and I want to make a radical suggestion just remove the %0 so no one can vote %0 and all the authors will be happy.

Personally, I voted zero for some multis because of their content or because they are too tedious and require more than 2 hours of work without revealing anything. I was thinking that it is my democratic right to say "no I don't like it" but after reading all the topics on this popularity issue I feel pressured not to vote %0 for any puzzle.

>Answers
>
>>When you have done the analysis, does eliminating votes change the popularity of most puzzles very much?
>
>The real change is viewable in extreme cases.
>How many times have you asked yourself "How could this puzzle get a 0 vote?" We took some examples of definite 75-100 puzzles, eliminated the same amount of 0 and 100 votes, and the results were amazing. There is a big difference.

I did a random check of griddles in four areas of difficulty. Basing it on the elimination of 5% of the high votes and 5% of the low votes, you are right there are extreme cases. For the highest difficulties where you have 200-400 solvers there was very little difference in the popularity because the people voting on these are actually voting on the merit of the puzzles. The really big difference was in the low difficulties where people are not voting 0% because of the merit but because of the fact that it was easy. Those voting 100% (like myself) may be doing so to counteract the 0 votes. In these low difficulties it seems to me that eliminating a percentage of votes from either side weeds out those of us who aren't really voting the way we want to. Since it doesn't make any difference in the more difficult griddlers, I see no problem in doing if for all the griddlers. Some of us have a bad day and vote 0 just for the heck of it and the next day we're on a super high and everything pleases us we can't take back that 0 or 100 vote that wasn't valid. If people know that there votes are going to be eliminated then maybe they will start voting the way they really feel.

>If you are going to eliminate some votes at the top and at the bottom (which seems like a good thing to do) then it needs to be a percentage of the total vote that you eliminate, not a flat number. So, say there are 200 votes on a puzzle, you might eliminate say the top 5% of the votes and the bottom 5% of the votes, so that the results are still being calculated on 90% of the total votes. Of course those who know this is happening will be much more likley to vote in the mid range rather than the top or the bottom. But it still somehow seems more fair this way.

Sharon

>Is it right to eliminate votes from the edges (0s and 100s)?

In my opinion, if anything should be eliminated, it should be like in skating competitions - amount of votes, but any votes. Not exactly 0s and 100s, but just a percaentage of random votes.

>How many votes should be eliminated?

No more than 10% But I think that they all should stay.

>Can you come up with another idea to calculate the votes?

This is a good question. I agree with the idea to vote in two-three categories (e.g. solvability, picture, pleasure of completing)

>Do we need a minimum number of votes?

There already ARE miminums. Minimums for the best griddlers of the month and ever. I can't see any point in creating a minimum number of votes for a puzzle to have any rating.

powsinoga

hi,

okay, some of these points are somewhat redundant with what others have posted, but i think it's good to know that more than one person thinks something.

A a vote is a vote, & none should be discounted just because of the choice (i.e., 0 or 100) the voter made.

B the knowledge that high & low votes might be removed will certainly influence people to use the mid-responses. you may as well remove the 0 & 100, & then just wait for the same issues to rise again with 25% & 75% until the only option is 50%...

C i think that a more telling statistic indicating popularity would be the mode response (the most common response). of the different types of descriptive statistics, extreme values (like 0 & 100) most strongly affect the mean, which is what we are seeing here.

for example, for a puzzle with 2 0's, 2 25's, 2 50's, *5* 75's, & 2 100's, the MEAN is 55.8, whereas the MODE is 75.

if my understanding of the purpose of "popularity" scores is correct, then what we want to know is what *most* people think about a puzzle, without being too skewed by a couple of people who are mad that colors are too similar or something. in the example above, i think that 75 more clearly reflects overall satisfaction with the puzzle, without limiting the scale or anyone's voice in the decision-making process.

in this way, authors of puzzles that truly "deserve" a 0 or a 100 will get the feedback they want based on what most people think, thereby removing the biases of a certain few (on both the high & the low end).


anyway, those are my thoughts. i'd be interested to hear what anyone thinks about my suggestion...

thanks.
seeya.
erin.

>Is it right to eliminate votes from the edges (0s and 100s)?
>How many votes should be eliminated?
>Can you come up with another idea to calculate the votes?
>Do we need a minimum number of votes?
>What is the minimum required?


A few random ideas!

If 0 is eliminated 25% will just become the "new" 0 with people upset at receiving that score!

I hate to take away any votes. This would skew the way voters choose their vvote and also disenfranchise the voters. They have a right to their opinion- even if it is an unpopular one!

Have you thought about only showing a popularity score for a puzzle after it reaches a threshhold of a certain number of votes? What I mean is that the more votes there are cast for a particular puzzle, the less impact each vote has on the actual score. So, if we didn't actually eliminate votes, but rather didn't show the votes until the number of votes reached an agreed upon number based upon some research, we might get a truer result. For instance, if we said that a puzzle needed a minimum of 10 votes , there could be a blinking "Please Vote" sign in place of the bar graph and Popularity Meter and only the voting buttons would be visible. Once there are 10 votes, the "Please Vote" would be replaced by the Popularity meter and the bars. (Perhaps there could even be a countdown- 3 more votes needed!)

Mind you, I am not a programmer and just the thought of this may cause the "boys" to start pulling out their hair! And, the number 10 was used only as an example. The number might need to be 15 or 25 or some other amount.

Just another idea!
) Chef

As with the original thread, this is an extremely interesting discussion and the ideas that are coming out of it are fascinating! My ideas overlap a lot with what's already been posted here, but I hope I'm adding something too.

Is it right to eliminate votes from the edges (0s and 100s)?
Definitely not. As many here have said, that merely makes 25 (or whatever the lowest vote is) the new 0 and the highest the new 100. Also, why should a puzzle that anyone considers to be "perfect" in all respects only get, say, a 75? (And likewise, a puzzle that really is terrible, like the one that had no title and wasn't a discernible picture, will still get a 25.)

The real problem here is not the range of votes being expressed as numbers and what those numbers should be, it is the "misuse", if you like, of the system, e.g. those voters who give a puzzle with a low difficulty rating a 0 because it's "easy".

How many votes should be eliminated?
None, as above.

Can you come up with another idea to calculate the votes?
Some ideas
1) What about making the votable range span 1 to 100, i.e. take away the rigid nature of the current five rating levels? Sometimes I really want to go for a rating in the middle of the 50 and 75, say, and if I could give a 63 for instance that would much better reflect my opinion of the puzzle.

2) One scale doesn't seem to be enough to rank all the aspects that affect my opinion of a puzzle. I would prefer to be able to rate
a) the level of difficulty, call it the "challenge factor" if you like
b) the resulting picture, call it the "artistic merit" if you like
c) the "aha factor", i.e. the surprise, or the cleverness of the picture and its relationship to the title, or the cuteness etc.
All of these would then go towards an overall puzzle rating.
I have often wavered - and frequently ended up not voting at all - because I loved the picture but felt the puzzle was too easy to give it a 100, or I loved the challenge of a more difficult puzzle but didn't really like the colours or composition, and again couldn't really warrant giving it a 100 or 75. Am I making sense?

Do we need a minimum number of votes?
This is something I am familiar with from sites such as the Internet Movie Database, which doesn't display a rating until at least 5 people have voted(they use stars, by the way). But considering the fact that you always give the number of people who have voted, I don't really see that there is a necessity for a minimum number of votes. It only takes a glance at the bottom of the column to see that the 100 was given by one person and is, therefore, not yet representative of a larger number of solvers. Ah, hold on a sec, I do see the problem a puzzle with a 100 rating from one solver will be very high up the most polpular scale - ok, then maybe it would be a good idea to wait for at least X number of votes to be cast before a puzzle goes into the pool.

What is the minimum required?
If you have five levels as at the moment (0, 25, 50 , 75, 100) you need a minimum of 5 so that there could, if necessary, be one vote in each to give a fair picture of even a small number of votes. If you decide to go to a flexible voting range of 1 - 100, you would need 100 votes as the minimum.

I can't make Friday's discussion either, but I really look forward to the results.

Have fun!
Punkin UK

>>>> A few random ideas!
>>
>>If 0 is eliminated 25% will just become the "new" 0 with people upset at receiving that score!
>
>We are not eliminating 0 or 100 only a percentage of 0 or 100. Neither can ever be eliminted so 25% will never become the new 0 and 75% will never become the old 100% . If someone really wants to vote 0 or 100 and based on a arbitrary 5% elimination amount, there would have to be 20 0 or 100 votes before 1 vote was eliminate. ( I think we're talking about the same thing) 5% of 20 =1? Actually we're not just talking about 0 and 100 but about 5% of low votes and high votes. There are some griddlers on the site that have no 0 votes at all.

>Have you thought about only showing a popularity score for a puzzle after it reaches a threshhold of a certain number of votes? What I mean is that the more votes there are cast for a particular puzzle, the less impact each vote has on the actual score. So, if we didn't actually eliminate votes, but rather didn't show the votes until the number of votes reached an agreed upon number based upon some research, we might get a truer result. For instance, if we said that a puzzle needed a minimum of 10 votes , there could be a blinking "Please Vote" sign in place of the bar graph and Popularity Meter and only the voting buttons would be visible. Once there are 10 votes, the "Please Vote" would be replaced by the Popularity meter and the bars. (Perhaps there could even be a countdown- 3 more votes needed!)
>
>) Chef

I made some calculations( of my own griddlers)
Number Title Date Votes 22.04.2004
6060 more berries 17.09.2003 8
6399 waiting for spring 30.10.2003 8
7573 nasty pack 04.02.2004 5
7569 chefmonster 04.02.2004 6

and i have more exsamples about very few votes.
Sometimes I have to wait first vote of my newest griddler three days or more.
Do you mean that the creator of harder griddlers must wait for months to read voting results? It's hard time to wait, I can tell you.
Mari

>
>I made some calculations( of my own griddlers)
>Number Title Date Votes 22.04.2004
>6060 more berries 17.09.2003 8
>6399 waiting for spring 30.10.2003 8
>7573 nasty pack 04.02.2004 5
>7569 chefmonster 04.02.2004 6
>
>and i have more exsamples about very few votes.
>Sometimes I have to wait first vote of my newest griddler three days or more.
>Do you mean that the creator of harder griddlers must wait for months to read
voting results? It's hard time to wait, I can tell you.
>Mari
Mari after reading your comment it seems to me that the only people who should be asked to make a decision on how the popularity should be decided is the authors themselves. These are the people who it has an impact on, so shouldn't they be the ones to decide? Thank you Mari for bringing this to light. I intend on going to your griddlers and solving some. Hopefully they aren't too hard.

Hi Barbara,

While I see what you mean in your comment to Mari that the authors are the people voting has an impact on, they are not the only ones for whom the voting is relevant. Many solvers have indicated in their comments here and also in the thread "Curious to know" that they do indeed use the popularity rating to decide whether they want to solve a puzzle. Therefore, both authors and solvers should be involved in the discussion and ultimate decision-making process regarding the way popularity is calculated. Many - perhaps even all - authors are also solvers, so they will be aware of how important the popularity rating can be for both sides of the puzzling fence, but I don't think it would be entirely fair on solvers if only authors decided how puzzles were graded. Isn't it better to get everyone involved?

Best wishes,
Punkin UK

Many - perhaps even all - authors are also solvers, so they will be aware of how important the popularity rating can be for both sides of the puzzling fence, but I don't think it would be entirely fair on solvers if only authors decided how puzzles were graded. Isn't it better to get everyone involved?
>
>Best wishes,
>Punkin UK
>
I didn't meaned that only authors decide it. Ofcourse it's part of the fun as a solver too.
Mari

> Many - perhaps even all - authors are also solvers, so they will be aware of how important the popularity rating can be for both sides of the puzzling fence, but I don't think it would be entirely fair on solvers if only authors decided how puzzles were graded. Isn't it better to get everyone involved?
>>
>>Best wishes,
>>Punkin UK
>>
>I didn't meaned that only authors decide it. Ofcourse it's part of the fun as a solver too.
>Mari
>
Hi Mari,

My comment was in response to Barbara's comment that perhaps only authors should be the ones to decide. Sorry if you felt I had misunderstood you - I agree with you that it's part of the fun for solvers too.

>>>If 0 is eliminated 25% will just become the "new" 0 with people upset at receiving that score!
>>
>>We are not eliminating 0 or 100 only a percentage of 0 or 100.

You're right. Your proposal does not include wiping out the 0 category. I was actually responding to Turquoise who recommended eliminating the 0 option. I think this would be a mistake.

I still worry about the perception voters would have that theirs might be the particular votes that don't "count" if they want to vote 0 or 100 on a particular puzzle. The difference here is that in the Olympics and in gymnastics, there is a sliding scale and rarely if ever are the highest and lowest votes used. Therefore, when they vote, they can't know for sure if their vote will be eliminated from the final score or even if it is a possibility until all of the vootes are known. Here, on the other hand, the voter will know that if they vote 0 or 100, they could be part of the fringe vote that is "eliminated".

I also understand the problem of authors having to wait a long while to see the popularity of the puzzle they have created. However, the number of votes required may not have to be that high, and may be able to be figured on a sliding scale. We may also need to come up with something that encourages people to vote after solving. This would help authors like Maristone who has only about 1/3 of her solvers vating on her puzzles, which I'm guessing is not unusual!

(BTW, Mari, I can't believe I never noticed your puzzle Chefmonster before! My humblest apologies! I'll get on it right away!)

Chefmomster

How about something simple.

Just drop the lowest vote...or 2 lowest votes.

If we must take away both high votes and low votes
Take away both the 2 (or 3) lowest and 2 (or 3) highest votes. Keep the number of votes needed for most popular 10 and 25. But this way they'd be calculated on 6 (4) or 21 (19) votes which is similar to what they were based on before (5 and 20).

I think no votes should be removed. It's easy to see how the puzzles get their ratings, since the number of voters is shown. And the minimums are used for the "most popular" groups.

>Hi Barbara,
>
>While I see what you mean in your comment to Mari that the authors are the people voting has an impact on, they are not the only ones for whom the voting is relevant. Many solvers have indicated in their comments here and also in the thread "Curious to know" that they do indeed use the popularity rating to decide whether they want to solve a puzzle. Therefore, both authors and solvers should be involved in the discussion and ultimate decision-making process regarding the way popularity is calculated. Many - perhaps even all - authors are also solvers, so they will be aware of how important the popularity rating can be for both sides of the puzzling fence, but I don't think it would be entirely fair on solvers if only authors decided how puzzles were graded. Isn't it better to get everyone involved?
>
>Best wishes,
>Punkin UK


You're right the popularities do have an inpact on the solvers too. I look at the popularities usually after I solve the griddler. I can't remember ever letting the popularity of a griddler determine whether or not I'd solve it, but thats just me. The popularity is usually off the screen with the comments and I don't usually scroll down. I usually solve for points, trying to do the most difficult first and spending my time on the easier ones when time is at a minimum. I have to admit that with the volume of griddlers being made I'm going to have to start looking at the popularities because I just can't keep up and I have a feeling I'm missing some really good griddlers. I think that is what is happening with Mari and the others. A page on the site gets filled up so fast that there are days that I don't even see what was on the previous page or two. Can we figure out a way to have more hours in the day.

Barb

>>Hi Barbara,
>>
>>While I see what you mean in your comment to Mari that the authors are the people voting has an impact on, they are not the only ones for whom the voting is relevant. Many solvers have indicated in their comments here and also in the thread "Curious to know" that they do indeed use the popularity rating to decide whether they want to solve a puzzle. Therefore, both authors and solvers should be involved in the discussion and ultimate decision-making process regarding the way popularity is calculated. Many - perhaps even all - authors are also solvers, so they will be aware of how important the popularity rating can be for both sides of the puzzling fence, but I don't think it would be entirely fair on solvers if only authors decided how puzzles were graded. Isn't it better to get everyone involved?
>>
>>Best wishes,
>>Punkin UK
>
>
>You're right the popularities do have an inpact on the solvers too. I look at the popularities usually after I solve the griddler. I can't remember ever letting the popularity of a griddler determine whether or not I'd solve it, but thats just me. The popularity is usually off the screen with the comments and I don't usually scroll down. I usually solve for points, trying to do the most difficult first and spending my time on the easier ones when time is at a minimum. I have to admit that with the volume of griddlers being made I'm going to have to start looking at the popularities because I just can't keep up and I have a feeling I'm missing some really good griddlers. I think that is what is happening with Mari and the others. A page on the site gets filled up so fast that there are days that I don't even see what was on the previous page or two. Can we figure out a way to have more hours in the day.
>
> Barb
>
Hello Barb,

Yes, I see your point and I admit that I work in the same way. I was thinking of those solvers who might be using the popularity rating as one of the criteria while they are still on the puzzles list page. But even there, the number of voters is given, so that completely takes the bite out of what I was saying!

As for having more hours in the day - I already have 32 in mine and that's still not enough!

Punkin UK

hi again.

i am really curious to know what people think of my suggestion in my previous post about just using the mode response.

should i take the lack of response as total disagreement?
if so, that's fine -- i'm just curious as to people's opinions on it. if it sounds bad, why do you think so? etc.

i'm just noticing a lot of replies to people's ideas, both agreeing & disagreeing with them, & wondered why mine inspired no reaction... (which, again, is fine. no hard feelings.)

okay, let me know.
thanks.
seeya.
erin.

Hi Erin - I loved your response - I think you are the statistician I was talking about in my post ;)

I like the idea of the mode - it does give the info most people want rather than the mean.

I also liked the idea someone had of being given more than 5 choices 0, 25, 50, 75, and 100% - a 0 to 10 scale or something might be better



>hi again.
>
>i am really curious to know what people think of my suggestion in my previous post about just using the mode response.
>
>should i take the lack of response as total disagreement?
>if so, that's fine -- i'm just curious as to people's opinions on it. if it sounds bad, why do you think so? etc.
>
>i'm just noticing a lot of replies to people's ideas, both agreeing & disagreeing with them, & wondered why mine inspired no reaction... (which, again, is fine. no hard feelings.)
>
>okay, let me know.
>thanks.
>seeya.
>erin.

>hi again.
>
>i am really curious to know what people think of my suggestion in my previous post about just using the mode response.
>
>should i take the lack of response as total disagreement?
>if so, that's fine -- i'm just curious as to people's opinions on it. if it sounds bad, why do you think so? etc.
>
>i'm just noticing a lot of replies to people's ideas, both agreeing & disagreeing with them, & wondered why mine inspired no reaction... (which, again, is fine. no hard feelings.)
>
>okay, let me know.
>thanks.
>seeya.
>erin.

Apologies Erin, what a bunch we are! Your idea is very worthy of consideration. I think it is a much more valid reflection of the "popularity" since, as you pointed out, it shows how the majority of solvers have voted. The mode is most definitely a better option that the mean.

Thank you for suggesting it.
Punkin UK

Maybe the best idea would be approximating the five values with a gauss curve and then finding its top. if unsuccesful, the curve would be approximated from the middle three. But I guess it's hard.

>We also realized that if we want to change the voting calculation (eliminate a certain amount of votes), a minimum of votes is required. This need for minimum raised other questions. Today, using the old voting system, a single vote sets the popularity ranking right away. Is that correct? Can a single solver speak for all? What is the right number of minimum votes?

10? just my guess

Well... I already stated my opinion. The voting system is fine as it is.

However, an addition of the mode, as Erin suggested, is a good one (your sugestion was not ignored, erin. It was only posted yesterday, right?). I'd like to see this too in the statistics, WITH the everage.

I also think the vote results should be visible starting from the first. people are smart enough to look at number of voters to decide how meaningful the everage is.

About giving a scale of scores, this will be again a problem for the older puzzles. in a continuous scale, I foresee less people voting the mininum and maximum scores. the best and worst puzzles will now have less 100 and 0 then they would have had under the previous system, which will make the comparison between the popularity of older and newer puzzles difficult.

Thanks for listening
T.

Hello everyone,

I have read through everyone's comments on this matter. As I am the one who will be programming this, I think it is a wise decision now to not disregard any votes. This will cause a difference in what is shown as the popularity of a puzzle in the puzzles page and the author's page, where it would not disregard any votes.

But, I have come to the conclusion that many vote untruthfully according to what they see in the votes prior to voting. I think this is where the major problem lies. You're voting to counter either a low or high vote. This is wrong.

I believe I have come up with two alternatives for this problem. Either we need to HIDE the individual votes until after you have voted, this will eliminate your option to change your vote, since then it would not help to hide them if you can change it after you see the votes, or remove the option of seeing the individual votes completely. Either would eliminate false voting, and require you as the solver to vote upon your thoughts of the puzzle, not someone else''s.

The suggestion of voting in different categories was also brought up. (Examples solvability, picture, and pleasure of completing). This would be ok, but is it really needed? The puzzle would really have no overall popularity percentage then.

There has also been the suggestion of adding more options to vote (10 or 100, instead of only 5). 100 would be a programming nightmare!! 10 would be good, but it would completely mess up any current votes that are already in place. What do 25% and 75% votes get placed in? 20 or 30 / 70 or 80? The only way to do something like this is to wipe all the votes, which I don't think you want, or split the 25 and 75 votes evenly to either side.

Please let me know what you think about these ideas.

TNT
Eric

Hey Eric - here are my thought on your post


> I believe I have come up with two alternatives for this problem. Either we need to HIDE the individual votes until after you have voted, this will eliminate your option to change your vote, since then it would not help to hide them if you can change it after you see the votes, or remove the option of seeing the individual votes completely.

I'm not sure I understand how this works - you can see the average vote, but not the individual votes that make up the average?


> The suggestion of voting in different categories was also brought up. (Examples solvability, picture, and pleasure of completing). This would be ok, but is it really needed? The puzzle would really have no overall popularity percentage then.

I think it would work if you made overall popularity a simple average of the 2 or 3 voting criteria - in terms of dealing with votes that have already been cast, at this point you could give a puzzle that has, say, a 58% popularity, 58% each in solveability, picture, and pleasure of completing. It wouldn't be perfect, but would be a reasonable place to start.

> There has also been the suggestion of adding more options to vote (10 or 100, instead of only 5). 100 would be a programming nightmare!! 10 would be good, but it would completely mess up any current votes that are already in place. What do 25% and 75% votes get placed in? 20 or 30 / 70 or 80? The only way to do something like this is to wipe all the votes, which I don't think you want, or split the 25 and 75 votes evenly to either side.

How about
0
12.5%
25%
37.5%
50%
62.5%
75%
87.5%
100%

This way you can put the old votes in without having to change them in any way.

What do you all think about this?

Mak

TNT, I love your ideas because they seem to most simply, from both a practical and a programming view, take care of the issues discussed in this forum.

>As I am the one who will be programming this, I think it is a wise decision now to not disregard any votes.

I agree with not deleting any votes for reasons I've mentioned in earlier posts, but this is in itself a reason to avoid this solution.

> Either we need to HIDE the individual votes until after you have voted or remove the option of seeing the individual votes completely.

I would vote for the second option. Learning that 3 people gave me a 0 for the puzzle I'd worked so hard on would not add to my knowledge a bit! Plus, authors who are hurt when they see the 0 scores wouldn't be aware of them. I agree also that the root problem is that votes are skewed by the voters knowing how the vote is going before they cast their own vote or by being able to change their vote in an attempt to change the overall score of a puzzle.

> The suggestion of voting in different categories was also brought up. (Examples solvability, picture, and pleasure of completing). This would be ok, but is it really needed? The puzzle would really have no overall popularity percentage then.

I agree with Makarios that an average of the different categories could be found, but I believe this option would just complicate the site and the voting system. I think it is legitimate to vote based on any of the items listed above and it isn't necessary to split it out, although I wouldn't be opposed to a very simple version of the idea.

> There has also been the suggestion of adding more options to vote (10 or 100, instead of only 5). 100 would be a programming nightmare!! 10 would be good, but it would completely mess up any current votes that are already in place.

Once again, I don't feel the complications of this would be worth the enhanced value. And, I think we have to weigh TNT's imput heavily since he is the programmer and will have to do the work of whatever we decide and is knowledgable about what can reasonably be accomplished.

Thanks for the info and ideas, Eric!

Hi TNT
Here's my opinion on what you said

> I think it is a wise decision now to not disregard any votes.

I totally agree with that.

> But, I have come to the conclusion that many vote untruthfully according to what they see in the votes prior to voting. I think this is where the major problem lies. You're voting to counter either a low or high vote. This is wrong.

You are correct in saying it's wrong. It happens, I think, because some people believe that zero voters do this for spite. I think voters should give other voters the benefit of the doubt here. I believe only very few solvers will vote a zero for no reason.

> I believe I have come up with two alternatives for this problem. Either we need to HIDE the individual votes until after you have voted, this will eliminate your option to change your vote, since then it would not help to hide them if you can change it after you see the votes, or remove the option of seeing the individual votes completely. Either would eliminate false voting, and require you as the solver to vote upon your thoughts of the puzzle, not someone else's.

On this, I would like to see the individual votes. It's always interesting to know if a puzzle was voted 50 because lots of people though it was medium quality, or because some people loved and others hated it. The later must be a very interesting puzzle indeed! This is even more interesting to me as a creator.
However, I do believe the solvers should vote first and see the votes later. but, since the everage popularity is still shown, people already know something about the score and this can influance their vote, anyway. I don't think people should be prevented from changing their votes if they made a mistake. I believe very few will bother to vote again, only because the votes split this or that way.

>
> The suggestion of voting in different categories was also brought up. (Examples solvability, picture, and pleasure of completing). This would be ok, but is it really needed? The puzzle would really have no overall popularity percentage then.

This will just complicate the vote. people might decide not to bother.

> There has also been the suggestion of adding more options to vote (10 or 100, instead of only 5). 100 would be a programming nightmare!! 10 would be good, but it would completely mess up any current votes that are already in place. What do 25% and 75% votes get placed in? 20 or 30 / 70 or 80? The only way to do something like this is to wipe all the votes, which I don't think you want, or split the 25 and 75 votes evenly to either side.

I prefare the system as it is, but if you decide to have more options, how about making it a smooth scale between 0 and 100? No splitting, just a slider. This will be easier to program. The votes can be shown on a line graph.

Thanks for listening
T.

Popularity should be a function of the total number of puzzlers, not just the number of puzzlers who have completed a puzzle thus far. If only two members have completed a given puzzle, their two votes could make the raw percentage (which you now use) seem very high, which is misleading. However, if a puzzle's popularity score gets higher (or lower) as more and more people complete the puzzle and vote, then a popularity score begins to take on meaning.

Just my 2 cents.

Sacker

I have not read all the comments on this subject, so please excuse me if I repeat something that has already been said. ;)

Personally, I do not vote on many puzzles, or make comments, for the simple reason that it is a pretty involved process. After you solve the puzzle the puzzle's main page refreshes, then if you want to vote you do so and click on the Vote button and the page refreshes again, as it does if you want to place a comment. I believe many more solvers would vote if the vote option was given in the same window as a just-solved puzzle, allowing a solver to view their newly solved puzzle, place a vote and/or comment, then close the puzzle window and that would automatically update the small puzzle image, the comments, and the popularity in one step.

I also believe that every vote should count. However, I think it would be nice if there were some very specific standards regarding what is acceptable as a 0 or 100 vote. I think there are a lot of solvers who vote 0 just because they don't like the subject matter (if the puzzle's a cat and they believe there are too many cat puzzles on the site) or they find it too easy, even when the difficulty is clearly posted. And honestly, when I see that on puzzles that are clearly artistic or unique or a solver's first effort, I feel like voting 100 just to offset those unfair 0s. I know it's impossible to make solvers go exactly by the set standards, but it might help eliminate some of the 0s and 100s at the voting process.

Regarding the minimum number of votes, it might be possible to make it so that a solver cannot continue until they place a vote. That would be easier with my idea above about the puzzle window and the votes/comments window being all-inclusive.

Hi
I suggest you consider using the bell curve.
I'm sure there's a better explanation of it on line somewhere, but here's something in short
Basically, instead of eliminating the highest and lowest scores, you use them to find the average.
The idea behind this method is that if the "true" score of the puzzle is N, and X people have voted, Half of those people will cast a score bigger than N, and the other half will vote a score smaller than N.
About the minimum of votes before the popularity is shown - I think it is necessary, and I think it should be calculated according to the puzzle's difficulty.
Many people solve easy puzzles. and quickly. while other puzzles can go unsolved for days. so the easier puzzles should have a bigger minimum of votes before their pupolarity is calculated.

I just thought of a way to softening the effect of the extreme fringe votes without actually eliminating any of them altogether. If there is a mechanism in place to keep track of whether or not a user voted on any given puzzle, you should be able to set something up whereby those who don't bother voting on a puzzle have an automatic vote of 50% until such time as they actually cast a vote on the puzzle.

Using a 50% "vote" for each non-vote, you then calculate the popularity over the number of people who solved the puzzle, rather than just the ones who actually voted. I believe that doing this will have an effect similar to putting the actual vots on the bell curve, and it also alleviates the concern that some people might have of a single voter speaking for everyone when only one actual vote is in.

Hello,

I am quite new here, but I feel I should contribute too.

I must admit I don't vote for puzzles, I totally agree with arilinde that if there was a possibility to vote for the puzzle just after the solving, it would be much better. I can imagine, after solving a puzzle, if a small window popped up with some information, like how much time the solving took me, how many completed puzzles and points for them now I have in total etc., partialy because it could be an interesting info for someone, and partially to 'disguise' a voting bar or slider or anything. By that 'disguise' I mean that thanks to those infos the window wouldn't look so empty only with the bar.

It would be optimal if the voting control served for closing the window too. There wouldn't be any close button (except for the natural button in the window caption for those who really don't want to vote). After closing the window, the main page would be refreshed. But I am afraid it would lead only to more 0's from people clicking 0% just to get rid of the window quickly.

Next, I agree with some ideas posted before. First, not to show the number or percentage of individual votes to anyone who hasn't yet solved the puzzle, and second, to make a wider scale, from 0 to 10 optimally. I know there is a problem what to do with those old votes not divisible by 10, but I think if random half of the 25's became 20's and half 30's, it would be quite ok. Similarly with 75's.

Unfortunately I don't know what to with the 0's, or I have an idea (based on dynamic values of votes - a 60% from someone who usually votes only 20% - 50% is much more valuable than from someone who uses 50% as the worst vote), but it is quite complicated and very demanding of programming, the server's storage space and computing power. I only agree that eliminating some of the votes isn't the right way. Maybe eshubel's or luweewu's ideas aren't so bad.

Thanks for your patience with reading this o)

P.S.
Sorry for posting this as a new thread, I didn't know that it will become new topic if I reply to the original hagitm's message. Someone can delete it so that this topic is not divided.

>I just thought of a way to softening the effect of the extreme fringe votes without actually eliminating any of them altogether. If there is a mechanism in place to keep track of whether or not a user voted on any given puzzle, you should be able to set something up whereby those who don't bother voting on a puzzle have an automatic vote of 50% until such time as they actually cast a vote on the puzzle.

I may be wrong but isn't this in effect making people cast a vote. Some people just don't want to vote for one reason or another. I don't vote on every griddler I solve and to have to go back and cast a vote later is time consuming. You're telling me that my vote would have been 50% if I had voted . How do you know. This is not a school test where you have concrete numbers and the bell curve can be used. These are peoples feelings, interpretations and opions that can't be guessed at nor have a preset number attached to it. That's the way I read your suggestion, so if I'm wrong please correct me. If you look at the number of voters compared to the number of solvers there's a majority of solvers who don't vote so the popularity on most of the griddlers would always hover aroung 50%.

>Using a 50% "vote" for each non-vote, you then calculate the popularity over the number of people who solved the puzzle, rather than just the ones who actually voted. I believe that doing this will have an effect similar to putting the actual vots on the bell curve, and it also alleviates the concern that some people might have of a single voter speaking for everyone when only one actual vote is in.

But you are saying that any puzzle is automatically worth only 50% What about the most popular puzzle that is (at last check) at 100%. On this one the people spoke. No griddler could ever be worth 100% again unless EVERY solver (sometimes hundreds of them) voted 100%. I may be totally off base here but I feel that only the actual votes cast should be used.

Just to show what a big effect 1 vote of zero can cause I will tell you this.

Last December I did a Countdown to Santa. A puzzle each day until Santa arrived. One day I checked and 3 of the puzzles were in the top 20. I was very happy about that...until I checked 2 hours later and all were gone. I checked and there was 1 zero in each puzzle. Sadness. Either three separate people didn't like them in that short a time span or one person voted zero on all three. Someone doesn't like puzzles, doesn't like Santa, someone doesn't like me, I don't know.

Fastforward to last night. I check the Most Popular and FOUR of my Santa puzzles made it on! What's going on? I immediately suspected Hagit of stripping away the zero's. She denies it. I believe her. The only other thought is that the person looked in this thread and changed his/her vote because of the views expressed.

Anyhow, my point is that one person voting zero will drop most (17) of the puzzles from the top 20 that are there now. (Check and see for yourselves!) My question is Should one person have that much power?

P.S. To the person who took away the zero's Thank you! Sure it's a big ego trip but I like to see my puzzles in the Most Popular.

>I am quite new here, but I feel I should contribute too.

Welcome! And thanks for caring enough to share your ideas!

>Sorry for posting this as a new thread, I didn't know that it will become new topic if I reply to the original hagitm's message. Someone can delete it so that this topic is not divided.

You did just fine! Our programmers work hard to make this site the clearest, simplest and easiest to navigate on the www, so you can hardly make a mistake! Well, maybe in your puzzle solving- but there you're on your own! )

Chef

PS Come to think of it, you're not alone there either! If you get hung up, just send up a red flag and dozens will come running to your assistance! It's a great community here! )

I think it is fine the way it is - this is the internet and you will never please everyone - and I would probably not bother to vote if I knew my 100's were just going to be tossed.

hello again,

yesterday I mentioned my idea of what to do with all of the unnecessary 0's, but I thought it would be too difficult. However I was thinking about it today and I got some new ideas, leading to IMHO relatively fair, usable and not so difficult solution.
For this method it is necessary to store all everybody's votes.

In my explanation I will refer to a picture I made for this purpose, located at
mujweb.cz/www/thommascz/solution1.gif
This text is there too:
mujweb.cz/www/thommascz/solution1.txt

I will be using a fictive man called XYZ. His votes are in figure 1 on the picture. (The voting scale goes from -5 to 5 but can be easily transformed into 0 to 10 or anything else. In the text I sometimes use percentages.) We can see that XYZ tends to vote low: he has voted -5 6 times, -4 2 times and -3 3 times. If he now happens to vote -2, which usually means bad, for him it means that the puzzle was the best he has ever seen. And I am sure many people overestimate or underestimate puzzles. Someone votes mostly from 20% to 60%, someone from 50% to 100%. The goal of my solution is to balance this, without eliminating any votes. The goal is, if somebody votes only 0%'s, they will be counted as 50%.

(Some technical stuff here, you can skip it - the purpose is to find XYZ's average vote (AV).)
See figure 2. I will apply some physics and mechanics - convert the voting bar into a see-saw, and votes to forces. From this we can compute a torque from the formula:

(T)orque = (F)orce * (r)adius

(For those interested, Wikipedia, the free online encyclopedia describes torque as a "rotational force".
http://en.wikipedia.org/wiki/Torque)

(In mechanics, radius is the distance from the stable point 0, so only absolute values are used, and if I remember it right, the sign of the torque is determined by the direction of the rotation (clockwise or counter-clockwise), but here I will use negative values where needed.)

So:
T1 = F1 * -5 = 6 * -5 = -30
T2 = F2 * -4 = 2 * -4 = -8
T3 = F3 * -3 = 3 * -3 = -9

We can get the total torque by a sum of all torques.
T = T1 + T2 + T3 = -30 + ( -8 ) + ( -9 ) = -47

We also have the total force:
F = F1 + F2 + F3 = 6 + 2 + 3 = 11

From this we can compute the position ( (R)adius ) of F, meaning where F has to be applied to have the same effect as F1, F2 and F3 together.
R = T / F = -47 / 11 = -4.27

The positive torque from positive votes (forces) is computed separately, and then the two total torques are summed. If the resulting total R is negative, the person underestimates puzzles and if positive, he/she overestimates.

(End of technical stuff)

So, we computed that in our example, the XYZ's AV is -4.27 . If he had voted 11 times -4.27, it would have had the same effect as his real voting.

So far, I am quite sure I am on the right way. But what to do with this value is a question that needs some discussing. I haven't had much time to really think about the possibilities, and I am not a mathematician, so I only developed one idea as I wanted to have at least one complete solution. But I think it could work.

See figure 3 - the real voting bar is shifted, or moved, by the value of the AV, so that 0, or generally the middle, corresponds to the XYZ's AV. It means, if XYZ votes -4.27, it will be counted as 0, if he votes -2, it will be counted as 2.27. The figure 4 shows the same as a graph. On the x axis is what XYZ clicked, y is what is counted. Notice that everything before and after the boundaries is rounded to the first, respective last number. So XYZ can't make a worse vote than -0.73 or better than 5. For the puzzle's rating it doesn't matter, if XYZ clicks 1 or 5, the only difference is for XYZ, because the AV is computed from the real votes. So if he votes 1, AV will become -3.83, if he votes 5, AV will become -3.5 (if I am counting right).

Now, some other rules have to be applied. I mean, if XYZ has voted only a few times, the AV is a very unprecise value. So maybe before XYZ makes 10 votes, they could be used normally. There can also be some other problems which I didn't think of.

All of this would be quite easy (I think), but to make it more precise, it would be good to store not only numbers of votes, but also which puzzles the people voted for, and after every change of a person's AV, all of the ratings of previously voted puzzles would be recomputed. So the ratings would be dynamic and IMHO even more accurate, but this would eat quite a lot of server's computing power, keeping it busy recomputing ratings.

The advantage of this method is, it almost doesn't affect people who vote fairly (the closer is AV to 0, the closer the vote sent is to the vote clicked). In a fictive ideal world, where everybody would spread his/her votes evenly, the result would be exactly the same as a normal arithmetic average of votes.

I hope I didn't make any mistakes, which would be very embarrassing. I really apologize for such a long text. Well, you don't have to read it, anyway. Oops, shouldn't I put that sentence rather in the beginning? :o

>I hope I didn't make any mistakes, which would be very embarrassing. I really apologize for such a long text. Well, you don't have to read it, anyway. Oops, shouldn't I put that sentence rather in the beginning? :o

I am just so impressed that you would put so much time, effort and thought into this. Right or wrong you shouldn't be embarassed! (And heaven knows, I wouldn't be able to tell!) We're lucky to have a person who cares so much and is so technically talented as a member of our community.

Thanks!
Chef