|
Note: I was going to post a simple suggestion, but it seems this post has turned into a bit of a brainstorming session. Feel free to add ideas of your own.
Introduction
I see that some people have also noticed voting outliers (votes that are way off of what others have voted an article). Seems like they typically suggest that those outliers be adjusted somehow or that votes below a certain number require a comment (which serves a purpose of providing feedback and revealing the person that "low voted"). I would like to provide an alternative.
Deviation
Rather than choosing an arbitrary number below which feedback must be provided, why not choose a certain deviation from the existing votes? An example would be to force feedback for any vote that is a rating of 1.5 different than the average rating. So, if the average rating is 5 and the user selects 3, they must provide feedback. The reasoning is something like "all these other people think this article is this vote, but you think something significantly different... why is that?" That forces them into a bit of introspection and it makes sense for the author, because they then get feedback or at least get to see the bastard that low voted him/her. But it need not be just for low votes... could be for beginners who high vote articles because they lack the comprehension to understand it is a bad article. If asked "why do you think this article is so good when everybody else says it stinks", they might think "oh, uh, guess they're probably right" (which is probably the case if everyone else is voting low).
Repeat Feedback
Another thing to consider would be if somebody already left article feedback. After all, if they already wrote a comment on the article, why force them to make another comment when they vote (if they happen to vote after they made other comments)? Not sure if that's currently done, but thought I'd mention it just in case it isn't.
Deviation Calculation
The method you use to choose the deviation (I have a constant of 1.5 as an example) could be different. For example, you might choose the deviation based on the number of votes. Perhaps the first voter gets to set the precedence. And the second voter could get, say, 3 points of deviation. However, once the article reaches 10 votes, that deviation might be down to 1.5. The reasoning being that the more people agree on a certain vote, the less likely it is that an outlier is correct that everybody else is wrong.
Retroactive Required Feedback
Retroactive required feedback would be interesting. If the first person that happened across an article votes it a 1 but the next 20 people vote it a 5, it's likely that 1 vote was invalid. An email could then be sent to the 1-voter saying "you have 10 days to give some feedback, or your vote will be discounted". It could also be the the article has been modified and the quality increased. If that is the case, might not be bad idea to force a revote (which somebody might do if they are told their vote will be discounted). What happens during the discounting could also vary. That vote could straight up be removed from the rest of the votes. Or, it could be given a lesser weight.
Deprecated Votes
Along the same lines of causing a revote, would be neat to have version based voting. That is, if an article changes, newer votes then count for more. Perhaps a combination of a certain amount of time elapsing and a certain number of votes after a given version will discard the older votes. The reasoning behind that is basically that article may have improved in quality, so those low votes may no longer be accurate. But maybe that wouldn't be a great idea, because maybe voting takes into account the initiative of the author (i.e., how much work they initially put into the article, rather than the state of the article at any given moment).
Conclusion
I realize these ideas are a lot to ask. But they are really just ideas, and those never hurt. Feel free to take from them what you will (or will not).
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
Depreciated votes is definately an interesting idea. There is one article inparticular that I wrote when I was a bit of a newb and to be honest - it was pretty cr@p. I did a complete rewrite of it a while ago but those original votes on what is now essentially a new article still stand. I could have just made a new article but that would have been pointless as it was the same subject.
A difficult one to work though as one word change marks an article as a new version in the current system so I'm sure people would (attempt to) abuse it to give inacurate voting totals. I'll be interested to read when Chris comments as he knows how viable this stuff is from the hamster's end.
|
|
|
|
|
DaveyM69 wrote: DepreciatedDeprecated votes
I know somebody else who always makes the same mistake all the time, but I never bothered to correct him. Still makes me cringe when he says it. Their meanings are similar, but not quite the same.
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
Interesting ideas. Most of them make my head hurt, but Deprecated Votes could be useful in the situation you describe. Obviously there is the potential for abuse with this, but a practical way to implement this could be to just send an email to Chris&Co., requesting a vote reset (or just request a reset when you send in the new version).
The way the voting system is now, is more or less deterministic; I don't have to think of any fudge factors when I look at the article rating, or try to figure out if the voting system is being gamed. I can look at the cumulative rating, and the # votes per rating, and see what's going on.
What you're suggesting with most of these ideas would make the article rating more like the popularity score, which I still haven't figured out, and never use. I just don't see an overwhelming benefit, although I agree the voting system may not be perfect.
In the long term, I think the rating is an accurate gauge of article goodness.
|
|
|
|
|
Hans Dietrich wrote: a practical way to implement this could be to just send an email to Chris&Co
Yeah, I was thinking along those lines too. If the process were automated (such as by clicking a "request revote" button), the number of times a user could do this per article could be limited. For example, maybe a person only gets 2 revotes per article. This could get out of hand quickly, so there'd have to be some limit enforced. And we'd want to avoid email because it's cumbersome for the sender and the reader (in this case, Chris). And maybe clicking this button only deprecates votes older than, say, a month. And it must leave the last half votes intact. That way, people aren't completely flushing votes away each time they hit this button. Since they only get 2 of these and each one can only ever halve the number of votes, that would mean that a maximum of 3/4ths of the votes could be flushed away (i.e., 1/4th of the votes would remain intact, at a minimum). The 1-month limit makes it so people actually spend some time pondering how to improve their articles. And the 1/4th attribute means that not everyone's votes will go to waste.
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
We have thought and talked long and hard about this and have already investigated basic noise reduction techniques to look at how we could identify spurious votes.
The conclusion we've come to, however, is that if you do this then you need to have a minimum number of votes present before this can be done, otherwise the initial votes will "lock down" the score and any (potentially) correcting votes will then be considered "noise" and would not be used in the calculation of an article's score. After enough "correcting" votes have appeared then these (potentially more correct) votes would then start taking over and the score would revert to a more reasonable average, but until that happens the perception of an article's score will be that everyone things it's worth X, and there would be little to show that other's think (perhaps rightly) it's actually worth Y.
And the biggest thing, to me, is you can have 10 people rate an article 5 because it's pretty, and an expert can rate it 1 because it's completely wrong, and that single vote must be allowed to stand.
Our system is based on the idea that over time the score will settle down to what it truly should be.
As to deprecated votes, this will simply encourage members to make trivial edits to remove bad scores.
cheers,
Chris Maunder
The Code Project Co-founder
Microsoft C++ MVP
|
|
|
|
|
Chris Maunder wrote: And the biggest thing, to me, is you can have 10 people rate an article 5 because it's pretty, and an expert can rate it 1 because it's completely wrong, and that single vote must be allowed to stand.
I agree, which is why I didn't suggest outliers be adjusted... merely, that those who are outliers explain themselves (or at least make themselves known so that an author can then choose to ask that person about their vote). This would be especially useful in the "expert voter" example you give. If they are an expert, they are in an especially good position to provide feedback.
Chris Maunder wrote: Our system is based on the idea that over time the score will settle down to what it truly should be.
Some topics are so specialized that not enough people understand it enough to vote. And there are enough idiots out there (people with vendettas, others on ego trips, still others who are jealous, and yet others who vote based on how useful something is to them rather than on quality) that they can skew a rating significantly.
Chris Maunder wrote: As to deprecated votes, this will simply encourage members to make trivial edits to remove bad scores.
I made some notes to an above poster about how to mitigate this.
Chris Maunder wrote: We have thought and talked long and hard about this and have already investigated basic noise reduction techniques to look at how we could identify spurious votes.
You have "already" investigated, eh? My ideas are certainly not the full spectrum of the rainbow. Rather than leave things to fate and hope that things will balance out, I say keep the ideas flowing (to implement them may be ill-advised, but to keep them flowing seems harmless enough).
Chris Maunder wrote: the initial votes will "lock down" the score and any (potentially) correcting votes will then be considered "noise"
The algorithms I suggest will continually adjust for the new average score after each vote is cast. There will be no initial condition to be locked into. In fact, the dynamic deviation method I present explicitly handles initial conditions by giving them more variance (as a greater sample size is required for more precise indicators).
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
I agree that the more ideas, the better. Your 'deviance' idea is actually nice and simple. We could even extend this: Currently we force a voter to give a reason if the vote is under 3. We could remove this and make voters provide a reason if their vote is outside the deviation.
This would potentially decrease the usefulness of having members suggest improvements (always good) and may discourage good votes (also a bad idea if the vote is warranted).
In the end, though, if someone wants to game the system they will game it. By making the system more complicated all we do is reduce the casual gamers slightly (games will game, regardless) and make the system less transparent to the members using it.
cheers,
Chris Maunder
The Code Project Co-founder
Microsoft C++ MVP
|
|
|
|
|
Chris Maunder wrote: if someone wants to game the system they will game it
True. Maybe. Kinda like assuming P != NP. While it may be untrue, seems kinda hard to prove so might as well assume untrue for now.
Chris Maunder wrote: By making the system more complicated all we do is reduce the casual gamers slightly
True. Maybe. Perhaps we want that. Maybe not. Maybe newbies use a simpler system and veterans use a more complicated system. Or maybe it would be unnecessary by that time. Oh well. *throws hands up* 
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
aspdotnetdev wrote: throws hands up
Now you see why I'm rapidly losing my hair!
Somewhere between niavely simple and obscurely complicated lies a neat method that will work.
Somewhere.
cheers,
Chris Maunder
The Code Project Co-founder
Microsoft C++ MVP
|
|
|
|
|
Chris Maunder wrote: I'm rapidly losing my hair
Lucky you! I have the world's thickest most quickly growing hair, which means I have to get haircuts all the bloody time. Whenever I get it cut short, they can't seem to help be leave a stupid mohawk in it (something about the direction my hair naturally lays). I for one can't wait to go bald, so I don't have to maintain it anymore... I only hope I have the genetics to make it happen sooner rather than later. I'd probably shave my head, if I weren't such a wuss. Kinda like somebody who can't commit suicide... they know they want it, they just don't have the balls to do it themselves, so they wave a gun at a bunch of cops instead. I'm waiting for nature to take its course and rid me of this blasted hair.
[end tirade]
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
Yes, having a full, luxuriant head of hair must really get on your nerves. I can just imagine.
Let me just get my violin. 
cheers,
Chris Maunder
The Code Project Co-founder
Microsoft C++ MVP
|
|
|
|
|
Chris Maunder wrote: Let me just get my violin
The grass is always greener...
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
Chris Maunder wrote: Somewhere between niavely simple and obscurely complicated lies a neat method that will work.
Well, John is currently out of work. Maybe you could dispatch him to talk to members up close and personal when they vote an outlier. 
|
|
|
|
|
Chris Maunder wrote: over time the score will settle down to what it truly should be.
I agree.
What I don't like about a running average and the deviance idea is that my needing or not needing to justify my vote would depend on the order in which other people are voting. If a couple of trolls down-vote and somewhat justify their actions, I then have to justify my 4 or 5-vote; whereas under normal circumstances I wouldn't have.
Also, as an article is likely to start with a 5-vote, the deviance idea wouldn't be very different from the current way of things.
OTOH I would not mind always having to leave a comment; when I spend a few minutes reading an article, I don't mind spending another 10 or 15 seconds to comment and vote; in fact replying on a good article is more of an encouragment than simply voting 5. So maybe the voting tool should simply be part of the reply editor page (only for new messages), not the article page; then we don't need running averages and deviances at all.
PS: if we can't get it very good, lets keep it very simple.
Luc Pattyn
I only read code that is properly indented, and rendered in a non-proportional font; hint: use PRE tags in forum messages
Local announcement (Antwerp region): Lange Wapper? 59.24% waren verstandig genoeg om NEEN te stemmen; bye bye viaduct.
|
|
|
|
|
Making the voting process so complex that people would avoid voting may not be a good idea. It's probably better to allow a few people to 1-vote unfairly than to risk losing valid critical votes from honest peers. People don't vote enough already - I have observed some articles with a very high view count after a few days, and they usually have several messages too but with few votes. Making voting more of a hassle will mean fewer votes.
But you've got some great ideas there for sure.
|
|
|
|
|
Nishant Sivakumar wrote: It's probably better to allow a few people to 1-vote unfairly than to risk losing valid critical votes from honest peers.
Agreed. The trick will be to improve the voting system more or less transparently, so that the honest user need not be hassled.
Nishant Sivakumar wrote: I have observed some articles with a very high view count after a few days, and they usually have several messages too but with few votes
Probably doesn't have much to do with the voting system itself. I typically see tons of articles, but I read even fewer, and vote on even less. I only vote on an article after I have read it completely and feel confident I understand it well enough to make an informed vote (actually, I low vote bad articles without reading them fully because it would be a waste of my time to read them fully). This, I suspect, is why there are so few votes on articles. As far as the articles with a bunch of votes, I'm guessing it's from people who haven't actually read the article but vote it high because it looks cool.
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
Nice ideas, and I like the retroactive feedback.
When an article is first published, and there are few votes, it is impossible to reliably detect outliers. The retroactive system would solve that. Like it.
|
|
|
|
|
I like the idea. A low vote with no feedback will not benefit any party.
|
|
|
|
|
This suggestion is pretty weak, but I thought I'd put it out there anyway, just in case you hadn't considered it...
I noticed that the Top Ranked Articles page links to articles with an anchor that has "TopArticle" set as its target. Perhaps there is some design decision for that. However, it can be a nuisance. Suppose I right click on one of the links and select "open in new tab". That tab will now be set as the "TopArticle" target. Now, suppose I just click one of the other links... rather than nagivate to that page from the current page, the other tab that was opened will navigate to that page. This causes a couple issues:
- That the page was actually navigated to may not be obvious, as the user is expecting a new tab, a new window, or for the current page to navigate to the desired destination page.
- It loses the other page (i.e., the user may have forgotten that they were at another page and they might have just thought the click opened a new tab, causing them to skip over that first page entirely).
Neither of those are big issues, but you might as well get rid of the problem. I'm not quite sure why it was decided to set the target to TopArticle. Perhaps you expect the user to read one article and then click a link for another, thereby navigating away from the "old" already read article. Usually this type of link target would be used with a frame, but there are no frames. Anyway, I don't really see a problem with setting the target to nothing or something else (_blank?).
FYI, I use IE8 on Windows XP.
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
Hmm - how about that.
Target=anything is kinda dumb. I've removed it and will upload the fix in the next day or so.
cheers,
Chris Maunder
The Code Project Co-founder
Microsoft C++ MVP
|
|
|
|
|
I got the following error while attempting to post a reply to a message at the bottom of one of my articles:
Items that need attention:
•You must supply an email and name for this message.
•There were problems with your new message
After I got that message, I tried again to post the message, but it was taking a while so I just closed the browser window. When I later visited my article, I noticed that my reply had been posted.
Here is a link to the message I was trying to reply to: Message
Here is a link to my reply message: Reply
Looks like somebody else had this same issue before and reported it as a bug: History Repeats Itself
By the way, CodeProject was being REALLY slow during the timeframe this problem occurred. It took what seemed like a minute to load the CodeProject homepage. I'd put money on this being related to that slowdown, in which case the error message should say something like "our servers are experiencing some problems" rather than "your message is missing info".
Visual Studio is an excellent GUIIDE.
|
|
|
|
|
This happened because the weekly database update was occurring and for about 3-5 minutes the entire site was on a go-slow.
The error message, howeever, is just plain unhelpful. I'll get that sorted out.
cheers,
Chris Maunder
The Code Project Co-founder
Microsoft C++ MVP
|
|
|
|
|
greetings from somebody else.
![Rose | [Rose]](https://codeproject.global.ssl.fastly.net/script/Forums/Images/rose.gif)
Luc Pattyn
I only read code that is properly indented, and rendered in a non-proportional font; hint: use PRE tags in forum messages
Local announcement (Antwerp region): Lange Wapper? 59.24% waren verstandig genoeg om NEEN te stemmen; bye bye viaduct.
|
|
|
|
|
I just clicked on a link that took me to http://www.codeproject.com/info/submit.asp which gave a 404 Server Error.
Can you redirect this url to http://www.codeproject.com/info/submit.aspx?
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
