Feedback on NI Discussion Forums

[Matthew_Willia…]

I think the issue is one of 'star-inflation', to the point where 5 stars represents adequate or better answers,
1-4 indicate degrees of inadequacy.  For a rating system to be useful, it really should be analog; the
current system is digital.

Ideally we could re-calibrate the community's sense of stars to where 3 stars represents an adequate,
correct answer.  Something like:

1 - wrong thread, factually incorrect, etc.
2 - misunderstood, poor implementation, poor style
3 - correct answer, reasonable approach
4 - nice solution, maybe with a screen shot, sample code, unique insight
5 - spot on answer, nicely documented; +1 beer @ NI Week

I don't know how to change the community perception; we may have to change the 5 star scale
to a 6 kumquat scale and then, and this is important, post the new definitions of the ratings
right next to where ratings are assigned.

Just my $0.02.  I pretty much ignore the current system, except to thank someone for a nice answer
with 5 stars.  This is probably a 3-4 kumquat post

Matt


Just in re-reading the above, it occurred to me that we may be using stars for crossed purposes.
We are trying to show appreciation for effort _and_ evaluate the correctness of the answer with
the same indicator.  Maybe we need two indicators, a smiley thumbs-up (binary) to say thanks
and an analog indication of correctness, utility, etc. as above.

Matt

Message Edited by Matthew Williams on 06-06-2007 12:58 PM

[Kevin_Price]

More opinions = less consensus.   Here's mine:

1. The star system as it exists today isn't particularly useful, IMHO.

2. If there is to be a useful rating system at all, it needs more granularity than just YAY or silence.  I wouldn't bother checking film reviews from anyone who refused to use any number other than 10 or 0.  Not every film above 5 is equally good, not every film below 5 is equally bad.  Having more granularity can potentially be helpful.  It *could* be valuable to know whether the person answering your question has previously been rated at 1.5 stars vs. 4.5 stars.  It *could* be valuable to know which threads contain highly-rated responses.

3. I don't see how you can hope to keep a rating system pristine if it's left open to misuse / abuse by those with no stake in maintaining its integrity.  I haven't really thought about this much, but maybe we should have to "earn" the privilege to give numerical ratings.  Maybe it's "earned" by being either (1) the poster of the original topic, (2) someone who has made at least 25 posts that have been given "passing grade" rating, (3) a blue-bar NI insider.  Until you earn the right to give numerical ratings, you're limited to either YAY or silence. 

I doubt this type of rule can be implemented by the forum software though.

-Kevin P.

[Ben]

"..."earn" the privilege to give numerical ratings."

Good point Kevin!

The contributors could earn the tomatoes (sp) they throw rather than being given them for free.

Ben

[smercurio_fc]

I really like the idea of kumquats!



I had a real laugh when I did a Google image search for kumquats and came up with the Miss Florida 2005 Highlights, with Melissa Marcus as Miss Kumquat! We can have her as our spokesperson!

Message Edited by smercurio_fc on 06-06-2007 01:42 PM

[smercurio_fc]

One recurring theme that seems to be emerging from the replies is that if the star system is kept, then it must be clearly explained how it should be used, since at the moment it's too vague, thereby leading to inconsistent interpretations and frustration for the people who try to help others, only to be rewarded with a feedback which seems to imply that it just doesn't cut mustard (never understood that idiom). This is a valid point. It says "Rate this message". Well, what exactly are we rating? The accuracy of the response, whether it's an acceptable response, whether the problem has been clearly stated, or how good the person's grasp of the English language is? If it's an acceptable response, does that make it any more correct or wrong than another response, and who's to decide what's acceptable, given that this can depend on far too many variables, and even the user's experience level?

[Ben]

We have talked about the short comings of the star system many times.

Although there are many ways it can be fixed, they all have been to complicated for NI to fix.

Just tossing those four buttons from template used to create each reply appears to be a quick way to eliminate the rotten vegetables.

Ben

[Ray.R]

---

Laura F. wrote:
Hi Ben,

I am sure that there are many other forum users that agree with you on this issue.

I don't currently know all the ramifications of changing the rating system, but I am sure that there would be some problems to overcome.  I will make note of this suggestion and keep it in mind for the next time that we evaluate how to improve the Discussion Forums.

Is there a particular incident of abuse of the rating system that I should be aware of?  I don't have much control, but I can investigate the issue and watch for users that abuse the system.

Thanks,
Laura
Web Support & Operations
National Instruments

---

Well,

There was the latest incedent with parthabe  http://forums.ni.com/ni/view_profile?user.id=49330 See thread http://forums.ni.com/ni/board/message?board.id=130&message.id=3220

Seems like there is a single star bandit who gives 1-star rating for no obvious reason.  (Maybe they don't like the avatar? or abuse of smileys)

In some views (including the bandit), there are many 5-star bandits..  Which is worse??    (I'll even raise my hand at being a 5-star bandit)

There was a valued member who gave up when receiving a 1-star rating too many (in the older board).  Well, actually, there were other valuable contributors who went to infolabview or lava and seem to stay away from this forum.  From what I read, it appears the star rating influenced their decisions.

If changing the rating system is difficult, then maybe something simpler like turning OFF the rating system for contributors who have submitted >1000 answers (> 500 different threads?).  Surely, someone who answers 1000 questions must have some reasonable knowledge of Labview.  Maybe the answers are not the best approach or most elegant..  However, the questio should be "Did it solve the problem?"  If yes, then that's good.  If the solution provided can be done better, then post that in the "Rube Goldberg" thread. 

It does appear that people are troubled with the current system because this topic does come up very often.  How about changing the star system to a simple "Thank You".  You can keep track of how many times someone has been thanked..

This place is for people to help each other grow and learn more about the language.  I'm sure even Dennis learns something new once in a while when reading posts..  Isn't that what it's all about?  Not who has so many stars..   I have too many stars..  Please share them.. 

My CDN$0.01 contribution to this thread..

[Laura F.]

Thank you all for your feedback on this issue.

I take your feedback very seriously and will work to address this issue in the future as we prioritize projects for the Discussion Forum.  Unfortunately, the fix is not as easy as hiding the 1-4 star options on the page.  We would have to change from the 1 - 5 stars system to a "Recommend this message? Yes / No" system.  Even this would cause some problems with the "No" choice.  I think from what you are saying is that it would be best to have only one option for rating - "Yes, this helps!".  

If and when we start to work on the rating problem, we will have to consider any upgrades Lithium may plan for the rating system and also the fact that we would have to overhaul the user ranks to no longer rely on an average rating. 

Because I know this issue has come up a number of times, even since I've been the forum moderator, I will put it at the top of my possible project list for the Forum.

Regards,
Laura
Web Support & Operations
National Instruments

[Ray.R]

---

Matthew Williams wrote:

Ideally we could re-calibrate the community's sense of stars to where 3 stars represents an adequate,
correct answer.  Something like:

1 - wrong thread, factually incorrect, etc.
2 - misunderstood, poor implementation, poor style
3 - correct answer, reasonable approach
4 - nice solution, maybe with a screen shot, sample code, unique insight
5 - spot on answer, nicely documented; +1 beer @ NI Week

---

Here is what the "Help" button provides as info on rating answers:

"How do I rate a message?
Please take the take the time to rate helpful messages. To do so, log into the Forum and look for the Rate This Message (Thread) feature at the bottom of messages and threads.

Note: authors cannot rate their own messages and users may rate a specific message only once. All ratings are automatically averaged."

Now, many 5 stars are given / received for reasons completely different than anything technical..  Most ratings are probably found in the Breakpoint forum.   Why?

Maybe because the joke was funny.. or the topic was "right on target"... or appreciation (look at the Congratulations Threads!!!)... or to buddies...  or even to help out those who got unnecessarily low ratings..  or whatever reason...

If people understood that the rating system has become a bunch of stars (0 to 5) and not something tied to technical content, then there is no star rating problem...

Or should the rating system be "re-defined" or more clearly "defined"...

That's another Cdn$0.01  (earning $$ by posting)

<< BTW, Matt, you owe me a beer..   or is that the other way around??? you got 5-stars >>

JLV (aka. 5-star-bandit)

Message Edited by JoeLabView on 06-07-2007 11:41 AM

[paulmw]

I have actually shied away from replying to messages even if I had an answer or a good responce due to getting a few 1-stars in the past.

How does everyone feel about storing a list or ratings and the user that gave the rating.  For example, if it recieved stars, it would say the message was rated X times.  And then provide a link to see who gave what rating.

I think that would scare of the 1-star bandits out there. But is also could allow retaliations, etc.

Also, I know I like it when I get more then a star but also a reply that thanks me...