Public participation is about sustainable decision making. When I look at projects such as the Spending Challenge, I ask: Is there a decision at stake, first of all? If so, what is the decision? Who is the decision maker? Where in the decision making process are we? What is the timeline? What level of impact will participants have? What are the project objectives and what defines a successful outcome? Etc.

More often than not these questions do not get considered, hardly ever are the answers made explicit (which is a problem to begin with). We can assume, though, that the convener’s core objective is usually to make better decisions. Wether an online consultation is useful depends on how well it achieves that objective.

There may be other aspects of usefulness, but my research is focused on how the tools in combination with the (not always intentionally designed) process can help or hinder good public participation.

But there are also a disruptive phenomena and I’m wary of being the “they” in Gandhi’s ““First they ignore you, then they laugh at you, then they fight you, then you win.”

So I’m aggressively open-minded about how online challenges are, currently and flawed as we see them to be, nonetheless useful to their sponsors, and I’m preoccupied about which of the flaws *we* see would be seen by their current and future sponsors as most useful to address. (E.g. Grandpa tells me that mp3s don’t have the audio range of his CDs, and I don’t care – Coldplay sounds fine to me. Grandpa tells me that DRM-encrypted mp3s could be “repossessed” by a server failure or when I copy my music to a new iPod, and I get that it’s an issue.)

However, here’s what usually happens (in no particular order):

* Voting takes place prematurely
* The simple thumbs up or thumbs down voting mechanism provides a very limited form of evaluating idea fragments
* Evaluation criteria are never clearly defined nor agreed upon upfront
* The conversation focuses on (half-baked) positions rather than underlying values and interests
* Lack of demographic information makes it difficult to put participants’ collective choices in perspective

At the end of the day the question is: what do the results mean and what can and will the decision maker do about them? Most projects I’ve seen don’t seem to think that part through to the end.

Re #1 – (a) It’s not inherently impossible to rate policy ideas in a way that’s acceptable to a broad array of people (fiendishly difficult, yes; impossible, no). The US Congressional Budget Office scores legislation for financial and other impact, so, if we agree to go by their estimates, we could have compared the various health care reform proposals “objectively”. (Big “if”, of course.)

(b) Cooperation does not require objective rating. If “winning” ideas must first garner at least 5% of the votes, say, a kind of cooperation and blending of higher scoring ideas might be encouraged. (This is in fact what happens in Congress – factions come together in all sorts of odd groupings to pass legislation.)

Thanks for chiming in!

True, the Netflix Prize is widely regarded as one of the success stories of corporate crowdsourcing.

However, their objective was the improvement of a mathematical algorithm, not the creation of public policy. Two key differences:

1. The competing teams’ algorithms could be compared and ranked applying objective criteria. This is not the case with policy ideas.

2. The leader board may have worked well for Netflix, but looking at previous efforts in the policy arena, exposing the leader board while the voting process is still ongoing always seems to lead to serious bias favoring the early leaders, at least based on what I’ve been able to observe (see http://www.intellitics.com/blog/tag/bias/).

I agree, though, that there are still many options that haven’t been fully explored. More iterative approaches that include the opportunity for merging similar ideas as well as more creative voting models come to mind, for example.

It seems to me that an even more important feature of the Netflix Prize – http://en.wikipedia.org/wiki/Netflix_Prize – was that teams were competing as much against “the course” – beating the existing algorithm by at least 10% – as against each other. (And indeed the winning team was a combination of three of the originally separate teams.)

So, encourage cooperation by (1) pitting contestants “against the course”; (2) making “the course” difficult, so it’s possible that no one wins; (3) using a leaderboard to highlight leading teams; (4) running the competition long enough that contestants have time to team up; and (5) having a prize worth winning. (Of course, there are other ways to encourage cooperation as well.)

In a simple “vote up ideas” contest, (1) and (2) might be handled by requiring that the winning ideas garner votes from at least, say, 5% of the voters.