In the second chapter from his upcoming book, Tom Atlee picks up the topic of learning in deliberation, something he has written about previously. It’s a thorny issue and generally applies to public participation as well.

One problem, as I see it, is that quality deliberation requires informed participants, but properly framing the issue and creating the necessary briefing materials can be quite costly. The challenge is how can we meaningfully involve participants in the co-creation of the briefing process and its outcomes in order to improve quality, lower costs and help make quality deliberation more widely available.

Here’s Tom’s take (as pre-released on Reality Sandwich last month):

Deliberapedia

As noted earlier, framing an issue for deliberation means providing balanced information that helps deliberators take into account the range of views on their issue and the trade-offs connected to whatever choices they might make. Traditionally, it involves condensing a lot of information about that issue into 3-5 approaches for addressing the issue — representing as broadly as possible the full public debate — with the arguments and evidence for and against each approach. Sometimes issue framings also include information about who supports and opposes each option, and a profile of the values that it represents and appeals to.

Most citizen deliberations are framed by professionals who produce “issue books”, videos and other briefing materials, many of which are available at low or no cost, but are quite expensive to put together in the first place. Framing for broad self-organized grassroots deliberations, in contrast, would be crowdsourced, using the fact that advocates for various solutions to a public problem have already developed arguments for their solution and against their opponents’ solutions. Our challenge is to create a context where opponents in the fight over an issue end up participating in co-creating a wiki that channels their information into a coherent frame that clarifies that issue for everyone else. Most of the partisans involved would not participate out of their civic-mindedness but because they wanted their viewpoint to be well represented in this public document. This is the idea behind “Deliberapedia”.

The Deliberapedia vision is inspired by Debatepedia, a leading debate society’s online forum to collectively work up and share arguments pro and con various propositions, creating a database that can be used by debaters everywhere. Deliberapedia would be a massive, readily searchable, rapidly expanding and developing wiki database of organized arguments for and against all sorts of policy solutions to all sorts of public issues.

Deliberapedia would provide a powerful — perhaps even necessary — foundation for a self-organizing grassroots citizen-based deliberative system capable of generating empowered public wisdom with minimal ongoing cost. It would also constitute one of the greatest contributions we could make to democracy even if the rest of the deliberative system for which it was designed is never developed.

Note from the author: The final version of Empowering Public Wisdom will include an appendix showing one way in which Deliberapedia could function, including a special network of grassroots groups focusing on issues they’ve chosen, as well as a chapter on the creation of official legislature of ordinary citizens, who could both contribute to and use Deliberapedia.

It’s still true today that “[i]nside every public participation program is a good public information program.” However, I think we need to be moving from simply informing participants (if understood as a top-down, one-way activity) towards a more participant-centric model that takes into account the entire learning experience. Last Spring at SXSW, I took the liberty to slightly rephrase the guideline as follows: ”Inside every public participation program is a good public learning program.”

The wiki approach Tom proposes has opportunities but also many challenges. In order for the whole to be greater than the sum of its parts some level of editorial leadership would still be required, and the need to coordinate non-expert contributors will create certain overhead costs which may at some point outweigh the benefits.

In the context of specific projects, though, one possible solution might be found somewhere in the middle between a top-down and a purely participant-driven (crowdsourced) approach. As I started to outline last year, I agree there is a lot of potential in granting the participants a much more active role in this important pre-phase of any deliberation, consultation or general public participation effort:

• Building on the diversity of participants’ knowledge and experiences could help improve briefing materials in terms of scope, accuracy and accessibility.
• Giving participants ownership of the research process and the outcomes it produces may build buy-in and increase trust.
• Offering participants more variety in the ways they can contribute (e.g. by taking on the role of “researcher, interviewer, fact checker, curator, editor etc.”) might increase overall engagement, with more people participating at a higher enjoyment factor and hence more likely to stick around, invite others etc.

I’d be interested to know who has integrated innovative content co-creation and e-learning components into their online consultations. If you have any leads, please leave a comment. Thanks!

Early on during the Expertnet consultation last year, I shared the following (sadly, the Expertnet wiki is locked due to its paid subscription having expired, but Google still has most of it cached):

re: Notifying Experts
tbonnema Dec 18, 2010 12:43 am

It looks like ExpertNet is trying to solve at least two distinct problems:

1) The matchmaking piece between experts (however defined) and opportunities to contribute: For some time now, I have suggested the creation of a national participation calendar (http://bit.ly/h7mwmv, http://bit.ly/6h18sc), so maybe this is something that ExpertNet could help accomplish.

2) The consultation piece once a group of participants has been assembled: There is already a broad range of tools available today to support these kinds of processes (see http://ParticipateDB.com). Depending on the circumstances, some are more appropriate than others, yet success will to a large extent depend on “soft factors” unrelated to the technology. My question is whether ExpertNet should build or buy a solution or whether it should rely on existing tools on an as-needed basis.

(Full disclosure: My company created ParticipateDB, and we’re developing a tool in this general e-consultation category.)

Just as a reminder, the range of processes Expertnet would potentially aim to support basically came down to this key question:

2. Fact-based advice or deliberative policy creation? The original draft focused on seeking verifiable, fact-based advice from citizen experts. However, later statements have hinted at a significantly broader scope that would allow officials “to pose questions to the public about any topic we’re working on.” Given the nature of many of the topics mentioned (job creation, preventing homelessness among veterans etc.), ExpertNet would have to support policy consultations that are much more deliberative by nature. This has huge design implications.

This general idea of a more agile approach whereby citizen expertise is solicited following a variety of processes and using a variety of existing tools was supported by several other participants. Here is Tim Huegerich (Google cache):

The benefits of using already established, external platforms for responses
TimHuegerich Dec 30, 2010 9:23 am

1. Easy and cheap – less work and expense for setting up ExpertNet
2. Allow the external organizations to perform moderation, etc., sidestepping concerns about how to moderate without the appearance of “Big Brother” trampling free speech.
3. Furthers the original principles:
   • Participation must be easy. (by allowing participation through platforms than citizens are already familiar with)
   • We are not building a new Facebook. (the original proposal does not really come to terms with this principle because it assumed that a community would form around the new platform to moderate responses, etc. – this alternative is much more consistent with the principle)
   • Innovation requires experimentation. (By allowing various partner platforms to compete in generating useful responses, this proposal encourages experimentation and innovation.)

David Karger also made the same point (on Januar 23, 2011), saying that “… I will argue as I have elsewhere that instead of building a [government-sponsored social networking site] we should look to leverage an existing system.” (Google cache).

They really did get a lot of smart answers during the Expertnet consultation, didn’t they?

To this date, there has never been an official follow-up or debriefing, so we can’t know for sure whether the current experiment is intentional or even related. But last week, the White House announced on their blog that they would be using social knowledge site Quora to solicit input on a number of questions: We Want to Hear from You on Quora: Announcing the Startup America Policy Challenge

From the post:

Yesterday, the White House announced $2 billion in public and private resources to help entrepreneurs start and grow their businesses. In the spirit of open and participatory government, we also announced the Startup America Policy Challenge. We’re calling on entrepreneurs and the broader public to share their ideas on how to accelerate entrepreneurial innovation in the areas of healthcare, energy and education. Aneesh Chopra, US Chief Technology Officer, kicked off the challenge in a post on Quora and asked a few questions to get the dialogue going.

[...]

All Americans are invited to reply. But I’m especially interested in hearing from entrepreneurs in these areas – and so is President Obama’s Cabinet, particularly Energy Secretary Steven Chu, Education Secretary Arne Duncan, and Health and Human Services Secretary Kathleen Sebelius.

You don’t have to be a policy wonk to respond, and you don’t need to describe a detailed policy solution—there are others in government and in academia who will help us with that, as part of the Startup America Policy Challenge. Just tell us what’s on your mind.

So have a good idea? Let us know…

Screenshot 1: Top of page

Screenshot 2: Question metadata

Let’s look at participation metrics to date for the three questions posted:

Education: In the U.S. education system, what can the government do to best enable the use of new learning technologies?

• 18 answers
• 12 comments
• ca. 70 votes
• ca. 6,000 word count (not including comments)
• ca. 325+ average word count per answer
• 516 views
• 53 followers

Energy: In the U.S. energy system, what can the government do to best enable the use of new clean energy technologies?

• 22 answers
• 29 comments
• ca. 50 votes
• ca. 6,000 word count (not including comments)
• ca. 250+ average word count per answer
• 609 views
• 65 followers

Healthcare: In the U.S. healthcare system, what can the government do to best enable the use of new health information technologies?

• 8 answers
• ca. 40 votes
• ca. 3,500 word count (not including comments)
• ca. 425+ average word count per answer
• 297 views
• 37 followers

At a glance, it looks like overall participation rates are still fairly modest so far. However, individual responses are much more detailed as compared to the roughly 125 average word count per comment we saw during previous online dialogues on Change.gov and the Open Government Dialogue (see our metrics update from June 2009).

Quora provides a few distinct features that allow participants to collaboratively improve their answers over time. Unfortunately, there is no easy way to measure the level of activity various answers have seen or whether they have been modified, but it is something that occurs regularly on the site. This an other design choices, such as the limitation to allow only one answer per participant, can greatly help improve quality and reduce overall word count.

As far as I’m aware, no end date has been defined by which input has to be submitted in order to be considered, nor has it been made clear what will happen with the input. Also, I’m not seeing the conveners (Aneesh Chopra and team) actively engage.

Having said that, this is a promising approach that will further advancing the idea of harnessing citizen experts. Assuming that the policy questions at hand are actually ripe for public input, using an existing tool is one way to get there relatively quickly. Given the intended audiences (technology entrepreneurs) and the type of input the White House is after (policy ideas), Quora appears to be an appropriate choice of tool.

If Expertnet were ever to be built as its own tool (which may or may not be necessary), it is experiments like this that will inform its design. Hopefully, we’ll see many more of these.

I’m quoted in the Economist today:

Government by (all) the people

[...]

Successful examples of legislation by the masses are rare. Most people don’t know how to write laws. Tim Bonnemann, the founder of Intellitics, an American firm specialising in public-participation tools, says a better method is to canvas views widely but use a small team to write a draft. The hard part is not the technology (a simple online discussion forum is adequate) but creating a fair and transparent process that assures people their voices have been heard.

For a more detailed take, check out our three-post series from March of this year:

• How To Create Policy on a Wiki
• Blast from the Past: Experiments in Wiki-Based Political Discourse from Way Back When
• Challenges for Wiki-Based Consultations

If you speak Russian and find the time to review the Wikivote.ru site, please share your thoughts in the comments. Thanks!

When Give a Minute launched in Chicago back in November, I asked what impact participants would have.

This recent article in Fast Company’s Co.Design seems to confirm my doubts: Looking for Bold Ideas to Fix the City, New York Turns to Crowd Sourcing

In Chicago, the project, which ran from the end of November through mid-December, generated more than 2,000 new ideas. But the mechanism for linking these ideas with actual action groups or agendas was not yet in place. In that sense, the New York project will be “Give a Minute 2.0″ — in which the infrastructure for actually addressing problems will be built into the platform.

Not only was there no capability yet for participants to link up, it doesn’t seem like their input made it in front of decision makers at all (assuming that was the goal of the exercise).

The screenshots shown in the Co.Design article now hint at some of the functionality that was still missing during the Chicago project (mainly, the ability for participants to join so-called action groups).

Give a Minute‘s New York edition is said to launch “in late April or early May”. According to the article, ideas will be “funneled to organizers in city government who will connect people with similar ideas to action groups organized around potential solutions.”

The project appears to have garnered the support of Mayor Bloomberg, who “plans to use Give a Minute to surface innovative ideas from citizens and give them a platform for organizing to turn their ideas into action.”

If you look carefully for any commitments regarding the impact participants will have on City decision making, there isn’t any. To me, what this translates to is this: The City will provide participants with a venue to connect around ideas, but participants will be on their own to figure out if and how their ideas can be realized.

If that’s the case, maybe Give a Minute is an effort that more accurately belongs in the civic engagement category, not the public participation category.

Peter Levine wonders whether we could “crowdsource civic renewal” and in his post shares a couple of key observations that relate to the crowdsourcing of policy, its challenges and the need for deliberation instead (emphasis mine):

At first national meeting of the Coffee Party, in Louisville, KY, the legal scholar and activist Lawrence Lessig electrified the audience with a proposal to “crowdsource” campaign finance reform. In contrast (although not in opposition) to the traditional campaign finance reform organizations, such as Common Cause and the League of Women Voters, Lessig had created a loose, online network called Fix Congress First to lobby for reform. Visitors to his website were asked to organize local house parties, “spread the word,” pledge not to contribute financially to any federal candidates who refused to back reform, and contact Members of Congress. Because of a combination of its goal (nonpartisan political reform) and its format (loose, voluntary, and viral) Fix Congress First was a perfect match for the Coffee Party.

It would be risky to make any predictions about these developments so early in their history. I certainly hope they succeed and believe that they will contribute to the goals I care about. Yet I doubt it is fully possible to crowdsource civic renewal. Jay Rosen, a journalism professor who has deep understanding of democratic theory and civic themes, has been experimenting with crowdsourced journalism projects: efforts to generate valuable news and information by issuing open calls to volunteers. He has observed three preconditions for success. First, in a crowdsourced project, because people no longer sit together to discuss assignments, you need “extreme clarity about tasks and goals.” Lawrence Lessig, for example, asks volunteers to call specific members of Congress to ask them to support particular legislation. Rosen has had equal success posting lists of people who need to be interviewed and asking volunteers to conduct the interviews and post their notes online. But asking people to construct a whole news article, design legislation, or govern a local asset would require too much discussion and deliberation to succeed by crowdsourcing.

Second, an open call for assistance must go to a pre-existing group with a “shared background narrative.” In Rosen’s example, participants in the liberal blog Talking Points Memo were able to collaborate online on very short notice to scan a ream of leaked Justice Department documents to find embarrassing evidence about the Attorney General. They were successful because they already agreed on that goal, its importance, and what would count as relevant evidence. Most of the prominent examples of successful crowdsourcing come from domains such as software design, in which the goals are fairly self-evident. But politics is laden with values and is profoundly contentious, so that virtually no two people have exactly the same political objectives and beliefs. If their relevant values and ideas differ, they need to talk before they collaborate. They can certainly talk online instead of face-to-face, but their conversation needs structure and moderation. They can only crowdsource a problem once it has become a discrete element of some larger political project that they already share.

This fits nicely with my earlier comparison between crowdsourcing and public participation.

Give a Minute Chicago is a new web and mobile application that asks citizens in the greater Chicago area to submit ideas about what would encourage them to walk, bike or take public transportation more often.

Here’s how CEOs for Cities, initiators behind this project, announced it on their website earlier this month:

Give a Minute for Chicago

“Hey Chicago, what would encourage you to walk, bike or take CTA more often?” CEOs for Cities launched the Give a Minute for Chicago campaign this week to help answer that question. Give a Minute is a new model for citizen participation. It’s an easy way to share ideas about how to make Chicago an easier place to get around without owning a car, connect those ideas with change-making community leaders, and make things happen. And citizens only need one minute of their time for this interaction.

Citizens can text their ideas to 312.380.0436 or post them to the Give a Minute website at www.giveaminute.info. These ideas will guide recommendations during the Connectivity Challenge in Chicago December 8-10. The Give a Minute for Chicago campaign was made possible through the generous support of the Chicago Transit Authority.

A couple of screenshots below:

I had already shared a few first thoughts and questions elsewhere which I’m collecting here.

Here’s the comment I left on November 15 on the Next American City blog: Creating a Culture of Civic Engagement

Looks nice!

A couple of questions:

1) What is the timeline for this project? When will the period for public input end?

2) Is this project tied into any actual government decision making processes? What are the “key local officials” committed to do once they receive the analysis?

Thanks!

Here’s another comment I left on the AmericaSpeaks blog from November 17:

The site promises participants they are entering into “dialogue with change-making community leaders”. Another article mentioned that an analysis of the input will be presented to “key local officials”. The question always is: what impact will citizens and their ideas have and when? My recommendation would be for the site to spell those things out in much more detail.

Otherwise, it’s a visually appealing presentation. The integration of text messaging is nice.

For the purpose of collecting large amounts of raw input I think this is a valid approach. Will be interesting to see what their participation rates are.

And finally today, in response to Allison Arieff’s enthusiastic review in the New York Times (see: The Public Square Goes Mobile), here’s the comment I just left over on the CityCamp Exchange:

Two basic questions to ask when reviewing projects like this one:

1. What level of impact will public input have?
2. Will the process and the outcomes meet participants’ expectations?

The level of impact is not clear. Sure, all ideas will be “reviewed” and some may even get personal responses from prominent local figures. But are decision makers on board with this? Are all the right stakeholders being involved? Has anyone committed to actually doing anything about these ideas? In order to have any impact that’s usually what’s involved *before* the public gets tapped for input.

Has a process been defined? Will the outcomes be in line with expectations? Obviously, that remains to be seen. But intentionally or not, the website raises high expectations. Describing the project as “a new kind of public dialogue” with “change-making community leaders” to help “[make] your city [a] better place” doesn’t exactly promise anything. However, to the average participant it has high impact written all over it. And once hopes and expectations have been raised it’s usually impossible to trim them back.

So will this “reinvent public participation for the 21st century” (as quoted in the NYT article)? At this point, I’d feel more reassured if I knew they had at least done their 20th century homework.

Too harsh? Maybe. At first glance, the site design is quite refreshing.  The use of mobile seems very appropriate. The issue is phrased in a positive and upbeat manner. And at 1,000 submissions since launch it looks like they’re starting to get some traction.

And who knows, sometimes what happens is that despite having done all the right planning a project may simply forget to outline or reference the basic parameters and background information on their participation app.

Looks like the Citizen Participation Platform, an idea introduced by the UK Conservative party during the campaign there earlier this year, won’t happen anytime soon.

According to this report in The Guardian:

[...] But another ambitious idea about how best to harness the power of the internet has also been dropped.

The Tories announced a competition last December to find a new technology platform that would enable the public to discuss how best to solve local problems online. Jeremy Hunt, then shadow culture secretary, said a future Conservative government would publish all green papers on the platform. That idea has been shelved too, despite its “big society” credentials, perhaps because the winner was due to receive a £1m prize. Handing out seven-figure sums to geeks is something the public is unlikely to stomach. Online petitions, meanwhile, are simply a headache for politicians.

Empty campaign promise or genuine re-assessment — anyone know what happened here?

This year’s Future Democracy ’10 will address the issue of crowdsourcing in policy making, among other things. From the latest bulletin:

The question of whether ‘crowdsourcing’ techniques – inviting open online comment from the public – are a meaningful way to develop government policy is one of the main topics of debate at this year’s Future Democracy ’10, Headstar’s annual event on the use of the internet and other new technologies to boost democracy.

Since coming to power this year, the UK government has run three major ‘crowdsourcing’ consultations: the ‘Programme for government’, letting people comment on the coalition’s plans; ‘Your Freedom’, asking people what laws they feel are unnecessary; and the ‘Spending Challenge’, inviting ideas for cuts. Results have been mixed, with critics attacking the quality of submissions to these websites, and defenders, including ministers and civil servants, pointing to a small number of concrete actions attributed to ‘crowdsourced’ ideas.

At Future Democracy ’10, set for 2 December in central London, Alex Butler, Transformational Strategy Director at the Central Office of Information (COI), will speak for the motion, with Anthony Zacharzewski of the Democratic Society opposing.

This should be an interesting conversation, especially now that we know that Programme for Goverment wasn’t at all intended to be a consultation and had nothing to do with crowdsourcing (see: Commenting is not crowdsourcing by Simon Dickson).

As for the other two projects, do we know yet to what extent the input gathered is actually informing the policy making process?

From an otherwise interesting post on Govfresh last week about the use of contests in citizen engagement: Collaborative innovation in open government: Is there an app for that?

Could contests help us realize the vision of participatory democracy outlined by Thomas Jefferson, where citizens collaborate with government to solve the nation’s most difficult problems? The White House hopes so. As the Federal Times reported this morning, agencies are trying to crowdsource their way out of problems.

These efforts won’t always work out as proponents might hope. To date, crowdsourcing government reform has had mixed results. The new British government’s first crowdsourcing attempt fails to alter Whitehall line. And as Wired’s Jeff Home observed last year, crowdsourcing and the President were a “failed marriage” when the new administration tried its first online town hall.

Not to be too picky, but here’s the comment I left (as of now still awaiting moderation):

I have one simple request:

Could we please stop referring to every online comment form under the sun as “crowdsourcing”? Please!

The “Programme for Government” website, which the new UK government launched shortly after it came into power in May of this year, had nothing whatsoever to do with crowdsourcing. Blindly repeating The Guardian’s misleading headline doesn’t make it so.

Here’s what actually happened:

After the usual negotiations, the Conservative Party and the Liberal Democrats had come to a definite agreement on which to base their coalition government. In the spirit of transparency, they presented the final document on a public website. Yes, they offered citizens the opportunity to comment. But no, it was never their intention to engage in a two-way conversation, let alone incorporate any of the feedback and make changes to their program based on citizens’ input.

No crowdsourcing there!

Granted, they did do a rather poor job communicating the scope of this exercise, which led many participants to believe the document was still open to modifications (hence the disappointment), but that’s another issue entirely.

So there.

Here’s what Simon Dickson, who built the Programme for Government website, had to say recently: Commenting is not crowdsourcing

Now, I’m speaking for nobody but myself here – but what the Guardian piece doesn’t fairly reflect is that it was not a crowdsourcing exercise, nor even a consultation.

It was the definitive statement of the outcome of negotiations between the two parties currently forming the country’s coalition government. It was not ‘give us some ideas for what you think we might have agreed.’ The comment box provided an opportunity for people to voice opinions or ask questions, and government promised it would listen.

There was no commitment to take the responses back for a second round of coalition negotiations. To do so would have been quite ridiculous. So I’d argue that it’s entirely reasonable for the departmental responses to take the position of ‘well, we’ve heard what you say, but…’.

I previously wrote about Programme for Government here and here.

Doing a bit of research on Spending Challenge the other night, Stephen Whitehead alerted me to his excellent post on the subject: Three lessons from the Treasury’s Spending Challenge fiasco

The article touches upon three important concepts (great analysis, make sure to read it in full):

1. Asking the right questions
2. Collaborative brainstorming
3. Objective-driven public participation

I wanted to highlight the section that deals with the crowdstorming aspects of the Spending Challenge online consultation:

2. Good ideas are like Lego: some assembly required

In the Spending Challenge, ideas are like property: you’ve got yours and I’ve got mine. Every suggestion made, no matter how bizarre or unworkable, is a complete, perfect whole. It can be commented on, rated, but never improved. And in the end, each idea is in competition with every other for the meaningless title of ‘highest rated’.

In the real world, that’s not how things work. The best ideas are shared, discussed, revised, merged, and split. They grow and evolve as people work together. We brainstorm, we collaborate, and at every stage our ideas get better.

The virtue of holding this kind of conversation online should be that it pools peoples’ [sic] knowledge to produce a whole that is greater than the sum of its parts. [...] But collaboration around ideas – not the ownership of ideas – must be built into the process from the very start.

To me, this is one of the central criticisms of the way large-group online idea generation efforts in the area of policy making have usually been designed to date. It applies to most if not all of the prominent “brainstorming” contests we’ve seen over the last couple of years (e.g. on Change.gov and during the Open Government Dialogue).

Here’s what I wrote back in January 2009:

Having looked at a number of large-scale online input gathering efforts recently (Change.gov, Rebuild the Party, ObamaCTO and others), I believe what’s missing in order for any of them to scale in meaningful ways are processes that allow the participants to spend some of their energy on collaboratively improving (e.g. to review, revise, clarify, categorize etc.) and synthesizing (e.g. to de-dupe, group, merge, summarize etc.) the massive amounts of content that they create.

There are other issues that remain to be solved, for example:

• Who are the participants and how well do they represent the public? Ranked lists of ideas are relatively easy to generate online but may fail the legitimacy test.
• How will ideas be evaluated and by whom? Evaluation process and evaluation criteria should be established and communicated before the brainstorming begins.
• How will participants know their ideas have been reviewed, considered or implemented? At last count, there were 19,630 ideas and 34,568 comments. Given these massive numbers, it’s not unlikely that a lot of necessary follow-up communication won’t happen.
• How do we know if a brainstorming exercise was a success? Once again, criteria and success metrics should be defined upfront rather than after the fact.

I have yet to see a project that has addressed these challenges (and well). If you’re aware of any examples, please leave a note in the comments. Until then, I say we’re just not quite there yet…

In what is turning out to be a truly fun excercise, This Week in Participation (our new little internet radio show) has meanwhile cranked out a couple more episodes:

• TWiP 2: Crowdstorming
• TWiP 3: Crowdsourcing in Urban Planning

Both sessions came in at under 20 minutes each and together make for a nice follow-up to a lot of posts on this blog. If you’ve been reading here over the past year, you should enjoy the insights shared by both our guests.

Last week, Federal Computer Week published an op-ed they had invited me to write on the issue of crowdsourcing, public participation and how the former might be applied in the context of the latter: The outer limits to the crowd’s wisdom

If you are a frequent reader of this blog, you know that this is something I’ve been trying to wrap my head around over the past few months, especially since the conversation on the web continues to be characterized by confusion about concepts and terminology. The article is a good interim summary of my efforts, and more aspects remain to be explored.

I would like to thank the many who have contributed to this discussion in the comments, via the NCDD mailing list, on GovLoop and elsewhere on the web. Sharing your thoughts so generously has certainly helped me clarify mine. Please keep it up!

This post is not a deep dive into the definition of crowdsourcing but rather a quick mental note for myself.

Jeff Howe, who coined the term crowdsourcing back in 2006, offers this “white paper version” of a definition in the sidebar of his blog:

Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.

His “soundbyte version” appears to be slightly broader in scope:

The application of Open Source principles to fields outside of software.

Howe’s original article appeared in the June 2006 issue of Wired Magazine: The Rise of Crowdsourcing

Here’s the list of examples that are mentioned in the article:

• Stock photography — The National Health Museum in Washington needed pictures of sick people and licensed 56 pictures through iStockphoto – for about $1 each.
• TV shows recycling user-generated content — Various TV shows have come up with the concept of featuring user-generated video content that has shown to be particularly popular (viral) among internet users during a given time period.
• Online marketplaces for R&D — InnoCentive has a network of (then) 90,000 amateur researchers that work on solving scientific problems the participating companies and R&D labs throw at them, with successful contribution earning considerable rewards.
• Amazon Mechanical Turk — Amazon Mechanical Turk is a web-based marketplace that “helps companies find people to perform tasks computers are generally lousy at” (e.g. image identifying, transcribing podcasts etc.)

Along with Wikipedia and Open Source in general, these are some of the most-commonly referenced examples in the discussions around crowdsourcing and public participation. However, as per my previous outline, I am not convinced they really apply.

At least at the involve/collaborate/empower level of IAP2’s Spectrum of Public Participation (PDF), public participation at the core is an integration process. The question remains if and to what degree this process can be crowdsourced.

Richard Fahey has a detailed post up about an interesting crowdsourcing idea that has been proposed by the Conservative Party in the UK: £1m prize for citizen participation platform

Earlier this week the UK Conservative party promised to offer a £1m cash prize to a person or team that creates an online platform that can be used to solve “common problems”.

The prize – which the party says will be the largest offered by a British government in modern times – will be awarded for a platform in which citizens can post ideas in relation to government policy. The exact specifics of the platform have not been outlined, but it’s envisioned as a mesh between Fixmystreet,  Facebook, Spigit, IdeaScale and MixedInk.

The platform will need to be able to sift through millions of online comments and highlight the most sensible suggestions from those with experience of the area in question. Most current idea generation platforms use digg-like voting mechanisms as a means of highlighting the most popular suggestions. The £1million prize is on offer to anyone who can devise a more sophisticated way of sifting through suggestions and weighting relevant ideas in an appropriate manner.

According to the press release (quoted on the IdealGovernment from an email), the end goal here is to create a citizen participation platform that enables the soon-to-be-elected new UK government to — among other things — “tap into the wisdom of crowds to resolve difficult policy challenges”.

The press release mentions a number of examples where some kind of online collaboration among citizens could be quite useful (e.g. identifying wasteful government spending, co-creating government how-to information or mapping out traffic routes around road construction sites). But it also goes into the area of public participation.

My comment on Richard’s blog makes for a nice follow-up to my previous attempt at comparing crowdsourcing and public participation, which is why I re-post it here:

Great post, thanks for the detailed write-up!

In the original email from MP Hunt (as quoted in the IdealGovernment post), he describes what they have in mind as “an online platform that enables us to tap into the wisdom of crowds to resolve difficult policy challenges. In government, we will use this platform to publish all Green Papers, and open up the entire policy making process to the public.” The press release goes on to state that using this platform the public would be able to “collaborate to improve government policy.”

Policy making ultimately means having to deal with difficult trade-offs and making tough choices. Contrary to the previous commenters, I’d argue that it remains a huge challenge to meaningfully engage citizens in this process, particularly online.

While the crowdsourcing initiatives that are often mentioned in this context (e.g. FixMyStreet, the Netflix Prize, the Next Stop Design contest etc.) may vary in terms of problem complexity and a few other aspects, they seem to share — to some degree, at least — a number of key characteristics:

• Nice-to have (non-critical projects, ok to cancel at any time)
• Not very time-bound
• Objective evaluation criteria or success metrics
• No concept of “stakeholders”
• No need for representativeness or inclusion (the requirement to have all stakeholders at the table)
• No need for consensus seeking/building among stakeholders/participants
• No need for deliberation

Unfortunately, public participation (engaging citizens in decision making) is almost never lucky enough to rely on conditions as easy as these.

There are a number of online tools out there that seem to support the process needs of public participation fairly well. However, they still tend to require a high degree of human moderation and facilitation (in essence, exception handling), which makes them really hard to scale. That, in my view, is the real challenge that a “citizen participation platform” contest might help address.

Just to state the obvious, here’s how an average policy issue at the local level, such as a broken city budget or a contested urban planning project, might differ from the conditions outlined above — further indication that crowdsourcing as we know it may not easily apply to public participation (or at least not in the straightforward ways that many seem to suggest all too eagerly):

• Critical issue (high impact and real consequences, decisions can’t be avoided)
• Critical timeline (internal/external dependencies, decisions can’t be postponed beyond a certain point)
• Often very subjective and/or conflicting evaluation criteria based on personal values and preferences (just to agree on the same success metrics or a formula for evaluating policy proposals and ideas may be a challenge of its own)
• Stakeholders (failure to involve the right people at the right time can seriously derail the overall process)
• Inclusion is key (failing to bring all major parties to the table can pose serious risks to the overall process)
• To achieve some degree of consensus is often desirable or needed (that means a lot of synthesizing and integrating of differing viewpoints and opinions is necessary, an often slow and painful process that requires good process design and skilled facilitation)
• Deliberation (required as one preferred method of allowing larger groups to work through a decision-making process)

This is just a quick list of differentiators I came up with on the fly; there may be more, of course.

At this point in time, I think it’s fair to say that a lot of work remains to be done before we see technology that can handle these requirements. Running a contest that might help improve our understanding of the challenges and how they might be solved (while producing some open source software along the way) maybe isn’t a bad idea at all. Definitely worth watching!

As I noted last week, I see widespread confusion around some of the key terms in the conversation about government 2.0 in general and participation in particular: public participation, crowdsourcing and “the wisdom of the crowds” — unless I am terribly mistaken, the three don’t mean the same thing and hence should not be used interchangeably.

Over on Andrea DiMaio’s blog, I just left a comment that takes a first stab at trying to clear up the confusion:

It seems to me there is quite a bit of confusion in terms of some of the terminologies that are being thrown around. In my view, crowdsourcing and public participation are not the same. In fact, there may be less overlap between the two than some people think.

Here’s where I see the main differences:

For example, crowdsourcing usually involves clearly defined goals and measurable outcomes (e.g. improving an algorithm, doing pharmaceutical research, vetting satellite imagery etc.). In cases where there aren’t any objective success criteria (e.g. some design contests), at least there is a committee that has full authority to rate and rank contributions and pick a winner. It is irrelevant who participates in a crowdsourcing initiative, and there is no concept of participants having to buy into or agree with the outcome or winning proposal.

Public participation, on the other hand, is about engaging citizens in decision making that often involves making difficult trade-offs based on conflicting values systems (very subjective). It is extremely important that all stakeholders have a seat at the table and feel they have ownership of the process. Most of the time the process will be guided by consensus seeking of some kind and achieving a certain level of agreement across all participants is required in the end in order for things to be able to move forward politically.

Can and should crowdsourcing as we know it be applied to help with certain pieces of policy making? Absolutely. But will it be sufficient to qualify as and achieve the goals of public participation? Most certainly not.

I think this is an important piece of the discussion. Let me know in the comments how you make sense of the two concepts, how the differ or complement each other. Thanks!

David Wilcox reports on a talk Clay Shirky gave at the London School of Economics this past week about collective action in a political context and some of the discussions that have since ensued: Clay Shirky: online crowds aren’t always wise

Clay Shirky, leading commentator on internet technologies and author of Here Comes Everybody, last night backed away from his earlier enthusiasm for the online wisdom of crowds in democratic decision-making. He suggested that Government use of social media should focus more on “small groups of smart people arguing with each other”, than national-scale engagement online.

We’ll, that’s my interpretation. You can listen for yourself … and find more on Twitter from last night and today’s at ICA.

A few years back Clay said that the ability of groups to organise online and challenge conventional engagement was “the glory of this medium”. He now believes we need more checks and balances.

Shirky’s examples include the spread of information following the 2008 Sichuan earthquake in China and the viral success of the will.i.am song “Yes We Can”). Starting at around 30:30 into the video, Shirky takes a look at Change.gov as an example of “wisdom of crowds instantiated in a political context”. He sees the fact that the call for legalization of medical marijuana ended up being the top-rated item in the Citizen’s Briefing Book as a sign that

… this is another place where certain kinds of special interests can make their feelings known. But it is not the same as saying anything that’s thrown to the top of Change.gov therefore has or should have priority in the president’s queue.

And so the problem we’ve got now isn’t a problem of capability, it’s a problem of legitimacy. Under what circumstances would you take advice from people primarily coordinated on the internet and headed for political action and under what circumstances would you ignore that advice. [...]

But unless there’s a principle by which you can say that all you’re really doing is saying it’s nice that you have this outlet (?) but we’re not going to take it seriously. But then if you go the other extreme and say you have write privileges to the president’s calendar, you can’t do that either. And the only way when we’re in situations where neither extreme solution works is to set up a set of checks and balances and that’s where I think the conversation is going.

A few thoughts as per my earlier comments here and here:

Not every crowd is wise. That realization shouldn’t really come as a surprise to anyone, especially when you consider that the groups that formed during the various e-participation efforts on Change.gov don’t even fit Surowiecki’s list of conditions:

1. Diversity of opinion: Each person should have private information even if it’s just an eccentric interpretation of the known facts.
2. Independence: People’s opinions aren’t determined by the opinions of those around them.
3. Decentralization: People are able to specialize and draw on local knowledge.
4. Aggregation: Some mechanism exists for turning private judgments into a collective decision.

By that standard, the groups that formed during the various e-participation efforts on Change.gov clearly weren’t wise. For example, in violation of the second rule above, not only were participants’ choices (aggregate number of votes for, say, an idea) exposed while voting was still in process but ideas were listed in order of popularity. This led to herding, favoring those relatively few ideas that managed to gain an early lead.

Moreover, it is debatable whether the concept of crowd wisdom applies to value judgments in just the same way it seems to apply to factual issues (I don’t remember any examples in Surowiecki’s book that would support the former). Since policy discussions are often comprised of both (they certainly were on Change.gov), I wonder to what extent the theories of crowd wisdom really apply here.

It’s also debatable whether what we saw on Change.gov really amounts to decision making. I’d argue it was some low-level form of general input gathering with ranked preferences for questions and ideas: fairly broad, with little structure and little to no process (and still flawed in many ways). On the actual issues no choices were made.

As far as I know, there was never any commitment on part of the transition team to let the Digg-style voting algorithms employed on Change.gov determine what Obama’s priorities would or should be. Instead, the Citizen’s Briefing Book was intended solely as a compilation of “facts and recommendations to be considered while crafting and enacting policies.” Nothing binding in there, just another channel for public input.

Applying IAP2′s Spectrum of Public Participation, the last two points illustrate that the e-participation efforts on Change.gov can be categorized as inform/consult types, not the more advanced involve/collaborate/empower types, in which case they may have had all the legitimacy they needed.

I generally agree with Shirky  that there are many issues facing efforts such as Change.gov with regard to legitimacy and process, just that a more differentiated analysis is needed to figure out where exactly the challenges lie and what conclusions we should draw. For government to abandon work on large-scale efforts and instead focus on small-group engagement alone seems premature advice at this point.