Monthly Archive for June, 2009

Intellitics at IAP2 Mini-Symposium “The Future of Public Participation”

Published by Tim on June 25, 2009 in E-participation, Events, Presentations and Research. 0 Comments Tags: , , , , .

On Monday night, the International Association for Public Participation (IAP2) Northern California chapter hosted a Mini-Symposium on “The Future of Public Participation” in San Francisco, CA.

I had been asked to give a brief talk about social media in public participation. Having recently spent a considerable amount of time and effort monitoring and tracking various e-participation initiatives in the US, I decided to share a few of my observations. I picked five example projects that have relied primarily on using off-the-shelf web 2.0 and social media tools:

  1. Rebuild the Party (UserVoice)
  2. Change.gov: “Join the Discussion: Healthcare” (IntenseDebate)
  3. Change.gov: “Open for Questions” (Google Moderator)
  4. Open Government Dialogue, Phase I (IdeaScale)
  5. #MyIdea4CA (Twitter)

It is important to note that a lot of the insights I present are still a work in progress and are often based on incomplete information. Due to the short time I had available at the event I wasn’t able to go into a lot of detail.

Aside from showing the data and interpreting the results from these specific projects, I was equally interested in drafting what might at some point become a general scorecard system that could be applied to e-participation projects of all kinds, shapes and forms. As I mentioned before, there are quite a few common metrics or key performance indicators that are necessary in order to monitor a project’s success. A standard evaluation framework could be incredibly useful for both conveners as well as tool providers. For example, such a framework could inform an Open Government project directory and knowledge base (and of course, a collection of case studies and project reports would in turn also help improve any framework).

Here are the categories I started out with to describe each project:

  • Title
  • Host
  • Objectives
  • Duration (start and end date)
  • Type (e.g. brainstorm, discussion etc.)
  • Tool(s)
  • Key participation metrics (e.g. number of participants, number of ideas, total word count etc.)
  • Challenges
  • Impact

So without much further ado, here are Monday’s slides:

Off-the-Shelf E-Participation? Five Recent Examples

I’m aware that there’s already work being done in this area and I’ll link to those efforts in one of the next posts on this topic.

What I’d like to work on next:

  • Get more specific regarding the public participation goal (as defined by IAP2 Spectrum of Public Participation)
  • Sanity check some of the assumptions or perceived results (e.g. by way of interviews with the conveners or organizers)

As always, feel free to chime in with comments or suggestions.

Open Government Dialogue: 26 Tips for Improving Phase 2

Published by Tim on June 10, 2009 in Events and Research. 9 Comments Tags: , , , , , , .

While working on the previous post, I couldn’t help but think how valuable it would be for participants, researchers and everyone else interested if these high-level participation metrics were readily available in real-time on the site that hosts the discussion, rather than having to dig them up manually and deal with incomplete information, assumptions and more or less wild guesses.

Some of the numbers I usually like to track over the course of an e-participation initiative include the following (these are all pretty straightforward and a number of tools recently used by the transition team and the new administration already expose some of them by default):

  • Number or registered users
  • Number of posts (or ideas, questions etc.)
  • Number of comments
  • Number of votes
  • Number of flags
  • Total word count
  • Participation per participant (e.g. average number of comments, median)
  • Leader boards (where applicable)

Others — both participants and observers — have pointed out other issues with the site that make it harder to use than necessary.

Based on their feedback and some of my own observations, I’ve compiled a list of 26 enhancement ideas that would considerably improve the participation experience during phase 2 of the Open Government Dialogue and increase the overall quantity and quality of participant input.

Some of these refer specifically to the tool used for phase 2 (Wordpress blogging engine for threaded discussions and a plug-in for comment rating) but most should be generally applicable across other tools as well.

I’m sure the list is far from complete. Please leave a comment below if you can think of anything else that could be improved.

Registration

  1. Improve the usability of the sign-up process (including a better fall-back solution for CAPTCHA, which a lot of users apparently can’t seem to figure out).

User profile

  1. Add a prominent link to the user profile self-management page, where participants can edit their name, contact information, password etc.
  2. Add a public user profile page that allows participants to voluntarily reveal more background on their real identity (e.g. by sharing their name, affiliation, a brief bio, photo/avatar, link to their personal blog etc.).
  3. On each participant’s profile page, list some basic participation metrics for that user and link back to all of her comments (this also makes finding one’s own comments a lot easier than, say, having to browse the entire archive).

Commenting

  1. Allow participants to edit or delete their comments for a reasonable time period after posting (e.g. to correct typos or remove duplicate entries).
  2. Consider imposing smart limits on the maximum number of characters allowed per comment to avoid overly lengthy submissions.

Navigation

  1. Add permalinks to comments for easier referencing and sharing across the web (e.g. using email, blogs, Twitter etc.).
  2. Highlight staff contributions more prominently.
  3. Display posts and threads in chronological order.
  4. Provide a more robust tree structure, one that properly associates replies with the comments they refer to (even if one or more comments from a thread have been hidden or removed) and which supports better browsing and sorting of comments and threads (e.g. show most recent posts or most recently active threads).
  5. Highlight recently added comments.
  6. Allow tagging of posts and comments.
  7. Add a tag cloud.
  8. Add a searchable directory of all registered users that supports various filters (e.g. sort by most active users, most recently joined, most highly voted etc.).

Notifications

  1. Add a prominent link to the general comment RSS feed as an alternative way to follow the discussions: http://blog.ostp.gov/comments/feed/
  2. Enable email notifications for new or updated blog posts, comments and replies.

Comment voting

  1. Allow participants to correct (take back or switch) their up or down votes on comments.
  2. In the meantime (and at a minimum), make participants better aware of the fact that votes cannot be changed once submitted.
  3. In addition, improve the usability of the voting (vote up, vote down), flagging and reply buttons: the icons aren’t clear enough and the buttons are fairly small, both factors that can lead to accidental mis-voting).
  4. In addition to net number of votes, expose the total number of positive and negative notes for each comment.
  5. Expose controversiality (the ratio of negative to positive votes).

Moderation

  1. For comments that have to be removed due to a violation of the terms of participation, leave a note that references the type of violation.
  2. In case a comment is removed, notify the original poster of her offense (we don’t know for sure if this is done consistently but judging from user feedback it’s not).

General

  1. Add an FAQ or help page.
  2. Add a statistics page that shows the number of registered users, number of comments, number of votes and number of flags over time.
  3. Add site-wide search (currently comments don’t seem to show up in search).

That’s all I could find over the past week. Please expand on this list as you come across other stumbling blocks.

Open Government Dialogue: Phase 2 Metrics Update

Published by Tim on June 9, 2009 in E-participation, Events and Research. 1 Comment Tags: , , , .

Phase 2 of the Open Government Dialogue has been under way for almost a week now. Following the brainstorm phase, which ran from May 22 through May 28 (see our coverage here, here and here), this discusson phase was launched June 3 and is being hosted by the Office for Science & Technology Policy (OSTP).

The tool used for this phase is Wordpress, a free and open source blogging engine that powers the OSTP blog. Threaded comments come standard with the latest Wordpress 2.7 release. For comment voting, it appears they added the Comments Vote plug-in:

This plug-in enables users of the website to vote comments up or down. This running tally can be displayed on the comment, and administrators have the ability to activate the collapsing of comments with a designated negative rating. The comment will still be available for viewing by clicking on a “show comment” link, which expands to show the collapsed comment’s contents.

Unlike IdeaScale (the tool used in phase 1), there doesn’t seem to be a page that shows key participation metrics (e.g. number of registered users, number of votes cast etc.). The only thing that’s shown is the number of comments on each blog post.

Based on this limited information plus some plain old manual counting, here’s some of the activity we’ve seen on the four blog posts as of this writing:

That’s a total of 394 comments over 6 days (note that this number doesn’t include comments that have been removed).

Comments that are deemed off-topic (or that otherwise violate the terms of participation) are moved to the flagged comments section. Here’s an overview of how many comments have been demoted since the discussion phase started (flagged comments appear under the date they were first posted):

To date, a total of 142 comments have been removed to the flagged comment section.

The total number of votes is not exposed but since most comments seem to get no more than a handful of net votes and very few comments gain ten or more net votes, it’s safe to assume that the total number of votes cast across all participants lies in the hundreds, rather than in the tens of thousands as was the case in the previous phase.

The number of registered users is also not exposed. Again, my guess would be that only several hundred participants have signed up so far (a lot fewer than during phase 1).

The word count for all 394 comments is approximately 48,000 words, or about 120 words per comment (for comparison, this number is very similar to what we saw during the Change.gov discussions).

Open Government Dialogue: Agenda for Phase 2 Discussion about Citizen Participation

Published by Tim on June 8, 2009 in E-participation, Events and Public Participation. 0 Comments Tags: , .

The White House Office of Public Engagement just shared in an email the agenda for the phase 2 discussion about citizen participation, which is scheduled to start Wednesday, June 10 over on the OSTP blog:

As we wrap up the transparency conversation with a final posting about information access and the Freedom of Information Act tomorrow, we want to preview what’s coming this week in the discussion about Citizen Participation and Civic Engagement. Beginning on Wednesday we’ll start conversations on Participation:

  • Creating More Opportunities for Citizen Participation in Government Decision making
  • Increasing Citizen Participation and Civic Education
  • E-Democracy: New Tools and Technologies for Participation
  • Web 2.0 Policy Framework
  • Public Participation in Federal Rulemaking

Good stuff! Really looking forward to this part.

Quality Participation Doesn’t Waste Participants’ Time!

Published by Tim on June 2, 2009 in Community, E-participation and Events. 0 Comments Tags: , , , , .

Over the past few days, the site that’s hosting the Open Government Brainstorm has been overrun with off-topic ideas and spam (most noticeably, there has been an avalanche of requests to “release President Obama’s birth certificate in order to prove his US citizenship”). See this snapshot of the tag cloud I took yesterday:

Open Government Brainstorm: Obama birth certificate spam attack

Steven Clift just posted an idea that suggests a tried-and-true way of how to better deal with this situation:

Move Off-Topic Submissions – Add an Out-of-Scope button to move off-topic posts to their own space

Since a few people seek to dominate any public space hosted by government regardless of the topic, re-establish some equity by allowing participants to flag submissions as “off-topic.” With 10 off-topic votes, instead of deleting such posts, simply move them to a proper tab for all *who wish to see* can view them. An online consultation should make engagement more efficient. The online consultation manager should be able to freeze this feature should a minority of users attempt to game the system by flagging clearly on topic posts as off-topic.

Why Is This Idea Important?

When government spends tax dollars (or lends their support to partners like NAPA) to host structured sessions to gather online input, it is a waste of resources if relatively few individuals seek to hijack the public space for their pet issue which is clearly off-topic from the promised theme or agenda. Such submissions should be channeled rather than deleted (which would be a potential violation of the first amendment in what appears to be a legally public forum).

Here’s the comment I left:

Leaving off-topic ideas and spam unmoderated also forces those participants who come to this site with a sincere intention to contribute to dig through layers and layers of unrelated material. This makes participating more cumbersome and a lot less productive (and less fun). Any convener of e-participation effort of this kind would be well advised to being more mindful of their participants’ precious time. In that sense, moderation and community management become a courtesy.

Aside from off-topic discussions and spam, I’ve also noticed an increased use of foul language, general rudeness, and name calling.

I’d argue that taking a hands-off approach to managing an e-participation effort such as the Open Government Brainstorm (especially in light of these spam attacks) is a violation of a couple of rules laid out in the Core Principles for Participation that were released a few weeks ago (see the  expanded text for reference). For example, the third principle advises to avoid “hostile, disrespectful or stilted conversations,” (which, of course, doesn’t usually happen on its own without any management).

In my view, it is the convener’s responsibility to design and manage a process, online or offline, that maximizes each participant’s chance of having an impact, and there’s a cost involved in not doing so: By tolerating abusive behavior and by allowing the quality of the conversation to degrade, a large portion of the participants’ overall time and effort (which in large-scale efforts such as this one can easily exceed tens if not hundreds of person months) will be spent on low-quality, low-impact activities (e.g. browsing duplicate entries) when the real opportunity lies in enabling participants to share the best they got.