As promised, I’ve been tracking the numbers from the ongoing Open Government discussions on a daily basis (follow the thread on GovLoop for details). We’re now almost three weeks into this initiative. Here’s a chart that shows how the numbers have developed since February 9 (a few days after launch):
Growth is pretty linear, nothing too surprising happening (yet). At the very least, it seems that some agencies are slowly making progress with their outreach efforts.
The total numbers include active and inactive agencies alike. For comparison, here’s the same chart for NASA, the agency with the most overall activity (by our measurements):
Ideas: 122
Comments: 428
Votes: 2,453
Users: 340
It’s a bit hard to see but the last two days have seen a significant increase in registered users and votes.
There are more than three weeks left. Plenty of time to double or triple these figures.
Via the Public Decisions blog (co-organizer of the Including the Excluded online conference we’ll be attending), I just found out about a new research effort by Involve, a UK-based not-for-profit offering public participation consulting and services: Quantifying the Value of Engagement: A call for case studies
Dear Colleague,
Involve are embarking on an ambitious project with Consumer Focus England, we are seeking to develop an equation for identifying costs and benefits associated with public engagement structures and processes. The proposed equation will be a practical tool, aimed at public sector workers who wish to make the case for effective engagement by exploring and quantifying the costs and benefits. It will also help authorities, civil society and the public to identify when public engagement is an effective use of public money. We need to work with public sector organisations to test our equation, in order to ensure it is useful and that it is able to be applied in a variety of circumstances.
[...]
Specifically we are looking for case studies which:
Can demonstrate some tangible improvements in service outcomes as a result of the engagement, for example reduced crime levels, improved health outcomes, lowered bureaucratic requirements, or more targeted spending.
Have some understanding of the costs and benefits of the exercise
Includes some element of engaging with members of the public
Occurred within the last twelve months
We are also interested in looking at examples which engaged people in issues around cuts in expenditure.
I look forward to their findings. In the spirit of moving this entire field forward, please make sure to make your cold, hard ROI data available.
As part of my ongoing efforts to monitor the various feedback channels that have been launched as part of the Open Government Directive, I’ll be tracking some basic metrics. I thought I’d share the raw data here on a regular basis so people can chime in and add any other insights they’ve come across.
23 agencies are using IdeaScale, which exposes the following in real-time:
Number of ideas
Number of comments
Number of votes
Number of registered users
So that’s nice. I’ll check in on the other two forums occasionally as well (here, here).
Aside from cold, hard numbers, I will be looking for examples that illustrate how various agencies are doing in terms of the other nine items on my list. Here it is again in short form:
Expectation management
Community ground rules
Level of convener involvement/participation
Quality of moderation
Quantity of participation over time
Outreach and diversity of participants
Conclusion and impact
Tech support
Project communications
Mood
I’m using this RSS feed for easy scanning but don’t expect to catch everything. Please share your impressions in the comments.
I hope this will attract enough metrics junkies so that we’ll end up with a fairly complete picture in the end. It would certainly be great to see not only a breadth of data points but also to have a discussion about what should be measured and how. After all, the participation piece of the upcoming dashboard still hasn’t been designed in detail yet. Maybe this little exercise will create a few valuable insights.
Now that a whole lot of agency.gov/open websites are live and many agencies have indeed set up a ”mechanism for the public to [...] [p]rovide input on the agency’s Open Government Plan” it’s time to figure out what to watch out for over the coming weeks and months in order to evaluate the success of these initiatives.
As I noted back in January, my hope is that these new projects will address and improve upon three key issues that we saw during last year’s Open Government Dialogue (namely, lack of convener involvement, insufficient moderation, herding).
All in all, I’ll keep an eye on the following (in no particular order):
Expectation management: Is the agency clear about the scope of their participation initiative and their promise to the public? Do participants know what impact they can reasonably expect and when?
Community ground rules: Every agency should have these “rules of engagement” in place and be ready to enforce them if needed. Bonus points for friendly, easy-to-understand language!
Level of convener involvement/participation: Does the agency become actively engaged in the discussions?
Quality of moderation: Will the agency manage to keep discussions on topic and moderate distractions in a fair but timely manner?
Quantity of participation over time: How many participants will sign up? How much content will they produce? (luckily, IdeaScale exposes a few basic metrics in real time, such as number of ideas, comments, votes and registered users)
Outreach and diversity of participants: Does the agency manage to attract a broad range of participants from various backgrounds? Or do usual suspects dominate the discussions?
Conclusion and impact:This one will be especially interesting as there doesn’t seem to be an end date defined for any of these initiatives. In case of ongoing participation programs, does the agency at least share interim results? Looks like this effort is scheduled to go through March 19, 2010.
Tech support: Does the agency address technical support questions and resolve any issues in a timely manner?
Project communications: Does the agency offer ways for participants to stay in the loop (or get up to speed quickly) with regard to current state of the discussion, frequently asked questions, highlights, interim results, next steps etc.?
Mood: Overall, how happy is everyone with the process? What’s the energy level? Are things productive? Etc.
What else should be on the radar? Sound off in the comments.
Peak Democracy is a “non-partisan company whose mission is to broaden civic engagement and build public trust in government.” Their Open City Hall™ service is a monitored online public comment process that promises “order and decorum of a government meeting” (see entry on ParticipateDB).
The City of Palo Alto uses the service to host discussions on various topics. Some comments are posted as “semi-anonymous”. Clicking the link opens a pop-up window with more information:
The pop-up reads:
Q: What are ’semi-anonymous’ statements and why do you allow them?
A: Statements whose authors are identified privately to Peak Democracy, but not publicly on the website are called ’semi-anonymous’. Per our privacy agreement, Peak Democracy does not identify the author of a semi-anonymous statement to anyone – not even to the government agency hosting the forum – unless we are required by law to do so.
We allow semi-anonymous statements because there are people with great ideas who will only share them anonymously. Some people cannot sign their statements because doing so will interfere with their job, business or other personal relationships.
At the same time, we have developed technology and procedures which safeguard against systematic abuse of this privacy feature.
As I mentioned before, allowing participants to “choose if and to what extent they want to share their real identity with the other participants or the general public” is a valuable feature for the exact reason given above.
Caroline Lee, Assistant Professor of Sociology at Lafayette College in Easton, PA, and Francesca Polletta, Professor of Sociology in the School of Social Sciences at UC, Irvine, have just released the results of a survey they did in September/October 2009 and which garnered responses from more than 400 practitioners: The 2009 Dialogue and Deliberation Practitioners Survey: What is the State of the Field?
We conducted the survey as part of two separate research agendas, both of which focus on multi-method analysis of issues in the dialogue and deliberation field. Caroline Lee has completed four years of fieldwork at dialogue and deliberation conferences, trainings, and courses– and is interested in testing the extent to which the perspectives and themes she is surfacing inductively are shared across a broad group of practitioners. Francesca Polletta is interested in building on earlier findings on gender in a study of online deliberation. We plan on comparing our survey results to the insights emerging from other qualitative and quantitative methods. This triangulation of methodological strategies seeks to strengthen the validity of research findings by accounting for the limitations inherent in every research method.
A few tidbits I found interesting:
According to “Q21: Methods Used Most Often in Current Practice”, 25% or respondents are reporting frequent use of “onine/digital media”. That’s pretty cool!
According to “Q23: Term that best describes the people and organizations currently leading D&D efforts”, 57% of respondents prefer “Community of Practice”
According to “Q23: Most important challenge facing the D&D community, based on challenges defined by NCDD conference attendees”, 20% of respondents mention “Demonstrating to powerholders that D&D works”
The full results are available on the website and are available for download (PDF).
Recent Comments