As reported by Federal Computer Week last week, the Congressional Research Service (CRS) recently published a report for Congress that evaluates the first year of the Open Government Directive: The Obama Administration’s Open Government Initiative: Issues for Congress (PDF)

From the summary:

The 112th Congress may have interest in accessing information and documents from the executive branch. This report examines and analyzes the Obama Administration’s initiative to make the executive branch more transparent, participatory, and collaborative. [...]

[...]

The 112th Congress may oversee the Administration’s open government efforts and has the authority to codify any parts of the initiative. This report reviews and discusses the centerpieces of President Obama’s transparency initiatives, the Open Government Initiative and the Open Government Directive. The report analyzes agency response to the OGI and the OGD and examines whether the OGD’s requirements can meet the stated goals of the Administration. The report discusses the three central tenets of the Administration’s OGD—transparency, public participation, and collaboration—and analyzes each one individually to determine whether agencies are meeting these requirements and whether the requirements may improve the effectiveness of the federal government.

Several people have already weighed in (Joe Goldman, Nancy Scola, Andrea di Maio and others).

The report puts a strong focus on objective-driven Open Government and the need for measurable results. And while I generally agree with that (a lot), I found the report’s analysis regarding public participation (page 30-31) somewhat lacking. Here’s why:

1. Fails to mention the need for good process design

In the analysis section, the report mentions a number of high-level concerns and challenges, paraphrased here:

• The public may not be well-informed enough.
• There may not be enough citizens who are motivated to engage in public policy deliberations and who are capable of doing so.
• Public comments may not be useful to the federal government.
• Responding to public comments may cause delays in government action.
• The resource requirements (dedicated employees, work hours) needed to respond to these comments are unclear.
• Most public participation may come from special interest groups.
• Web-based public participation may give certain participants unfairly greater access to policy makers.

These are all valid and important points. However, what the report doesn’t mention is that good process design will almost always help address these challenges. For example, good process design takes into account the resources the convener has available to process participant input and will only make commitments that are feasible. Good process design creates ways for participants to learn about the issue (remember, inside every public participation program is a good public information — or, as I would phrase it today — public learning program). Good process design helps bring all voices to the table, not just special interests.

2. Generalizes based on one bad example

The report looks back on the Open Government Dialogue in 2009 and observes how the suggestions were “of varied relevance and utility.” It also quotes one editorial that concluded that the quality of public comments during the Open Government Dialogue “was not consistently encouraging.” While both statements are true, undoubtedly, they do not generally apply to all public participation, online or offline. What is not mentioned is that the Open Government Dialogue was, despite all its good intentions, poorly designed and poorly managed. In all likelihood, better process design and a more hands-on management would have yielded much better results.

* * *

On a final note, while it’s true that “[t]he OGD presumably was created, in part, using suggestions from the public”, there was never a proper follow-up that informed participants about if and how their input had influenced the final document. Again, something which a good public participation design would have taken care of in a timely manner.

Despite these minor shortcomings in a report that I found generally on target, one take-away for me is that the public participation field still has a lot of work to do in better documenting their successes, developing a consistent framework for measuring return on investment, and to much better share this information with any organization in the public, private or non-profit sector that is involved in engaging their constituents in public participation.

It’s not enough to know better, it’s essential that this knowledge becomes readily available to a much broader audience than it is today. As I mentioned on Twitter, this is something I would like to make a focus area of IAP2 USA over the next year or two.

For some time, I’ve been suggesting the creation of a national public participation calendar and project directory as a key component of the Open Government Directive:

• May 2009: Open Government Dialogue: “Create an Open Government Project Directory and Knowledge Base”
• January 2010: Open Government Needs Public Participation Calendars

Based on this post on NextGov, it looks like the U.S. Environmental Protection Agency (EPA) is moving in that direction: EPA Web site paving the way to transparency

As more agencies deploy online score cards that publicly chart the progress of specific missions, the Environmental Protection Agency’s new Web site for tracking rulemaking could be a model, some government transparency activists say.

EPA launched its site, the Rulemaking Gateway, on Thursday to inform the public of the status of high-priority regulatory actions, such as proposals to control greenhouse gas emissions in heavy-duty vehicles and revise vehicle fuel economy labels.

[...]

EPA has committed to releasing rulemaking plans earlier than in the past. As soon as an agency regulatory policy officer determines it is appropriate to start developing a rule, information will be posted on the gateway, officials said. A regulation could appear on the site months or even years before a file is created on the governmentwide rule-tracking site Regulations.gov.

[...]

The gateway is tightly tied to Regulations.gov to increase public participation in the rulemaking process, said Madia, a federal regulatory policy analyst at the group.

[...]

The EPA Web site will show updated proposals monthly, as decisions are made. Time-sensitive information, such as announcements about public meetings, will be refreshed daily.

You can check out EPA’s new Rulemaking Gateway here.

Things are starting to look really interesting…

A few days ago, a new site was launched that helps track activity across the 23 federal agencies that are using IdeaScale for their consultations on Open Government: OpenGov Tracker

This webpage makes use of the IdeaScale API to aggregate data for all agencies which are making use of that platform. USDA and Health and Human Services also have public feedback sites; however they are not using IdeaScale and so unfortunately, are not included here. We encourage you to visit their sites, too!

Totals reflect all ideas including those submitted to the “Site Feedback” category. The “Top Ideas Across Government” section only highlights ideas in the categories of Transparency, Participation, Collaboration, and Innovation– eg., those related to the Open Government Plans themselves.

The site was created by Robbie Schingler and Jessy Cowan-Sharp, who both work at NASA.

This is a most excellent example of how participation tools can expose some of the quantitative information about an ongoing consultation (IdeaScale itself does some of the basics, though does not offer a convenient view across all 23 projects).

It’s especially nice to see how metrics are being made actionable. The section “Needs Some Love” lists those agencies that haven’t received a lot of input yet along with the call to action: “They need some love. Please help!”

This metrics dashboard is a nice addition to my tracking thread on GovLoop where we also try to keep an eye on qualitative data.

There is of course a lot more that the perfect dashboard could include but this is definitely a refreshing step forward.

Please leave a comment below if you know of any tools that offer this kind of eye candy out of the box.

Following up on my list of ten things to monitor, I’ve started a thread on GovLoop: Monitoring and continuous evaluation of OpenGov forums

Here’s my initial post:

As part of my ongoing efforts to monitor the various feedback channels that have been launched as part of the Open Government Directive, I’ll be tracking some basic metrics. I thought I’d share the raw data here on a regular basis so people can chime in and add any other insights they’ve come across.

23 agencies are using IdeaScale, which exposes the following in real-time:

• Number of ideas
• Number of comments
• Number of votes
• Number of registered users

So that’s nice. I’ll check in on the other two forums occasionally as well (here, here).

Aside from cold, hard numbers, I will be looking for examples that illustrate how various agencies are doing in terms of the other nine items on my list. Here it is again in short form:

• Expectation management
• Community ground rules
• Level of convener involvement/participation
• Quality of moderation
• Quantity of participation over time
• Outreach and diversity of participants
• Conclusion and impact
• Tech support
• Project communications
• Mood

I’m using this RSS feed for easy scanning but don’t expect to catch everything. Please share your impressions in the comments.

A set of screenshots will be available on Flickr.

I hope this will attract enough metrics junkies so that we’ll end up with a fairly complete picture in the end. It would certainly be great to see not only a breadth of data points but also to have a discussion about what should be measured and how. After all, the participation piece of the upcoming dashboard still hasn’t been designed in detail yet. Maybe this little exercise will create a few valuable insights.

Now that a whole lot of agency.gov/open websites are live and many agencies have indeed set up a ”mechanism for the public to [...] [p]rovide input on the agency’s Open Government Plan” it’s time to figure out what to watch out for over the coming weeks and months in order to evaluate the success of these initiatives.

As I noted back in January, my hope is that these new projects will address and improve upon three key issues that we saw during last year’s Open Government Dialogue (namely, lack of convener involvement, insufficient moderation, herding).

All in all, I’ll keep an eye on the following (in no particular order):

1. Expectation management: Is the agency clear about the scope of their participation initiative and their promise to the public? Do participants know what impact they can reasonably expect and when?
2. Community ground rules: Every agency should have these “rules of engagement” in place and be ready to enforce them if needed. Bonus points for friendly, easy-to-understand language!
3. Level of convener involvement/participation: Does the agency become actively engaged in the discussions?
4. Quality of moderation: Will the agency manage to keep discussions on topic and moderate distractions in a fair but timely manner?
5. Quantity of participation over time: How many participants will sign up? How much content will they produce? (luckily, IdeaScale exposes a few basic metrics in real time, such as number of ideas, comments, votes and registered users)
6. Outreach and diversity of participants: Does the agency manage to attract a broad range of participants from various backgrounds? Or do usual suspects dominate the discussions?
7. Conclusion and impact: This one will be especially interesting as there doesn’t seem to be an end date defined for any of these initiatives. In case of ongoing participation programs, does the agency at least share interim results? Looks like this effort is scheduled to go through March 19, 2010.
8. Tech support: Does the agency address technical support questions and resolve any issues in a timely manner?
9. Project communications: Does the agency offer ways for participants to stay in the loop (or get up to speed quickly) with regard to current state of the discussion, frequently asked questions, highlights, interim results, next steps etc.?
10. Mood: Overall, how happy is everyone with the process? What’s the energy level? Are things productive? Etc.

What else should be on the radar? Sound off in the comments.

Update 2010/02/15: Added feeds for Department of Health and Human Services (HHS) to the list below, added National Archives to Pipes feed.

The Open Government Directive, issued on December 8, 2009 stated:

1. Publish Government Information Online

[...]

e.  Within 60 days, each agency shall create an Open Government Webpage located at http://www.[agency].gov/open to serve as the gateway for agency activities related to the Open Government Directive and shall maintain and update that webpage in a timely fashion.

f.  Each Open Government Webpage shall incorporate a mechanism for the public to:

i. Give feedback on and assessment of the quality of published information;
ii. Provide input about which information to prioritize for publication; and
iii. Provide input on the agency’s Open Government Plan (see 3.a.).

That day is today!

Here are three sites that track which of the agency.gov/open pages are already live or have feedback options in place:

• http://projects.propublica.org/transparency/
• http://www.sunlightlabs.com/open/
• http://www.usa.gov/webcontent/open/tool_agency_poc.shtml

As was expected, most of these sites are using IdeaScale for input gathering and discussion. Some of you may remember this tool from phase 1 of the Open Government Dialogue (see here, here).

Below is a list of RSS feeds available from the various idea gathering and discussion forums (scroll down for link to the ultimate combo firehose feed):

1. Federal Communications Commission

Over at Sunlight Labs, they have some design suggestions today around how federal agencies should approach their new /open website sections with regard to data.

Here’s the comment I just left (pretty much the same point I made on the OSTP blog a few weeks ago when they were seeking input on an Open Government Dashboard):

In addition to exposing the list of available data sets, agencies should publish — at the very minimum — a calendar of ongoing as well as upcoming consultations or any other participation initiatives they offer.

This would be a first step towards tracking the scope and quality of agencies’ public participation efforts.

Obviously, it would also make it easier for citizens to find the participation opportunities they are most interested in.

A lot of thought leaders in the area of public participation have freely and generously shared their advice over the past year, and agencies should draw on this knowledge as they continue to become more participatory.  The key success factors to meaningful, effective participation aren’t a secret, and over time I’m sure we’ll see smart solutions for consistently tracking, measuring and evaluating public participation efforts at all levels of government.

However, the very first step towards this goal is to simply give people (citizens, researchers and other participation evangelists) an easy way to find out about upcoming participation opportunities.

And who knows, making this information available may pave the way for other, more fancy ideas in this area.

As was already mentioned at last week’s Open Government Directive Workshop event in DC (see video, starts at around 1:14:20), the U.S. General Services Administration (GSA) has announced it will offer IdeaScale to federal agencies free of charge in order to help them comply with the tight deadlines presented in the Open Government Directive. From the press release: Further Opens Government, Launches Online Public Dialog Tool for Agencies

WASHINGTON – To help federal agencies fulfill President Obama’s Open Government Directive, the U.S. General Services Administration has introduced a public dialog tool to federal agencies at no cost. GSA hosted more than 70 government officials last week from more than 20 agencies to introduce the tool, answer questions, and gather feedback.

“To help agencies answer President Obama’s call for a more citizen-centered, open government, GSA is making sure those agencies have the necessary tools to meaningfully engage the public,” said David McClure, Associate Administrator for GSA’s Office of Citizen Services and Communications. “By leveraging a single solution governmentwide, GSA can simplify the public engagement process for both agencies and the citizen, helping to build and offer uniformity and consistency in how the public engages with their government and move toward making the process as efficient as possible.”

On Dec. 8, 2009, President Obama issued the Open Government Directive requiring federal agencies to launch an open government Web page that incorporates a public feedback mechanism by Feb. 6, 2010. To help agencies meet this benchmark, GSA is providing this tool at no cost to agencies, along with technical, management, and policy support. Agency officials will be able to devote their attention to running, moderating, and analyzing public input.

GSA’s public dialog tool will allow agencies to provide citizens a forum to share ideas, give feedback, and engage in Web-based discussions with their government.

GSA’s Office of Citizen Services, which fosters public engagement by using innovative technologies to connect citizens to their government, will continue to develop additional tools and practices to help agencies provide more opportunities for the public to engage with their government.

CNET News has more details: Feds get new tool for online public feedback

The new tool, which was built by a company called Ideascale, is very cheap to work with and implement, said Bev Godwin, the GSA’s director of new media and citizen engagement. All told, she added, it is expected to cost the GSA just $3,000 and will be free to each agency that wishes to use it.

[...]

Although the tool will display real-time results of the most popular publicly-submitted ideas on each agency’s Web site, there are no guarantees that those ideas will ultimately be implemented. But the point, said Dave McClure, who heads up the GSA’s Office of Citizen Services, is that the tool empowers the spirit of the Open Government Directive, by giving the public an easy way to submit ideas and become involved in the decision-making process.

[...]

All told, McClure said, 21 out of 24 “major” federal departments have signed on to use the tool, which is not mandatory under the Open Government Directive’s guidelines.

As you may remember, IdeaScale was used during phase 1 of the Open Government Dialogue back in May of last year. Three issues were widely noted then:

• Little participation by the convener/host
• Lack of moderation
• Information cascades/herding: like most other tools in this space that allow idea generation and up-or-down voting to happen in parallel (while also exposing the leader board, often as the default view), IdeaScale tends to favor those ideas that gain an early advantage (see my detailed tracking notes on Flickr)

It will be very interesting to see if and how these issues will be addressed during upcoming projects.

To see how IdeaScale has already been used in practice, check out the IdeaScale entry on ParticipateDB. A handful of e-participation projects using this tool have already been captured. Looks like quite a few more will have to be added shortly.

In a post on the White House blog today, Aneesh Chopra, Federal Chief Technology Officer and the Associate Director of the Office of Science and Technology Policy, follows up on some of the questions that went unanswered during the FAQ on Tuesday: Answering Your Questions on the Open Government Directive

Russ Gaskin of Washington, DC commented: [W]ould like an example of what citizen participation might look like under this directive.

Russ, I expect citizen participation initiatives to build on the outburst of creativity and experimentation we’ve seen in this space in the first 10 months of this Administration.

For example, Open for Questions gave Americans across the nation a direct line to the Administration to ask exactly what they wanted to know about the Administration’s efforts to get the economy back on track. Openinternet.gov enriched the official record on net neutrality with more than 22,000 comments.   Across the country and online, Secretary of Education Arne Duncan has been seeking the best ideas for the next generation of school reform through his Listening and Learning Tour.  A Health IT Online Forum is currently drawing on the expertise of stakeholders on the front lines of healthcare delivery to uncover new strategies to accelerate the adoption of Health IT.  And, just yesterday, the White House Office of Science and Technology Policy launched the Public Access Policy Forum to better understand how the Executive Branch might best enhance public access to peer reviewed papers arising from all federal science and technology agencies.

Of course, I would be remiss not to mention the unprecedented three-phase public consultation process  (brainstorming, discussion, drafting) that shaped the Open Government Directive itself.   You can learn more about the Open Government Initiative public consultation process and other innovations in participatory decision making in the Open Government Progress Report to the American People and in the White House Open Government Innovations Gallery.

At the same time, I hope that all of you will engage in the agency public consultation processes that will shape their Open Government plans. I know that Washington does not have a monopoly on the best ideas and want your ideas for how we can make participation opportunities more meaningful for citizens.

Earlier this year, a number of organizations in the participation field collaborated on several occasions to provide the White House with detailed input on exactly this question — how to make participation meaningful. The following two efforts were probably the most notable:

• The Public Engagement Principles (PEP) Project was launched in mid-February 2009 to create clarity in our field about what we consider to be the fundamental components of quality public engagement, and to support Barack Obama’s January 21st memorandum on transparency and open government.  The following principles were developed collaboratively by members and leaders of NCDD, IAP2 (the International Association of Public Participation), the Co-Intelligence Institute, and many others. — PEP website, PDF download (1.6MB)
• Strengthening our Nations Democracy II. On August 2-4, 2009, “experts and advocates for strengthening our nation’s democracy” came together to create new momentum and plans for bringing together the emerging democracy reform movement behind a common set of priorities. At the second Strengthening Our Nation’s Democracy conference, participants shared what they have been learning from their work across the country, and rolled up their sleeves to create collective recommendations and action steps. — PDF download (1.1MB)

Unlike the eight Open Government Data Principles, a set of guidelines developed two years ago by the Open Government Working Group (see meeting notes) which are reflected in the Open Government Directive almost verbatim, the above mentioned recommendations on participation were not explicitly included in the Directive. It remains to be seen to what degree they’ll be found in the Open Government Plans that are now being crafted at the agency level.

Now that the Open Government Directive is finally out, it’s been interesting to sift through the reactions and read what people think about it.

As several people have noted (here, here), the Open Government Directive’s immediate focus — at least quantitatively — seems to be much more on transparency than on the other two pillars, participation and collaboration.

Not surprisingly, the transparency community is pretty happy (see, for example, GovFresh’s nice list of a dozen statements from some of the more well-known voices).

But what about participation? Here are three comments that seem to illustrate the overall mood pretty well, at least according to what I’ve been hearing.

Peter Levine, Director of CIRCLE, The Center for Information and Research on Civic Learning and Engagement (2009/12/08): the Open Government Directive

Meanwhile, the “collaboration” and “participation” aspects of the president’s original memorandum were extremely promising. Real participation by citizens and real collaboration with nonprofits and communities would change government and enhance civic skills. But those sections of today’s Directive are very short and vague, and the concrete passages disappoint me. For instance:

“The Plan should include descriptions of and links to appropriate websites where the public can engage in existing participatory processes of your agency.”

“The Plan should include proposals to use technology platforms to improve collaboration among people within and outside your agency.”

“The Plan should include innovative methods, such as prizes and competitions, to obtain ideas from and to increase collaboration with those in the private sector, non-profit, and academic communities.”

I predict that organized stakeholders will dominate open online forums and will win most of the prizes and competitions, leaving most Americans with no new ways to participate. But I could certainly be wrong, and I hope I am. I also look forward to future initiatives, because I assume that the original Memorandum remains a promise that can inspire further action.

Brad Rourke, President, The Mannakee Circle Group (2009/12/09): White House Issues Open Government Directive

While the move was applauded by the transparency community (for instance, the policy director of the Sunlight Foundation referred to it as “enormous,” and indeed the list of commitments from agencies when it comes to transparency is already impressive), many in the civic participation field are less sanguine.

“I was underwhelmed,” wrote Fielding Graduate University professor emeritus W. Barnett Pearce in a post to an influential mailing list run by the National Coalition for Dialogue and Deliberation. “[I]t seemed very much like the ‘town hall meeting’ concept – the government shows/tells/lets us look on the website to see what they are doing, and then we can line up for our three minutes/send in our comments to their email inboxes or a listserve.”

There were a few positive comments as well.

Roger Bernier, senior advisor at the Centers for Disease Control and Prevention (2009/12/10 via the NCDD mailing list):

As a federal agency official promoting public engagement, I believe the Directive raises the bar on how we intereact with the public and could be a powerful tool for stimulating more active participatory processes. I say this because the Directive calls for changing management and administrative policies to increase public participation in agency activities and calls for the creation of new and easier methods of public engagement. Also, it recognizes the importance of agency employee engagement to help create an internal culture of participation as well as an external one.

So, a bit of a mixed bag. It seems that even though the participation community (NCDD and others) have been engaging with the White House extensively over the past few months, the real work is only about to begin.

I’d like to see a comprehensive list of all agencies that will be working on this over the coming months and their activities (incl. key people, meeting calendars, calls for public input etc.). As a field, we can’t assume that anyone at the agency level is aware of the recommendations or resources we’ve shared so far. Another coordinated effort will be needed to help put the participation piece on the right track.

Being eagerly awaited by many in the field, the Open Government Directive was released this morning: Open Government Directive (PDF, 84 KB)

Here’s a summary of snippets that deal with participation and engagement:

Participation allows members of the public to contribute ideas and expertise so that their government can make policies with the benefit of information that is widely dispersed in society.

[...]

3. Components of the plan

[...]

b. Participation: To create more informed and effective policies, the Federal Government should promote opportunities for the public to participate throughout the decision-making process.  Your agency’s Open Government Plan should explain in detail how your agency will improve participation, including steps your agency will take to revise its current practices to increase opportunities for public participation in and feedback on the agency’s core mission activities.  The specific details should include proposed changes to internal management and administrative policies to improve participation.

i. The Plan should include descriptions of and links to appropriate websites where the public can engage in existing participatory processes of your agency.

ii. The Plan should include proposals for new feedback mechanisms, including innovative tools and practices that create new and easier methods for public engagement.

[...]

e) Public and Agency Involvement: Your agency’s Open Government Plan should include, but not be limited to, the requirements set forth in this attachment.  Extensive public and employee engagement should take place during the formation of this plan, which should lead to the incorporation of relevant and useful ideas developed in that dialogue.  Public engagement should continue to be part of your agency’s periodic review and modification of its plan.  Your agency should respond to public feedback on a regular basis.

3. Components of the plan

b. Participation:  To create more informed and effective policies, the Federal

Government should promote opportunities for the public to participate

throughout the decision-making process.  Your agency’s Open Government

Plan should explain in detail how your agency will improve participation,

including steps your agency will take to revise its current practices to increase

opportunities for public participation in and feedback on the agency’s core

mission activities.  The specific details should include proposed changes to

internal management and administrative policies to improve participation.

i. The Plan should include descriptions of and links to appropriate

websites where the public can engage in existing participatory

processes of your agency.

ii. The Plan should include proposals for new feedback mechanisms,

including innovative tools and practices that create new and easier

methods for public engagement.