I’ve been writing and rewriting a more formal response to the paper, but I’d like to share my thoughts first, especially as “The Invisible Hand” gets more coverage (with an Ars Technica article today).

First, I’d like to say that I’m delighted to see this topic addressed in an academic setting, and I also see nothing but potential for expansion from third party government information providers.

Probably the reason that I find Government Information and the Invisible Hand as provocative as I do is that the paper seems to imply that empowering third party sites will involve reducing the federal IT footprint.  Specifically, when the essay lists its strategy as to “reduce the federal role in presenting important government information to citizens.”

My issue with this suggestion is that government information providing sites are also service providers, and that the services they provide are often a demonstrable public good, or justified monopolies, and the regulations that govern these services’ provision are reflective of their societal role, rather than the information’s position in an information ecosystem.

Certainly, data based services have only to gain from better data management practices, more open policies, and the innovations of the digitally public marketplace of ideas and creative problem solving.  The image, however, of government service providers as defenders of outdated inefficient turf may have some truth to it, and government IT contractors are likely just as entrenched and as dependent on public money as defense contractors.

My note of caution comes from what I see as somewhat of a conflation of government service provision and government data availabililty.  The essay suggests that “elaborate” government Web sites stand in opposition to the sort of well-considered data policies that lead to creative third party sites, like govtrack.us.  This strikes me as a somewhat dangerous simplification.

Framing the choice as one between overwrought government sites fraught with bureacratic complexity on one hand, and open efficient creative data sharing on the other, overlooks the importance of the restrictions and reliability that we justifiably demand of our government services.

The essay suggests that authentication is a valid concern, and that public mechanisms may develop to take this into account, using electronic signatures or trusted intermediaries to filter out trustable information.  This may be a sufficient solution, but the myriad of regulations promulgated on government web sites each exist for a reason (although they may be imperfect).  As creative data sharing moves into a broader public domain, the accountability mechanisms that govern those services need to be translated into the same public realm.

The handling of private information, the archiving of historical documents, and many many other concerns that have been well established within a government context, and a solution that to public administration problems needs to acknowledge that third parties don’t operate under the same mechanisms of public accountability that the government does (or at least should), and never will.

All that said, I agree with the tone and focus of the Invisible Hand. I’d just like to caution against conflating data and public service provision, and to suggest also that public accountability mechanisms are indispensable.

In fact, I think those same mechanisms should be used to foster exactly the type of online creativity that the article describes, and I’m looking forward to seeing more reviews of just how the specific policy suggestions in the paper might work in government — specifically the government employee/public citizen data parity requirement.

Something pretty exciting is happening over at honda.house.gov; (Congressman Mike Honda’s Web site). They have posted their new education related legislation on the site, along with justifications, background, endorsements, and even the ability to leave public comments through the blog area of the site.

This approach to introducing and building support for legislation recognizes the importance of real public input and evaluation, and gives the public the tools they need to understand what legislation is really about: an explanation, a place to link to, a summary, even a list of institutional support.

We’re excited to see more legislative initiatives recognize the potential of public engagement online, and also to see an OpenCongress.org widget being used to help track the legislative details.

There’s a tipping point somewhere ahead, where public expectations lead to advocacy and legislating functioning through the merits of arguments made in a public setting. We’re certainly not there yet, but small steps like publicmarkup.org, legislation 2.0, and now Rep. Honda’s online work are all steps in that direction.

The guys over at Princeton’s new Center for Information Technology Policy wrote a really great paper for the Yale Journal of Law & Technology on the role data should have, compared to websites, in government. It articulates a point that I think many of us subconsciously have had in mind:

“The new administration should specify that the federal government’s primary objective as an online publisher is to provide data that is easy for others to reuse, rather than to help citizens use the data in one particular way or another.”

And they suggest an interesting way to push that forward:

“The policy route to realizing this principle is to require that federal government websites retrieve the underlying data using the same infrastructure that they have made available to the public. Such a rule incentivizes government bodies to keep this infrastructure in good working order, and ensures that private parties will have no less an opportunity to use public data than the government itself does. The rule prevents the situation, sadly typical of government websites today, in which governmental interest in presenting data in a particular fashion distracts from, and thereby impedes, the provision of data to users for their own purposes.”

I think this is a worthwhile addition to the opengovdata and publicmarkup.org policy documents — if not as a direct recommendation (because I think it may be too much to ask for in a grand form) then noted as a long-term goal or (in terms of the second paragraph I quoted) as a benchmark, a concrete way to tell whether data is open.

The full citation is: Robinson, David, Yu, Harlan, Zeller, William P and Felten, Edward W, “Government Data and the Invisible Hand” (2008). Yale Journal of Law & Technology, Vol. 11, 2008

The current Harper’s first selection features stunningly direct testimony about data. This particular Senate committee receives scalding criticism over the way the GDP is used by the government and the media to measure societal well-being, for which purpose he argues the GDP is wholly inadequate, rather than to measure economic activity, which it actually measures.

As more public data sources become accessibly public, and therefore become integrated into the way we perceive our lives and our country, we’ll need to learn some new lessons about what specific data sets mean, and how to draw well-reasoned conclusions. Easy communication creates new questions about disclosure policy, data provenance, and shared responsibility for societal information. Among those questions should certainly be “What does this actually mean?” or, “What judgments can we actually make on this basis?”.

The testimony is damning.

I got three videos through email that make a strong stong point about the international bottom up movement of online information activism that is occuring right now. They’re embedded and linked below, so be sure to check them out.

The videos give a brief history of the Legal Information Institute, which has come to be part of the name of a great variety of organizatins around the world that have one set of goals in common. They’re working for public access to legal information.

All around the world, without centralized planning, institutes have sprung up in response to a pressing need: non-lawyers have a real use for legal information, but can’t get it. In countries across several continents, new initiatives online are successfully giving the general public information that they wouldn’t have been able to search before, information that used to be controlled exclusively by the legal information publishing businesses. As businesses, they have a mandate to make profitable decisions, and not necessarily to serve the greater needs of a society. As a result, the public gets locked out of the very laws that control their lives, unable to understand and analyze the legislation or case history that forms the legal structures under which their actions are evaluated by the government.

Not anymore, though. A new transnational initiative has sprung up around the world, responding to the needs of the public information consumer, who may occasionally need real, substantive legal information. While this might sound like some sort of abstract concept, access to information, and especially legal information, are a fundamental source of our ability to be agents as humans. Our framework under which we function as humans involves our day to day knowledge of physics, social interactions, and the like, and the knowledge is necessary for us to move around in a physical world, have friends and business relationships, etc. The traditional world of legal information, however, has failed on even this basic level to provide the public information necessary to allow the public to develop to their full potential as substantively relevant agents in the legal world.

The Legal Information Institutes springing up in the US, Canada, and throughout the rest of the world, are filling the void of public legal information, and also discovering that despite a lack of substantive information, there is a real need for legal substance. Some sites have enormous traffic stats, seeing millions of hits a week. Some see much less. The sites’ developers claim, however that site traffic is an almost irrelevant way of measuring public legal information’s impact. Something fundamental changes when legal information is offered online.

People become agents in a legal or legislative or judicial realm where they before would have only been relevant through hired services. They can see the reach of the established law in their countries stretching into their lives, and evaluate it on their own, looking up history or international comparisons rather than relying on talk show hysteria to guide them.

And they’re only just starting their work.

When the legal information world has collected a body of law and made it public, online, in a truly comprehensive fashion, then a real fundamental shift will have occurred, where the normative binding structures with which we weave our societies together are posted for all to see, without a fee, when legislative debates link to laws passed, and on to agency rules promulgated and enforcement action, then the most basic needs for civic information will be filled, and people, as citizens online, will have the very first thing they need to interact effectively with their government and the rest of their society.

This legal information revolution, as seen in the popular growth of the legal information institute, has a parallel in the recent struggle to release and publish legislative information, legal information’s more participatory, electorally relevant older cousin. I’m proud to be working at the Sunlight Foundation on the problems of public access to legislative information, and to have filled in another piece in the larger context in which we operate, whereby people operating largely independently recognize opportunities to improve the world and are empowered by digital technology to do so.

Another point from a panel I attended this morning…

Apparently, justifying IT and infrastructure spending is difficult, especially given that budget requests come up through administrative structures, themselves anathema to inter-agency coordination.  This is probably why e-government spending in the executive branch is inefficiently distributed among agencies and generally resented.

If those people selecting spending priorities and making spending requests make justifications on the basis of technology that is always being developed, and should be coordinated among departments, agencies, and even branches of government, then doesn’t it make sense that that spending case could only be made by a larger collaborative effort?  If meaningful communication is usually grounded in practical circumstances, then what practical circumstances (either natural or imposed) exist to force administrators to coordinate their technology spending and development priorities?  If administrators aren’t even efficiently communicating with each other, we shouldn’t expect policy experts, academic researchers, industry voices, or especially the public to have a chance to offer meaningful input.

What are the options to staff, organizers, or appropriators who want to fill this coordinative hole?  Some extra-agency coordinative bodies exist already, like the Council of CIOs, or NASCIO, and these functions are certainly being addressed to some degree by the GSA, OMB, and the good example of some agency information-policy setters.

The coordination, however, isn’t nearly enough.

The public’s ability to affect public information administration may be even more limited than their ability to affect public policy creation (Congress), since there’s at least an expectation of representation from elected officials.  That doesn’t mean there’s not a similar public stake in administrative and executive information, however.  The same coalitions that exist around congressional information presumably exist for the executive, where strong network effects should magnify the benefits of effective information distribution for both public and governmental agents.

Some models for coordinative and advisory committees are outlined in these CRS reports (rls 22725 and 33313) on advisory committees, although this description doesn’t nearly take into account the potential offered by the broader public in playing a technological advisory role.

The degree to which conferences, universities, non-profits, blogs, the media, and other citizen groups will fill this coordinative gap is uncertain, but if the government doesn’t coordinate better within itself, pressure and expectations will only grow from the outside, perhaps making the ultimate adjustments less comfortable, and the lag between the private and public sectors only more glaring.

Perhaps the most basic way to think about institutions is that they are concentrations of certain kinds of expertise, within stable incentive structures.  Democratizing communications through digital technology normalizes both incentives and membership, by permitting participation on the basis of any sort of incentive, by anyone with any kind of expertise.  As the Cluetrain Manifesto says, “hyperlinks subvert hierarchy”.

Merely clearing a space for participation isn’t necessarily enough to produce more constructive collaboration however, since people communicate with different specialized vocabularies and with different goals, and since online communities often suffer from the ’spiral of silence’, whereby under-represented minority views express themselves less.

This leads me to think that deliberative processes designed in online spaces should take into account the excesses and deficiencies that existing institutions suffer from, and alleviate them by introducing new areas of expertise, and new incentive structures.

I wish the Open Access movement were further along, and academic publications were posted in public online, because that would make it easier to share some of the things I’ve learned in the last two days at the Digital Government Society Conference.  Since I can’t link to most of the research I’m learning about, I’ll have to just offer my basic reactions:

I approached this gathering initially from the perspective that there is an unfortunate disconnect between public policy and academic research, and my suspicions have been confirmed.  The research and technological development I’ve seen so far have exceeded my expectations, despite their overall absence from the congressional policy sphere.  Here are some of my observations so far:  (many more available via twitter)

• Natural Language Processing appears poised to soon become far more relevant to public administration and policy creation.  Political problems from e-government services providing to portals creation to text annotation to policy deliberation are all seeing significant attention from developers and theorists working from a semantic processing perspective.  By extracting entities from text and then applying various processing techniques, developers hope to highlight new connections between data sets, suggest solutions to problems like emergency response systems and disaster management, identify dispositions among participants in deliberative contexts (labeling arguments as vitriolic, clarificatory, diplomatic, etc), allowing similar concepts to be linked automatically (by detecting all legal references, all members of Congress, or all cities), or even detecting the position or ideology of a speaker.  While these technologies haven’t gotten much of a presence in our daily lives so far, the sheer amount of convergent semantic development suggests to me that there will soon exist an emergent political semantic web.  Technologies like “active learning” and socially constructed ontologies promise to only accelerate these developments.
• Social research into IT coordination and policy makers’ motivations reminds me of social media commentary from blogs: sometimes revelatory and entirely valuable, sometimes speculative and of questionable value.  Clearly many of the discussions about staff and administrators’ motivations could use a greater pool of examples to draw from, just as the administrators could probably benefit from more abstract reflection (what are the implications of this exclusive contract I’m about to sign?)  One set of researchers presented a grid explaining the types of options presented to governments when choosing to release public data.  While some options were missing (like “sell to the public”, like some GPO products), the examination gave concrete explanations of the decisions and evaluations that public administrators deal with when facing public access decisions.
• The Center for Technology in Government from the University at Albany’s Center for Technology in Government seems full of things I/we should read.  They’re heavily involved in researching and developing inter and intra-governmental collaboration, working with both e-government implementation and trans-national research projects.  I’m looking forward to reading more essays like this, or this.  One of their researchers gave a great presentation about the ways governments define “borders”, which has implications for immigration, social policy, and economic policies like subsidies or tariffs.
• Apparently some areas are using the detectable density of cell phones within an area to detect traffic patterns.  This is neat.
• Anyone working on web traffic and social visualizations should check out SIOC, or the Semantically Interlinked Online Communities project, providing an RDF framework for visualizing blog and website relationships (as far as my basic understanding reaches, that is–it probably does more than that description gives it justice).  This is one of those things that I come across and say “Josh Tauberer should check this out”.
• One commenter drew a parallel between two panels that everyone enjoyed:  Privacy advocates suggest (apparently) that “it’s not the technology, it’s the policy”.  This was apparently bolstered by the example of some new Canadian food handling practices, where cows and eggs and other food sources are labelled individually with codes or RFID tags, rendering each food item and distribution and production site very trackable.  I’m not sure what the conclusion is from the two examples, but the privacy experts seemed quite amused by the cattle-to-human comparison.
• I saw a project that tracked the three dimensional shape of individual tree’s canopies, to help local governments decide which to cut down and which to leave standing.  The charts showing 3d rendered versions of more or less desirable trees seemed emblematic of the way digital technology is enabling new kinds of evaluations to be made on governmental decisions.  They’re also quite pleasing aesthetically.  Here are some screen-caps.

That’s all for today, I should have more tomorrow, or check out the twitter stream for more live commentary.

Here are my cleaned-up notes from a workshop on electronic rulemaking at the dg.o conference happening now. (still disjointed, but no doubt more interesting posted here than alone on my laptop)

I’m interested in the event because of the relevance of structuring deliberative processes online, developing listening tools to make governmental staff jobs easier, and because I’d like to see what sort of executive branch IT coordination and development projects are underway already, in both government and academia.

Rulemaking, the promulgation of regulations by federal agencies under congressional authorization, has the binding effect of law. Despite its fundamental legal role, agencies’ handling of the public comments on federal regulations are non-standard to a fault, with some agencies counting handwritten comments as two comments when tallying interest. This non-standardized handling of public comments leads to mistrust of administrators regulatory abilities, which hampers implementing regulations, compliance, and overall public trust.

Most rules have something like ten to fifty comments, some get up to 700,000 comments. There is a very strong parallel between public commentary and interest in rulewriting and congressional advocacy email campaigns. Just as very few rules get the majority of public comments submitted, very few people write more than one comment, and very few congressional bills get the majority of calls and emails.

It seems that federal agencies should have greater abillity to take advantage of outside email and text processing systems, as the attendees of this workshop are designing. See here for an example.

Many arguments for rulemaking’s public role are going unmade: public commentary often leads to changed approved rules, has an effect on congressional oversight staff, and affects what sort of challenges are even admissible in court later. Without critical public comment on pending rules, some legal challenges are less likely to be admissible.

Like congressional staff, agency employees “drink from a firehose” of public input on some regulations, and duplication detection enhanced with advanced natural language processing and structured data tools can save significant time in processing public input.

Comments significantly increased under the Bush administration, although the pattern of most comments coming on a few contentious rules still holds.

OSHA once told national parks rangers to stand inside kiosks to avoid snowmobile pollutants (if true, wow.)

Rubrics for what count as a comment differ from agency to agency.

“Spiral of Silence” = tendency of underrepresented groups to abstain from vocalizing concerns, especially online. Arrow’s impossibility theorem: inherent difficulty in consensus forming through representative voting.

I’d like to go over this in more detail soon, but NARA has published a new digital preservation strategy, described in more detail here. (via NARA’s RSS)