The page updated is here.

The updates were a combination of suggestions from Scott Horvath and Jeremy Fee at the USGS, Kol Peterson from EPA, and me, and really big thanks go to Scott and Kol for reaching out to others for input on Monday and getting the feedback back to Bev Godwin at GSA who runs webcontent.gov who published the changes only a few days later. Scott also notes that additional suggestions could still be considered (his email address is at the bottom of that page).

In making my suggestions, I turned to the Open Government Data Principles and tried to squeeze in as much as I could without overloading the document, and I drew from ideas that came up in the preparation of the Open House Project report. Some of the changes made were:

• It now provides examples of data as being documents, audio/visual recordings, and databases.
• It now says to support “the widest practical range of public uses of
  the data”. It had formerly suggested supporting the “intended” use of
  the website by visitors.
• It notes the benefit of providing data: “New uses of your agency’s
  data may become a valuable public resource that would be out of the
  scope of your own website, such as helping to keep the public informed
  about the work of your agency and supporting civic education and
  participation.”
• There is a new paragraph that I might be misunderstanding but which
  seems to make a suggestion along the lines of the recent “Invisible
  Hand” paper about the agency’s website getting the data the same way the
  public does: “Providing a uniform method to access raw data can also be
  the first step in internal development, accomplishing both goals at
  once. When a uniform method to access data is available, developers and
  web–services can focus on data presentation.”
• It notes that the availability of bulk downloads of data is something
  to consider when building data access.
• It notes some disadvantages of using proprietary formats.
• It recommends that if a proprietary format is needed, a
  non-proprietary format should be used in addition.
• It adds a benchmark to test for success: “One benchmark for
  determining whether data is made sufficiently available is whether the
  public has all of the data needed to replicate any searching, sorting,
  and display functionality provided on the agency’s own website.”
• It notes that consulting the public in the development of data access
  seems to be entailed from OMB policy: “When choosing data formats and
  distribution methods, keep in mind that your agency’s visitors are the
  best judges of their own needs. Agencies must “establish and maintain
  communications with members of the public and with State and local
  governments to ensure your agency creates information dissemination
  products meeting their respective needs” (OMB Policies for Federal
  Public Websites #4A).”

We have a real success story here.

There are several passages, however, which could be taken to add up to as much:

The free flow of information between the government and the public is essential to a democratic society. It is also essential that the government minimize the Federal paperwork burden on the public, minimize the cost of its information activities, and maximize the usefulness of government information.

Because the public disclosure of government information is essential to the operation of a democracy, the management of Federal information resources should protect the public’s right of access to government information.

Systematic attention to the management of government records is an essential component of sound public resources management which ensures public accountability. Together with records preservation, it protects the government’s historical record and guards the legal and financial rights of the government and the public.

The open and efficient exchange of scientific and technical government information, subject to applicable national security controls and the proprietary rights of others, fosters excellence in scientific research and effective use of Federal research and development funds.

Federal Government information resources management policies and activities can affect, and be affected by, the information policies and activities of other nations.

The availability of government information in diverse media, including electronic formats, permits agencies and the public greater flexibility in using the information.

The Chief Information Officers Council and the Information Technology Resources Board will help in the development and operation of interagency and interoperable shared information resources to support the performance of government missions.

(d) In determining whether and how to disseminate information to the public, agencies will: (i) Disseminate information in a manner that achieves the best balance between the goals of maximizing the usefulness of the information and minimizing the cost to the government and the public;

(ii) Disseminate information dissemination products on equitable and timely terms;

(iii) Take advantage of all dissemination channels, Federal and nonfederal, including State and local governments, libraries and private sector entities, in discharging agency information dissemination responsibilities;
(iv) Help the public locate government information maintained by or for the agency.

How must agencies avoid improperly restrictive practices?

Agencies will:

(a) Avoid establishing, or permitting others to establish on their behalf, exclusive, restricted, or other distribution arrangements that interfere with the availability of information dissemination products on a timely and equitable basis;

There’s a lot there, but they never seem to fully address the problems that come along with the use of proprietary data formats. I’m wondering how helpful it would be to have this addressed more formally, and perhaps in a way that applies to both Congress and the executive (since this is an OMB circular).

Also, notably, this circular directs each agency to appoint a CIO, and outlines accountability mechanisms for them, and also mentions the Council of CIOs.

The reason this is interesting to me is that we like to capture reality in data. The Library of Congress and GovTrack both systematize (or in computer jargon “normalize”) the bill-becomes-a-law process. At every point in the game, a bill, in our data formats, is either in-progress, enacted, dead, etc. It must be in one of these states. After all, the constitution outlines exactly what states a bill can be in, so any bill *must* be in one of these states.

But if we’re not sure what state a bill is in, what state do we put it in in our data? There’s also the more important question- What do the lawmakers do if they disagree about what state a bill is in? (Actually, I would prefer to phrase it as “what state they are in”, but that’s another story.) Wikipedia describes (what the editors of the page claim is) a current debacle over H.R. 1585: National Defense Authorization Act FY 2008:

In December of 2007, President George W. Bush pushed the pocket veto into murky waters by claiming that he had pocket vetoed H.R. 1585, the “National Defense Authorization Act for Fiscal Year 2008,” even though the House of Representatives had designated agents to receive presidential messages before adjourning. The bill had been previously passed by veto-proof majorities in both the House and the Senate [JT: and thus a traditional veto would have been futile].

So was the bill (pocket) vetoed or not? Is the bill still in-progress? Assuming it was not pocket vetoed, after 10 legislative days without a traditional veto it becomes law, and us citizens would hate to be on that 11th day without either resolution on the pocket veto matter or a traditional veto, because then we as a country will not know whether this bill has become law. (Another question: How might the Supreme Court assert jurisdiction over this question.)

But back to the data. At one point, some time after Dec. 28, someone in the House responsible for updating the bill status information shown on THOMAS entered a new status line:

Dec 28, 2007: Pocket Vetoed by President.

GovTrack picked up on the change and shows that status currently, much to the confusion of several people emailing me about it. Looking back at THOMAS, it seems like someone realized that that was apparently quite a constitutional (if not political) claim and retracted that update, because it not longer says that.

In many cases citizens complain when the government takes things back, hiding information previously made public. That’s definitely not what I am getting at here. THOMAS is forced to show *something*, and when it doubt… well, what can you do but roll back history until we figure out what the next legislative step actually *was*.

The OTA was recently reinstated, although in a different form, when Congress passed (and President Bush signed) the most recent legislative branch appropriations bill. I work for the Sunlight Foundation, and we helped push for this measure to be adopted. (see http://www.theopenhouseproject.com/2007/06/04/ota-endorsement/ or http://www.theopenhouseproject.com/2007/06/07/ota-renewed-gets-25-million/ )

Instead of reinvoking funding for the still-authorized OTA department, the new approps bill adds funding to the GAO’s budget, letting them venture explicitly into the realm of technological assessment. This is probably a good thing, given GAO’s excellent reputation, and that their work is public, as are their evaluative criteria.

Having a congressional support agency with an explicit mandate to provide research on technological issues will go a long way toward giving congressional decisions on technology the background they need to be sound, and re-funding an OTA, whether as a stand-alone agency or as part of the GAO is the right way to go about it.

A separate issue entirely, however, is whether Congress has adequately organized their administrative appendages to address their own technological coordination issues. Having dedicated researchers available will help lead to better policy, but without clear jurisdiction, proper implementation and foresight get passed over, as agencies and departments struggle to prioritize and act within their budgets. Complex incentives surround the question of whether or not to take on a problem that is outside your department’s statutorily authorized responsibilities. A central non-political body coordinating these responsibilities across Congress would streamline the process of implementing and planning new technological transitions, a very complicated task for such a complex institution.

What I didn’t get into, however, is the main thrust of Lessig’s post, that is, should the US have a CTO position?

I don’t know the answer to that question, but I’m inclined to suggest that centralizing and institutionalizing technological expertise leads to better policy and decisions. The issue I have with this suggestion, however, is one of jurisdiction. How does the complex of fought-over jurisdictions, executive orders, tradition, and expectations come to cough up a new cabinet level position? The history American choices addressing emergent needs by creating new administrative appendages is probably full of illustrative examples, which might suggest the best way to centralize technological expertise in American executive administration.

Is this best filled by an informal advisory role, a cabinet level position, an Office of Management and Budget office, a separate technological affairs office, a temporary task force? Consciously considering the choices beforehand makes it less likely that we choose whatever is most expedient when the issue is raised in earnest.

There is a clear public benefit, however,when search terms like “Colorado census 1990″, “federal childhood immunology standards”, “Pennsylvania superfund sites”, or “Congressional Record 1930 Stock Market” result in the information the searcher is obviously interested in–government information.

The solution to this problem is a non-proprietary standard championed by Google, called the sitemap protocol. Implementing this standard helps automated web-crawlers (the stuff of search engines) find their way around your entire site. Given the immense number of government databases and agencies to reach, getting all government information to show up in web searches will take some time–unless you have a flexible bureaucracy, or an administrative commitment to modern digital government.

Enter Google’s public policy team, and the Senate Homeland Security and Government Affairs Committee:

The Senate Homeland Security and Government Affairs Committee will consider S. 2321, which extends and updates the E-Government Act of 2002. Part of the bill directs the Office of Management and Budget to create guidance and best practices for federal agencies to make their websites more accessible to search engine crawlers, and thus to citizens who rely on search engines to access information provided by their government. It also requires federal agencies to ensure their compliance with that guidance and directs OMB to report annually to Congress on agencies’ progress.

From the bill:

`(i) GUIDELINES- Not later than 1 year after the date of enactment of the E-Government Reauthorization Act of 2007, the Director shall promulgate guidance and best practices to ensure that publicly available online Federal Government information and services are made more accessible to external search capabilities, including commercial and governmental search capabilities. The guidance and best practices shall include guidelines for each agency to test the accessibility of the websites of that agency to external search capabilities.

This measure will go a long way toward making governmental information relevant to people’s lives, by making them accessible in the places we would first expect to find them.

For a little more detail on the foundations of that relationship in terms of oversight, here are two recent CRS reports on Inspectors General, one about the offices’ history and statutory basis, and one about their independence and legal reform.

What is surprising, though, is the degree to which hyperlinks throw a wrench into well established communications standards. Websites in house.gov are required to use a redirect page, putting a barrier up between hosted and outside content. The State Department just put up a new blog, “Dipnote”, and their “legal” page is a great demonstration of how Internet communications stretches institutional quality control beyond its typical comfort levels.

As more official communications move to a hyperlinked parallel interactive world, we should expect to see a proliferation of careful distinctions between “official” releases, and what here seems to be more like “officially sponsored” interaction.

This blog does not represent official U.S. Department of State communications. The Digital Communications Center, Bureau of Public Affairs, manages this blog as a portal for information from the U.S. State Department. External links to other Internet sites should not be construed as an endorsement of the views or privacy policies contained therein.

While the distinctions and warnings may be somewhat clumsy, State department officials should be lauded for stepping into e-government, however hesitantly. I can only imagine how much initiative this project may have taken to get by the institutional concerns of an unfamiliar medium.