The two paragraphs were added by Congressman Mike Honda of California, one of our champions of the use of technology to further transparency and civic engagement. John Wonderlich of Sunlight Foundation, Rob Pierson in Honda’s office, and I collaborated on this over a long period of time. Honda got involved in 2007 asking the Library to look into this and then in 2008 getting the paragraphs added to the bill markup.

So here they are:

Congressional Technology Coordination.-The House of Representatives needs a strategic and coordinated plan that will prepare for the future technology needs of the institution. A 2006 report commissioned by the Chief Administrative Officer and the Committee on House Administration, entitled Strategic Technology Road Map for the Ten Year Vision of Technology in the House of Representatives, provided a suggested structure for Information Technology evaluation and decision making. The Chief Administrative Officer, the Clerk, and the Sergeant at Arms are asked to prepare a report by June 30, 2009 on their efforts or plans to develop House-wide data-sharing standards; implement standard legislative document formats; address the increasing resource challenges of Member offices; and identify disparate systems throughout the institution that prevent it from taking advantage of economies of scale. [page 2]

and

Public Access to Legislative Data. There is support for enhancing public access to legislative documents, bill status, summary information, and other legislative data through more direct methods such as bulk data downloads and other means of no-charge digital access to legislative databases. The Library of Congress, Congressional Research Service, and Government Printing Office and the appropriate entities of the House of Representatives are directed to prepare a report on the feasibility of providing advanced search capabilities. This report is to be provided to the Committees on Appropriations of the House and Senate within 120 days of the release of Legislative Information System 2.0. [page 11]

According to an article in Wired: “In our web 2.0 world, we can empower the public by providing them with raw data that they can remix and reuse in new and innovative ways,” says Honda, who is vice chairman of the Appropriations Subcommittee on the Legislative Branch. “With these tools, the public can collaborate on projects that can help legislators to create better policies to address the pressing challenges facing our nation.” There’s also a good article at Mother Jones and a nice post by Tim O’Reilly.

The concept of bulk data downloads hasn’t been missed by many parts of the government. The Census Bureau and the Federal Elections Commission, for instance, are fantastic at sharing with the public as much as they can. In the latter case it is electronic versions of campaign contribution filings, which is obviously very important for preventing corruption. But, there are significant gaps in other areas of the government where a little legislating is necessary. Here we’re talking about information on bills in congress going back around two decades, and the information going forward.

The Library of Congress has a database of this information but they don’t share it with the public. Sharing it would mean that creating sites like GovTrack — and the various other sites that use data from GovTrack including OpenCongress and MAPLight.org — would be a little easier, but also a little more accurate. Right now GovTrack goes through a roundabout process to reverse-engineer the same information we are seeking from this database. Basically, we already have the information by scraping it off of thomas.loc.gov — we’d just rather get it directly rather than the way it is assembled now. So because I go through so much trouble to reverse-engineer the data I want, not so many things will change in an obvious way on GovTrack — it’ll just be that my life will be a little easier and the information will be a little more complete and up to date. But, you can expect to see other sites spring up doing new and interesting things with the information — ways of visualizing the congressional process that we couldn’t yet imagine.

The Government Printing Office is mentioned because of how they make legislative documents like the text of bills available to the public. PDFs and text-only versions are made available for free already. No problem there. But they have other files that would be useful to sites like GovTrack which they sell at ridiculously high subscription prices. Those files would make comparisons of bill text easier to produce (although GovTrack already has this feature, again by essentially going about it the hard way). If you think about it from the perspective that some bills go through Congress so fast no one has time to read them through, being able to apply technology to the process is so important, like to detect changes in the text of bills between versions to make it easier for people to get through it. This is what GPO is preventing by selling some of its files, rather than providing them to the public for free (which it is essentially mandated to do for most documents — why they exempt certain documents is not known).

Now, it’s not that the Library doesn’t necessarily *want* to share its database. It’s just that sharing it wasn’t a part of their mandate from Congress and they don’t want to upset Congress by stepping out of their mandate. The omnibus bill is an indication from the House to the Library that this would be something supported by Congress. (My understanding is that the Library has been seeking permission from Congress to do some of these things, probably in response to a previous push for this, but the omnibus legislation has been in the works concurrently.)

It’s not very pretty because while bill writers have been posting the PDFs, PDFs don’t make it easy to make comparisons. The bill writers are composing the bills in XML, and if they made those available we the public would have an easier time. Maybe we wouldn’t complain to our reps so much either because we could actually understand what is going on better!

His office also submitted a second paragraph which I’ll get to below.

Rob Pierson in Honda’s office writes on the OHP mail list:

I’ve mentioned on the list some of the steps my boss (Congressman Honda) has been taking, with counsel from many folks on this list, to guide Congressional policies on the path towards effectively leveraging technology to open up access to the public. There are actually quite a few other staffers who also follow this list, and we’ve certainly learned quite a bit from the conversations posted here, so I wanted to throw out a quick note of appreciation to everyone who has been contributing to the discussions.

With guidance from the conversations on this list (and the OHP report), Congressman Honda recently submitted the following sections into the House Legislative Branch Appropriations Report. The following (or possibly very similar versions) were included in the Leg Branch Subcommittee markup of the bill:

*Public Access to Legislative Data (as submitted)*

The Committee believes that the public should have improved access to legislative information through more advanced search capabilities such as those available through the Library of Congress’ Legislative
Information System. The Committee also supports enhancing public access to legislative documents, bill status, summary information, and other legislative data, through more direct methods such as bulk data downloads and other means of no-charge digital access to legislative databases. The Committee requests that the Library and Government Printing Office report on the progress towards these goals within 90 days of enactment of this Act.

Note that the GPO has also been stuck in there. More more on that, see this post.

The second paragraph that Honda’s office submitted John noted was parallel to the final chapter of our report, Coordinating Web Standards. (Hmm, I principally wrote that chapter too….)

*Congressional Technology Coordination (as submitted)*

The Committee recognizes the need for the House of Representatives to develop a strategic and coordinated plan that will prepare for the future technology needs of the institution. A 2006 report commissioned by the Chief Administrative Officer and the Committee on House Administration, entitled /Strategic Technology Road Map for the Ten Year Vision of Technology in the House of Representatives/ provided a suggested structure for an IT evaluation and decision-making process.
No later than 90 days after the enactment of this Act, the Committee requests that the Chief Administrative Officer, the Clerk, and the Sergeant at Arms report to the Committee of their efforts to develop House-wide data-sharing standards; implement standard legislative document formats; address the increasing resource challenges of Member offices; and identify disparate systems throughout the institution, which prevent it from taking advantage of economies of scale.

This is of course fantastic news for anyone that supports transparency, which is, well, everyone in their right mind, I think. So thanks to Congressman Honda for taking the initiative on this!

(Other links: last year’s leg branch appropriations blog post, my first or one of my first posts here about structured data)

The database is great itself, but the cooler website is MyPyramid Menu Planner (mypyramidtracker.gov) (also out of the USDA). You can enter a typical daily roster of what you eat (with a nice sound effect) and it will tell you how it stacks up for a recommended diet for your age (or for me, how to gain weight to a recommended amount for my age). It feels a little over-simplified, but the simplicity keeps me on the site. I find, not surprisingly, that I probably eat about half of the recommended calories and clearly not enough grain or fruit. Well, I knew this in the abstract, but quantifying it helps direct me to fixing the problem.

I’m sure there are other websites that do similar things, but it’s nice to find a case where the government has both published a comprehensive (well structured, well documented) database and has also built a really nice interface for the data. And on a topic that is really very important to daily life, too.

And with that, I think I will take the rest of the weekend off from civics!

The recommendation serves to help congressional staff manage incoming communication. It’s a method of triage on the one hand, and a tool to help tally communications by position on the other. Critical as this may be, I find tallying to be incredibly superficial — and it really reveals, I think, that the world of communicating with Congress has become extremely narrow. (But I’ve written on that before.)

The page updated is here.

The updates were a combination of suggestions from Scott Horvath and Jeremy Fee at the USGS, Kol Peterson from EPA, and me, and really big thanks go to Scott and Kol for reaching out to others for input on Monday and getting the feedback back to Bev Godwin at GSA who runs webcontent.gov who published the changes only a few days later. Scott also notes that additional suggestions could still be considered (his email address is at the bottom of that page).

In making my suggestions, I turned to the Open Government Data Principles and tried to squeeze in as much as I could without overloading the document, and I drew from ideas that came up in the preparation of the Open House Project report. Some of the changes made were:

• It now provides examples of data as being documents, audio/visual recordings, and databases.
• It now says to support “the widest practical range of public uses of
  the data”. It had formerly suggested supporting the “intended” use of
  the website by visitors.
• It notes the benefit of providing data: “New uses of your agency’s
  data may become a valuable public resource that would be out of the
  scope of your own website, such as helping to keep the public informed
  about the work of your agency and supporting civic education and
  participation.”
• There is a new paragraph that I might be misunderstanding but which
  seems to make a suggestion along the lines of the recent “Invisible
  Hand” paper about the agency’s website getting the data the same way the
  public does: “Providing a uniform method to access raw data can also be
  the first step in internal development, accomplishing both goals at
  once. When a uniform method to access data is available, developers and
  web–services can focus on data presentation.”
• It notes that the availability of bulk downloads of data is something
  to consider when building data access.
• It notes some disadvantages of using proprietary formats.
• It recommends that if a proprietary format is needed, a
  non-proprietary format should be used in addition.
• It adds a benchmark to test for success: “One benchmark for
  determining whether data is made sufficiently available is whether the
  public has all of the data needed to replicate any searching, sorting,
  and display functionality provided on the agency’s own website.”
• It notes that consulting the public in the development of data access
  seems to be entailed from OMB policy: “When choosing data formats and
  distribution methods, keep in mind that your agency’s visitors are the
  best judges of their own needs. Agencies must “establish and maintain
  communications with members of the public and with State and local
  governments to ensure your agency creates information dissemination
  products meeting their respective needs” (OMB Policies for Federal
  Public Websites #4A).”

We have a real success story here.

“The new administration should specify that the federal government’s primary objective as an online publisher is to provide data that is easy for others to reuse, rather than to help citizens use the data in one particular way or another.”

And they suggest an interesting way to push that forward:

“The policy route to realizing this principle is to require that federal government websites retrieve the underlying data using the same infrastructure that they have made available to the public. Such a rule incentivizes government bodies to keep this infrastructure in good working order, and ensures that private parties will have no less an opportunity to use public data than the government itself does. The rule prevents the situation, sadly typical of government websites today, in which governmental interest in presenting data in a particular fashion distracts from, and thereby impedes, the provision of data to users for their own purposes.”

I think this is a worthwhile addition to the opengovdata and publicmarkup.org policy documents — if not as a direct recommendation (because I think it may be too much to ask for in a grand form) then noted as a long-term goal or (in terms of the second paragraph I quoted) as a benchmark, a concrete way to tell whether data is open.

The full citation is: Robinson, David, Yu, Harlan, Zeller, William P and Felten, Edward W, “Government Data and the Invisible Hand” (2008). Yale Journal of Law & Technology, Vol. 11, 2008

Closely watching legislation progress through the House can be difficult, especially given the discretion the Speaker and the Rules Committee have in scheduling and controlling the floor.

The House Rules Committee is making legislation a little easier to follow online, by voluntarily posting updates of special rules, as they are approved, to their website in a structured format. This means that you can receive updates on newly approved special rules that structure floor consideration for impending bills. They’re also offering a feed of amendments submitted to the committee.

The only way to get faster updates would be to attend the committee hearing.

In an article last week in the National Journal’s Technology Daily, Honda compared the benefits of an open legislative database to what the world has gotten out of wikis and open source software like Linux.

Then on Wednesday, Honda blogged (a perfect medium to express the sentiment) about the issue, citing the Open House project’s report. He captures the issue really well:

I have been working on an initiative to make Congressional legislative information more accessible to the public. I believe that public information should be provided in a format that takes advantage of the innovative technologies that are revolutionizing the Internet, sometimes known as Web 2.0.

Making Congress’ legislative database open to the public “would enable independent Web sites to use information in new and creative ways, including educating the public about Congress and providing citizens with customized views of its proceedings,� according to a report from the Open House Project, an organization supporting this proposal.

Offering legislative information in a way that other websites can reuse could lead to revolutionary changes in the way our government functions, eventually allowing Congress to better tap into the knowledge and wisdom of the American people.

The reason this is interesting to me is that we like to capture reality in data. The Library of Congress and GovTrack both systematize (or in computer jargon “normalize”) the bill-becomes-a-law process. At every point in the game, a bill, in our data formats, is either in-progress, enacted, dead, etc. It must be in one of these states. After all, the constitution outlines exactly what states a bill can be in, so any bill *must* be in one of these states.

But if we’re not sure what state a bill is in, what state do we put it in in our data? There’s also the more important question- What do the lawmakers do if they disagree about what state a bill is in? (Actually, I would prefer to phrase it as “what state they are in”, but that’s another story.) Wikipedia describes (what the editors of the page claim is) a current debacle over H.R. 1585: National Defense Authorization Act FY 2008:

In December of 2007, President George W. Bush pushed the pocket veto into murky waters by claiming that he had pocket vetoed H.R. 1585, the “National Defense Authorization Act for Fiscal Year 2008,” even though the House of Representatives had designated agents to receive presidential messages before adjourning. The bill had been previously passed by veto-proof majorities in both the House and the Senate [JT: and thus a traditional veto would have been futile].

So was the bill (pocket) vetoed or not? Is the bill still in-progress? Assuming it was not pocket vetoed, after 10 legislative days without a traditional veto it becomes law, and us citizens would hate to be on that 11th day without either resolution on the pocket veto matter or a traditional veto, because then we as a country will not know whether this bill has become law. (Another question: How might the Supreme Court assert jurisdiction over this question.)

But back to the data. At one point, some time after Dec. 28, someone in the House responsible for updating the bill status information shown on THOMAS entered a new status line:

Dec 28, 2007: Pocket Vetoed by President.

GovTrack picked up on the change and shows that status currently, much to the confusion of several people emailing me about it. Looking back at THOMAS, it seems like someone realized that that was apparently quite a constitutional (if not political) claim and retracted that update, because it not longer says that.

In many cases citizens complain when the government takes things back, hiding information previously made public. That’s definitely not what I am getting at here. THOMAS is forced to show *something*, and when it doubt… well, what can you do but roll back history until we figure out what the next legislative step actually *was*.