The two paragraphs were added by Congressman Mike Honda of California, one of our champions of the use of technology to further transparency and civic engagement. John Wonderlich of Sunlight Foundation, Rob Pierson in Honda’s office, and I collaborated on this over a long period of time. Honda got involved in 2007 asking the Library to look into this and then in 2008 getting the paragraphs added to the bill markup.

So here they are:

Congressional Technology Coordination.-The House of Representatives needs a strategic and coordinated plan that will prepare for the future technology needs of the institution. A 2006 report commissioned by the Chief Administrative Officer and the Committee on House Administration, entitled Strategic Technology Road Map for the Ten Year Vision of Technology in the House of Representatives, provided a suggested structure for Information Technology evaluation and decision making. The Chief Administrative Officer, the Clerk, and the Sergeant at Arms are asked to prepare a report by June 30, 2009 on their efforts or plans to develop House-wide data-sharing standards; implement standard legislative document formats; address the increasing resource challenges of Member offices; and identify disparate systems throughout the institution that prevent it from taking advantage of economies of scale. [page 2]

and

Public Access to Legislative Data. There is support for enhancing public access to legislative documents, bill status, summary information, and other legislative data through more direct methods such as bulk data downloads and other means of no-charge digital access to legislative databases. The Library of Congress, Congressional Research Service, and Government Printing Office and the appropriate entities of the House of Representatives are directed to prepare a report on the feasibility of providing advanced search capabilities. This report is to be provided to the Committees on Appropriations of the House and Senate within 120 days of the release of Legislative Information System 2.0. [page 11]

According to an article in Wired: “In our web 2.0 world, we can empower the public by providing them with raw data that they can remix and reuse in new and innovative ways,” says Honda, who is vice chairman of the Appropriations Subcommittee on the Legislative Branch. “With these tools, the public can collaborate on projects that can help legislators to create better policies to address the pressing challenges facing our nation.” There’s also a good article at Mother Jones and a nice post by Tim O’Reilly.

The concept of bulk data downloads hasn’t been missed by many parts of the government. The Census Bureau and the Federal Elections Commission, for instance, are fantastic at sharing with the public as much as they can. In the latter case it is electronic versions of campaign contribution filings, which is obviously very important for preventing corruption. But, there are significant gaps in other areas of the government where a little legislating is necessary. Here we’re talking about information on bills in congress going back around two decades, and the information going forward.

The Library of Congress has a database of this information but they don’t share it with the public. Sharing it would mean that creating sites like GovTrack — and the various other sites that use data from GovTrack including OpenCongress and MAPLight.org — would be a little easier, but also a little more accurate. Right now GovTrack goes through a roundabout process to reverse-engineer the same information we are seeking from this database. Basically, we already have the information by scraping it off of thomas.loc.gov — we’d just rather get it directly rather than the way it is assembled now. So because I go through so much trouble to reverse-engineer the data I want, not so many things will change in an obvious way on GovTrack — it’ll just be that my life will be a little easier and the information will be a little more complete and up to date. But, you can expect to see other sites spring up doing new and interesting things with the information — ways of visualizing the congressional process that we couldn’t yet imagine.

The Government Printing Office is mentioned because of how they make legislative documents like the text of bills available to the public. PDFs and text-only versions are made available for free already. No problem there. But they have other files that would be useful to sites like GovTrack which they sell at ridiculously high subscription prices. Those files would make comparisons of bill text easier to produce (although GovTrack already has this feature, again by essentially going about it the hard way). If you think about it from the perspective that some bills go through Congress so fast no one has time to read them through, being able to apply technology to the process is so important, like to detect changes in the text of bills between versions to make it easier for people to get through it. This is what GPO is preventing by selling some of its files, rather than providing them to the public for free (which it is essentially mandated to do for most documents — why they exempt certain documents is not known).

Now, it’s not that the Library doesn’t necessarily *want* to share its database. It’s just that sharing it wasn’t a part of their mandate from Congress and they don’t want to upset Congress by stepping out of their mandate. The omnibus bill is an indication from the House to the Library that this would be something supported by Congress. (My understanding is that the Library has been seeking permission from Congress to do some of these things, probably in response to a previous push for this, but the omnibus legislation has been in the works concurrently.)

It’s not very pretty because while bill writers have been posting the PDFs, PDFs don’t make it easy to make comparisons. The bill writers are composing the bills in XML, and if they made those available we the public would have an easier time. Maybe we wouldn’t complain to our reps so much either because we could actually understand what is going on better!

Congressional Republicans recently released a high production value video on the FISA fight, which prompted a quick response video in return. Political positions and identities are crafted in slicker and more appealing ways, witnessed also in recent discussions of presidentials’ skill at creating a coherent brand. While self-promotion is somewhat essential to electoral success, political messaging, much like commercial advertising, is sculpted around the expectations of the viewer. Savvier voters require savvier messaging (the intentionally lo-fi notwithstanding).

Lucky for us, the same forces that are making it easier to create propoganda are granting us a new view of our civic world. I’m starting to collect political data visualizations more consciously, with an eye to what might be missing.

A quick search already uncovered this gem: a visualization of the words in former Attorney General Alberto Gonzales testimony. I love the added semantic context; the words can be rearranged and positioned within the sentences spoken, like a syntactic tag cloud.

Here’s a blog just about turning census data into KML files (which can be run within Google Maps). They have a great shot of all of the zip codes in Maine, arranged by color. (Census Data KML Visualization, aptly enough.)

Josh Tauberer’s GovTrack.us is full of ingeniously autogenerated mapping and visualizations, like his cartograms for each vote that make geography and representation equivalent, or the map of all congressional districts overlayed on a google map tile.

MySociety.org’s time travel maps allow users to define a point (near London), and then dynamically shade the map with travel time and property prices, which could be obviously valuable when home hunting. (If this is confusing, check it out… It’s easier to see than to explain.)

Good policy only comes from good information, which is only as helpful as it is comprehensible. That’s why I think we’re so drawn to new or novel data visualizations; we function as data visualizers all the time, and intuitively create schemas to help us navigate the world. Seeing facts and trends given new intuitive force through design and aesthetics is very pleasing.

Anyone have good congressional data visualizations?

Here’s a video of our panel; the other panels are accessible here.

The main thing I took away from the event is that the tension between centralized and decentralized computing will force us to rethink all aspects of digital culture. Privacy means something entirely different when data all goes through centralized servers, the open source movement is fundamentally affected when software becomes web-based (and therefore not even installed on individuals’ computers), security concerns change along with architectural developments, and new relationships between institutions breed new power struggles, new regulatory contexts, and new opportunities for organization.

The reason this is interesting to me is that we like to capture reality in data. The Library of Congress and GovTrack both systematize (or in computer jargon “normalize”) the bill-becomes-a-law process. At every point in the game, a bill, in our data formats, is either in-progress, enacted, dead, etc. It must be in one of these states. After all, the constitution outlines exactly what states a bill can be in, so any bill *must* be in one of these states.

But if we’re not sure what state a bill is in, what state do we put it in in our data? There’s also the more important question- What do the lawmakers do if they disagree about what state a bill is in? (Actually, I would prefer to phrase it as “what state they are in”, but that’s another story.) Wikipedia describes (what the editors of the page claim is) a current debacle over H.R. 1585: National Defense Authorization Act FY 2008:

In December of 2007, President George W. Bush pushed the pocket veto into murky waters by claiming that he had pocket vetoed H.R. 1585, the “National Defense Authorization Act for Fiscal Year 2008,” even though the House of Representatives had designated agents to receive presidential messages before adjourning. The bill had been previously passed by veto-proof majorities in both the House and the Senate [JT: and thus a traditional veto would have been futile].

So was the bill (pocket) vetoed or not? Is the bill still in-progress? Assuming it was not pocket vetoed, after 10 legislative days without a traditional veto it becomes law, and us citizens would hate to be on that 11th day without either resolution on the pocket veto matter or a traditional veto, because then we as a country will not know whether this bill has become law. (Another question: How might the Supreme Court assert jurisdiction over this question.)

But back to the data. At one point, some time after Dec. 28, someone in the House responsible for updating the bill status information shown on THOMAS entered a new status line:

Dec 28, 2007: Pocket Vetoed by President.

GovTrack picked up on the change and shows that status currently, much to the confusion of several people emailing me about it. Looking back at THOMAS, it seems like someone realized that that was apparently quite a constitutional (if not political) claim and retracted that update, because it not longer says that.

In many cases citizens complain when the government takes things back, hiding information previously made public. That’s definitely not what I am getting at here. THOMAS is forced to show *something*, and when it doubt… well, what can you do but roll back history until we figure out what the next legislative step actually *was*.

After returning earlier this week from a trip to London to meet with mysociety.org, I’m starting to catch up on everything we’ve been up to.

Whips and Structured Data: For one, I’ve reached out to leadership on both sides to hopefully start a discussion about using whip publications as a resource for indexing votes to party positions for research purposes.  The parties have a clear interest in having their position be easily determined (even long after the fact), and structured data seems to be the best solution.

There are probably two levels of data coordination here: the first is to publish whip packs as RSS (which neither side is doing at the moment)–this makes the content accesible without needing to screenscrape.  Second, the party positions for each bill should be indexed to each bill.  I’m not sure whether there is a single format for this purpose (I suspect one of Josh Tauberer’s conventions may be the de facto standard?), but indexing would make the task of linking legislation back to the party line much easier.  (I should add that we’re lucky to have public whip notices at all, this information is unavailable in the UK.)  Here’s an illustration of someone trying to derive the party line by poster Democratic Luntz at daily kos (part of a labor-intensive series), which also ties into the usefullness of structured data in posting votes information.

Preservation: Next, I’ve been on a mission to research the Advisory Committee on the Records of Congress.  This group meets occasionally, and features membership like the Secretary of the Senate, the Clerk of the House, the House Archivist, and others.  This seems to me to be the best example of cross-departmental cooperation, which is reassuring given how important it is to access such fundamentally important information.

They frequently reference a document from 1992, written in participation with a community of archivists, on documenting Congress.  The document is S.Pub 102-20, which I’ve spent a great deal of time trying to access for the last few weeks.  Research librarians at the LOC first suggested that S.Pub wasn’t actually a class of document, and were later able to send me a detailed citation.  A visit to the LOC then led me on a search through the main reading room, the microform room, and then to a delayed visit to the law library.  Everyone was really helpful and knowledgable, I must say, but I do have to remark that I’m amused that a document about making congressional information public can be so difficult to find, even for the LOC.  Is cross departmental cooperation so unusual?

In any case, I’ve finally gotten to read S.Pub 102-20, and it’s really the holy grail of congressional information surveys, delivering 120 pages of detailed analysis and recommendations, remarking on the nascent internet and the state of documents availability for every aspect of Congress, a survey I wish I had had a year ago.  I’ll be scanning the document from microfilm over the next few weeks, and will post it whenever I’m able to.  (If anyone has access to it in a more accessible format, I’d be very grateful.)

This work fits in closely with the idea that information availability is more than an abstract idea about good government, but really a necessary condition for meaningful deliberation; the societal equivalent of an operational memory, whose function is impaired by a technology that develops before archiving practices can keep up.

I’ve got several more things to write about, but that’s probably more than enough for one email.  More soon.

To that end, I’m going to be writing more about what I’ve been doing, and more about the current state of transparency reform in the House (and Senate, soon hopefully).

We’re also going to have a lot to say in the coming months about awareness online, and political and civic information.

To get everyone caught up on what’s been happening, I’m going to be writing a series of emails and blog posts on each section of our report, and then doing probably weekly updates on what new developments I’ve got to discuss.

Legislative Database:

Josh’s post on the OHP blog gives the best update about what we’re looking for and what already exists. I don’t really have much to add to what he wrote (in great detail) in that post, except that GovTrack continues to provide really compelling examples of how better implementation of structured data can lead to useful and creative combinations of data later. As more data streams gain a semantic component, through either RDF or some similar stopgap measure such as crowdsourcing (digg, wikis, etc), paid staff adding value (cf. Congressional Quarterly), or new alternatives such as daylife.com, structured data will become a minimum expectation.

The transition to XML has been happening in the house for quite some time, and will probably continue to do so, given all of the separate sets of data, and users, and the need to make any kind of IT transition VERY smoothly and VERY securely. When compared with the potential consequences private companies work under, I think it’s easier to understand why Congress lags behind the private sector in adopting new technology. Clear priorities, well thought out transitions, and funding immune to political manipulation will all be essential to a Congress which becomes more technologically capable, efficient, and transparent.

Preservation:

The other section I’d like to discuss today is preservation.

Concerns over preserving born-digital congressional documents come up constantly and in reference to all of the other sections of the report.

Some of the biggest potential for large improvement exists in this area, since we’re doing such an incomplete job of this type of archiving and preservation now. A good first step would be to fully fund the NDIIP program, as I wrote in reviewing the House Legislative Branch Appropriations report:

I’d also like to be able to more clearly articulate the way in which an empowered FDLP could help NARA, FDLP, or NDIIP to comprehensively back up web-based congressional documents. Further help fleshing that out would be appreciated.

I’ll be writing more in the next few days about a few other sections of the report.

Josh Tauberer, Govtrack’s creator and proprietor, has gone far beyond building a simple tool to help track congressional proceedings; Josh’s creation has become a fundamental fixture in terms of both government information and structured data, a result of his extensive knowledge of both advanced linguistics, and computer programming. Josh’s willingness to volunteer his expertise also led to him helping to form and author the recommendations of the Open House Project, a separate Sunlight project.

Here’s Josh Tauberer briefly telling the story of Govtrack: (click below to play)

http://www.sunlightfoundation.com/files/mp3/govtrack.mp3

GovTrack’s user oriented design and creative combinations of different data sources have garnered praise from notable sources, including Peggy Garvin, The New York Times, and The Washington Post, and also help make GovTrack useful for a variety of different users.

There’s a lot more to this review; click below to keep reading…

Key to becoming an Insanely Useful Site is the ease of access to the site’s data for an amateur seeker of political information — or in Internet speak, a noob. GovTrack provides numerous tools to help amateurs find information that is relevant to their immediate interests in Congress, whether that be electoral (should I vote for my current congressperson?) or because of a specific interest in a particular bill. Usually the point of entry to government information for an amateur is a desire to learn about who represents them.

So, say I have a member of Congress, which, in reality, I don’t (D.C.), and let’s say I live in Newton, Massachusetts, which means that Rep. Barney Frank (D-MA) is my congressman. But let’s pretend that I don’t know that. When I get to the home page of GovTrack I notice in the upper left hand corner that I can search for my representative. If I don’t know his/her last name I can click to a “zoomable district map“. The “zoomable district map” — it takes a second to load — is a Google mash-up map with every Congressional district visible and clickable. I can zoom into Massachusetts and see that Newton is within the boundaries of the 4th CD of Massachusetts and therefore I am represented by Rep. Frank.

Now that I’ve found my Representative I can click through to their individual page. GovTrack’s individual member pages are incredibly detailed and use supplemental information often not found on member’s official Web sites or easily accessible through the THOMAS Web site. Each individual page contains a list of recent votes, a link to past votes, sponsored and cosponsored bills, speeches on the floor of the House, committee membership, and links to campaign contributions and video clips. For someone simply searching for a way to easily track their member of Congress, GovTrack provides RSS feeds of all information related to your representative and the ability to receive e-mail alerts on your member of Congress’ activity.

If you want to go deeper into the Congressional process you can follow the committees on which your member of Congress sits. For this example, Barney Frank is chairman of the House Financial Services Committee. The committee’s page holds similar information about the committee that is available on member pages. Each committee page provides a list of members on the committee with pictures, all bills before the committee, all bills that have been enacted into law and bills that have not been enacted. RSS feeds and e-mail updates are also available for the committee. Unfortunately, quite a good deal of committee information is simply not released to the public either at all (committee votes) or in a timely fashion (committee transcripts), so GovTrack is not able to track these important pieces of committee data.

The best part about these member and committee pages is the intuitive design of each one and the ease of understanding the information. The data is not cluttered and confusing, as THOMAS often can be, and the information is presented in a thoughtful way, so that the amateur can find the information they need without wading through piles of irrelevant and aggravating information or doing multiple unsuccessful searches.

Legislative Detail

While GovTrack’s greatest asset is its accessible presentation of information and ease of use, the expert and political professional will find numerous features useful that the amateur may not be seeking.

The bill pages are an excellent example of GovTrack’s accessible presentation directed at expert users. For example, GovTrack presents bills with highlighted text to show which changes have been made as the bill makes its way through Congress. The recent bill for relief after the bridge collapse in Minneapolis is a perfect example. As you can see the colors red, yellow, and blue respectively denote removed text, changed text, and inserted text into the bill. GovTrack also provides all previous versions of the bill with highlighted text so you can track changes from introduction to passage. This allows any researcher or political expert to immediately look at a bill that they are following and know what was changed without having to spend fifteen minutes or an hour parsing the text.

Another great feature of GovTrack for researchers is the ability to search votes all the way back to 1993. If you are trying to compile or compare information on particular members of Congress and their voting records, GovTrack is clearly the place to do your research.

A favorite feature is the ability to search by subject terms. While, unfortunately, you can’t add your own, there are a large number of subject terms to search by. Tracked events alerts are available for each subject term and are very useful for a busy researcher or blogger who needs immediate updates on bills in their issue area of choice.

Always important to anyone doing political work professionally is the ability to check primary sources. Each bill page provides a link to the original text on THOMAS if you need to check the information presented on GovTrack to that provided by the Library of Congress. (For example, just this morning we used GovTrack to find S.1, followed the link to THOMAS, and used THOMAS to find a good .pdf version for printing.)
The massive number of RSS feeds on vote, member, and bill information is also a great asset of GovTrack. Finally, GovTrack provides an RSS feed on all upcoming committee hearings. This is an excellent feature if you need to keep up on what’s happening on the Hill in coming weeks.

Web Developers and Tech Enthusiasts

For developers and tech enthusiasts, GovTrack.us is a vast experiment in the political application of structured information. By converting the THOMAS versions of bills into RDF and XML, the data gains new uses, more easily mixed with other data sources. For example, OpenCongress.org takes GovTrack’s bill data and combines it with news feeds and blog coverage from technorati to provide further cultural context for legislative action. Whereabill.org uses GovTrack data to physically locate the bill in the Capitol complex, and highlight it on a google map.

APIs (Application Programming Interfaces) are the techincal mechanism by which database proprietors permit other servers to access their information. A well known example would probably be google maps. If you’ve ever used a “store locator” when shopping, and had the location appear in a google map, then the site where you entered the search also sent a query to the google maps database, and then combined your entry with the results from the google maps result.

GovTrack accesses APIs to function, and also creates APIs to empower other sites to build on the data sets that Josh is creating. For example, the zoomable district maps page is built on an API, and is then available as a new API for others to use.

Political and legislative information are becoming more useful as they become both more standardized, and also more decentralized (and easily distributed). These developments only happen when new data standards are developed and utilized, and the information released in its most useful form, without copyright restriction. GovTrack is a huge leap in that direction, implementing well conceived data standards and empowering web designers to integrate congressional data into their projects.

Written by John Wonderlich and Paul Blumenthal