In an article last week in the National Journal’s Technology Daily, Honda compared the benefits of an open legislative database to what the world has gotten out of wikis and open source software like Linux.

Then on Wednesday, Honda blogged (a perfect medium to express the sentiment) about the issue, citing the Open House project’s report. He captures the issue really well:

I have been working on an initiative to make Congressional legislative information more accessible to the public. I believe that public information should be provided in a format that takes advantage of the innovative technologies that are revolutionizing the Internet, sometimes known as Web 2.0.

Making Congress’ legislative database open to the public “would enable independent Web sites to use information in new and creative ways, including educating the public about Congress and providing citizens with customized views of its proceedings,� according to a report from the Open House Project, an organization supporting this proposal.

Offering legislative information in a way that other websites can reuse could lead to revolutionary changes in the way our government functions, eventually allowing Congress to better tap into the knowledge and wisdom of the American people.

Even though that video centers on intellectual property issues, Lessig talks about how his focus came to shift away from hoping Congress would pass rational policy. He remarks that the "economies of influence" that dictate congressional policy are fundamentally corrupt, as a system. That made me reflect that Sunlight’s mission is, in a sense, to cultivate an ecology of popular influence, to build the culture of information and deliberation that is necessary for good governance. I wonder how often it is that people are drawn to process reform by virtue of their frustration in working on other issues? The leap shouldn’t be difficult for Lessig to make, since the cultural shift he describes as changing the nature of creativity (and, therefore, the landscape of intellectual property) — that cultural shift to creativity and digital empowerment is exactly the cultural sphere in which Sunlight is thriving. Instead of mashups of disparate clips of audio and video, we’re working with legislative data, creating an approachable and relevant congressional pallette of civic information.

This video speaks also to that cultural shift, discussing exponential growth of digital culture.

Summary: The Senate’s current position on publishing voting records online is analogous to a reference library that has no copy machine. I explain below why the Senate website should publish its roll call vote records in “XML format”, to facilitate educating the public and strengthening transparency, and why any reluctance there may be should be reevaluated in light of the experience from the House’s use of XML for roll call votes and the presence today of unauthoritative XML for Senate votes. Current Senate website policy should be revised to encourage the use of this “structured data format”.

Though everyone believes an electorate must be informed to make wise decisions at the polls, the complexities of what happens in the Congress are indeed difficult to distill and share with the public. Roll call voting records are of crucial importance to the public for obvious reasons, but at the same time fail to capture the nuances of each situation that may have played a central role in a Senator’s decision making. How voting records, which are easy to convey but oversimplify the big picture, should be responsibly shared with the public is a question for debate. I suggest below that the Senate website publish its roll call vote records in “XML format” (in addition to what is currently available) to help keep the public informed, and that any fears about how the information in XML may be used are not strong enough reasons to avoid this technology.

The Senate’s current position on publishing voting records online is analogous to a reference library that has no copy machine. In a reference library without a copy machine, the information in the stacks is certainly made available, but library members can’t easily share the information with others. They can instruct others how to find the information in the library (i.e. a link), and they can copy the information by hand and make copies at Kinkos, but library members are unable to use the latest technology to help them share the information outside the library. In such a world, the library members’ response is likely to be to haul in their own copy machines into the library. This is exactly what has happened with Senate voting records.

Leaving the metaphor, long ago the Senate took the important step of publishing voting records on its website. Though the votes webpages themselves cannot capture all of the nuances of each vote, these webpages complement what exists elsewhere on the web. For instance, the websites of newspapers, which do try to explain the back-story of legislative issues to present a larger picture, often link to the Senate’s roll call webpages as, in a sense, an extension of their own reporting, that is, so they can provide not just the big picture but also the crucial details. The roll call webpages thus have an important role in educating the electorate and promoting transparency.

The metaphorical copy machine represents what is called structured data, for example “XML.” XML allows computers to more easily process information, and for voting records would help that information be disseminated more widely and in novel ways to the public. While structured data is a part of today’s so-called “Web 2.0″, the current policy understood to be coming from Senate Administration is that the Senate website is not to publish structured data for roll call votes, with the reason understood to be that Senators prefer to have their votes be published not as isolated factoids, where they could be misrepresented, but rather only as part of a larger picture.

This policy warrants review on two accounts. On the one hand, even such isolated facts have a crucial role of complementing the larger picture presented elsewhere, as does the existing Senate webpages for votes as explained above. But further, for several years the House has published its voting records in XML. The New York Times, for instance, makes use of these files to enhance their own coverage of legislation by including visual representations of votes along with their articles — the big picture and the crucial details. XML made the voting information more easily transformed into visual form, a form that has educational value to the public, and so using XML is in this respect in the public interest. The Senate does not publish XML, and while as with the metaphorical reference library this does not prevent wholesale access to the information, it is holding back on technology that facilitates educating others. The Senate should adopt a similar policy as the House to encourage the dissemination of voting information, knowing from the experience of the House that it will be used often to complement reporting of the nuances and the big picture.

Because it does not publish votes in XML, the public has hauled in its own copy machine — and the effect is that Senate vote XML files are available to the public, Senate rules notwithstanding. The independent website GovTrack.us publishes its own XML files for Senate votes, and these are used by several other websites to enhance the public’s understanding of the Congress. Any fears Senators might have had for a future with XML can thus be evaluated today. However, this unauthoritative source for voting information is not an optimal solution, on account of the fact that on rare occasions it disseminates incorrect information to some hundreds of thousands of monthly visitors of the websites using these XML files. An authoritative source of roll call vote XML files from the Senate directly would rectify this problem.

As there is virtually no cost to publishing XML files for roll call votes, and in light of the experience that can be gathered from the House’s use of XML and the presence today of (unauthoritative) XML for Senate votes, the current policy regarding the use of structured data on the Senate website should be reevaluated. The use of structured data should be encouraged for all public information on the Senate website, especially starting with roll call votes, and would signal a renewed commitment to using technology to promote transparency.

(this video is really amazing. follow the above link.)

One of the tools I rely on to stay aware is netvibes. Several similar aggregators work basically the same, and I’m accustomed to this one. Feed aggregators’ usefullness is multiplied by services like yahoo pipes, which allow multiple feeds to be combined, or even transformed with basic operations and filters. My favorite pipe is the blended Sunlight Foundation feed, an option on the left side of sunlightfoundation.com, which lets me follow every post from all of the Sunlight Foundation blogs, along with citations from Paul Blumenthal’s Ethics Watch, featuring links to ethics articles, often from the Hill and Roll Call.

Here’s the feed as it appears on my customized netvibes page:

and here’s what the feed looks like when being edited in yahoo pipes:

When Paul recently searched for members of Congress with blogs, I combined them into one unwieldy pipe, which lets you see every blog post from Congress (or most of them) in order. I’m not sure what the practical side of this might be, but if someone wants to play with it, that feed is available here.

Data visualizations’ effects reach well beyond the aesthetically creative, as this post observes:

Transparent recycling bins in Shibuya station.

The extent that transparency of what is disposed changes consumption habits. to what extent is a consumer less likely to buy products with a social stigma such as a pornographic magazine or more virtuous products when the act of consumption and disposal is transparent?

It seems that encouraging public awareness has an effect on incentive structures, even outside the public sector, even with respect to garbage bag design.

A more clearly compelling example of public information presented graphically would be Sunlight Labs’ earmark visualization project, which demonstrates spending allocation priorities much more effectively than text. Simple data sets have greater public appeal when presented visually.

Digital analysis can also lead to new types of connections and data sets, built on newer tools that have a semantic component, utilizing languages like RDF, natural language processing, or other complex analysis methods, like topic maps.

Some robust news collection tools that are developing semantic components include textmap.com, featuring ambitiously detailed relationship processing (click around there–very interesting stuff), or daylife.com, with a pleasing interface to semantically sorted news items.

Even with a more limited semantic component, however, the broader adoption web 2.0 style data presentation will ensure that basic political information is better diffused throughout society. For a pertinent example, see this Utah blogger who has taken OpenCongress.org’s tools, combined them in widgetbox, and used them to create a “blidget” (portmanteau of blog and widget) featuring their representatives’ latest votes. Very cool.

We made an entry a week or so ago for Gmail’s promotional video contest, featuring us throwing around an enlargement of the Gmail sign. Using Google’s intuitively designed tools, I drew up a basic map of where most of the video took place. Click on the blue pins in the map to see the video and details.

Make sure to watch the bloopers video, which was pretty hilariously edited by Nisha.

View Larger Map

For more details on how to create your own map, see the Gmail blog here.

To that end, I’m going to be writing more about what I’ve been doing, and more about the current state of transparency reform in the House (and Senate, soon hopefully).

We’re also going to have a lot to say in the coming months about awareness online, and political and civic information.

To get everyone caught up on what’s been happening, I’m going to be writing a series of emails and blog posts on each section of our report, and then doing probably weekly updates on what new developments I’ve got to discuss.

Legislative Database:

Josh’s post on the OHP blog gives the best update about what we’re looking for and what already exists. I don’t really have much to add to what he wrote (in great detail) in that post, except that GovTrack continues to provide really compelling examples of how better implementation of structured data can lead to useful and creative combinations of data later. As more data streams gain a semantic component, through either RDF or some similar stopgap measure such as crowdsourcing (digg, wikis, etc), paid staff adding value (cf. Congressional Quarterly), or new alternatives such as daylife.com, structured data will become a minimum expectation.

The transition to XML has been happening in the house for quite some time, and will probably continue to do so, given all of the separate sets of data, and users, and the need to make any kind of IT transition VERY smoothly and VERY securely. When compared with the potential consequences private companies work under, I think it’s easier to understand why Congress lags behind the private sector in adopting new technology. Clear priorities, well thought out transitions, and funding immune to political manipulation will all be essential to a Congress which becomes more technologically capable, efficient, and transparent.

Preservation:

The other section I’d like to discuss today is preservation.

Concerns over preserving born-digital congressional documents come up constantly and in reference to all of the other sections of the report.

Some of the biggest potential for large improvement exists in this area, since we’re doing such an incomplete job of this type of archiving and preservation now. A good first step would be to fully fund the NDIIP program, as I wrote in reviewing the House Legislative Branch Appropriations report:

I’d also like to be able to more clearly articulate the way in which an empowered FDLP could help NARA, FDLP, or NDIIP to comprehensively back up web-based congressional documents. Further help fleshing that out would be appreciated.

I’ll be writing more in the next few days about a few other sections of the report.

Josh Tauberer, Govtrack’s creator and proprietor, has gone far beyond building a simple tool to help track congressional proceedings; Josh’s creation has become a fundamental fixture in terms of both government information and structured data, a result of his extensive knowledge of both advanced linguistics, and computer programming. Josh’s willingness to volunteer his expertise also led to him helping to form and author the recommendations of the Open House Project, a separate Sunlight project.

Here’s Josh Tauberer briefly telling the story of Govtrack: (click below to play)

http://www.sunlightfoundation.com/files/mp3/govtrack.mp3

GovTrack’s user oriented design and creative combinations of different data sources have garnered praise from notable sources, including Peggy Garvin, The New York Times, and The Washington Post, and also help make GovTrack useful for a variety of different users.

There’s a lot more to this review; click below to keep reading…

Key to becoming an Insanely Useful Site is the ease of access to the site’s data for an amateur seeker of political information — or in Internet speak, a noob. GovTrack provides numerous tools to help amateurs find information that is relevant to their immediate interests in Congress, whether that be electoral (should I vote for my current congressperson?) or because of a specific interest in a particular bill. Usually the point of entry to government information for an amateur is a desire to learn about who represents them.

So, say I have a member of Congress, which, in reality, I don’t (D.C.), and let’s say I live in Newton, Massachusetts, which means that Rep. Barney Frank (D-MA) is my congressman. But let’s pretend that I don’t know that. When I get to the home page of GovTrack I notice in the upper left hand corner that I can search for my representative. If I don’t know his/her last name I can click to a “zoomable district map“. The “zoomable district map” — it takes a second to load — is a Google mash-up map with every Congressional district visible and clickable. I can zoom into Massachusetts and see that Newton is within the boundaries of the 4th CD of Massachusetts and therefore I am represented by Rep. Frank.

Now that I’ve found my Representative I can click through to their individual page. GovTrack’s individual member pages are incredibly detailed and use supplemental information often not found on member’s official Web sites or easily accessible through the THOMAS Web site. Each individual page contains a list of recent votes, a link to past votes, sponsored and cosponsored bills, speeches on the floor of the House, committee membership, and links to campaign contributions and video clips. For someone simply searching for a way to easily track their member of Congress, GovTrack provides RSS feeds of all information related to your representative and the ability to receive e-mail alerts on your member of Congress’ activity.

If you want to go deeper into the Congressional process you can follow the committees on which your member of Congress sits. For this example, Barney Frank is chairman of the House Financial Services Committee. The committee’s page holds similar information about the committee that is available on member pages. Each committee page provides a list of members on the committee with pictures, all bills before the committee, all bills that have been enacted into law and bills that have not been enacted. RSS feeds and e-mail updates are also available for the committee. Unfortunately, quite a good deal of committee information is simply not released to the public either at all (committee votes) or in a timely fashion (committee transcripts), so GovTrack is not able to track these important pieces of committee data.

The best part about these member and committee pages is the intuitive design of each one and the ease of understanding the information. The data is not cluttered and confusing, as THOMAS often can be, and the information is presented in a thoughtful way, so that the amateur can find the information they need without wading through piles of irrelevant and aggravating information or doing multiple unsuccessful searches.

Legislative Detail

While GovTrack’s greatest asset is its accessible presentation of information and ease of use, the expert and political professional will find numerous features useful that the amateur may not be seeking.

The bill pages are an excellent example of GovTrack’s accessible presentation directed at expert users. For example, GovTrack presents bills with highlighted text to show which changes have been made as the bill makes its way through Congress. The recent bill for relief after the bridge collapse in Minneapolis is a perfect example. As you can see the colors red, yellow, and blue respectively denote removed text, changed text, and inserted text into the bill. GovTrack also provides all previous versions of the bill with highlighted text so you can track changes from introduction to passage. This allows any researcher or political expert to immediately look at a bill that they are following and know what was changed without having to spend fifteen minutes or an hour parsing the text.

Another great feature of GovTrack for researchers is the ability to search votes all the way back to 1993. If you are trying to compile or compare information on particular members of Congress and their voting records, GovTrack is clearly the place to do your research.

A favorite feature is the ability to search by subject terms. While, unfortunately, you can’t add your own, there are a large number of subject terms to search by. Tracked events alerts are available for each subject term and are very useful for a busy researcher or blogger who needs immediate updates on bills in their issue area of choice.

Always important to anyone doing political work professionally is the ability to check primary sources. Each bill page provides a link to the original text on THOMAS if you need to check the information presented on GovTrack to that provided by the Library of Congress. (For example, just this morning we used GovTrack to find S.1, followed the link to THOMAS, and used THOMAS to find a good .pdf version for printing.)
The massive number of RSS feeds on vote, member, and bill information is also a great asset of GovTrack. Finally, GovTrack provides an RSS feed on all upcoming committee hearings. This is an excellent feature if you need to keep up on what’s happening on the Hill in coming weeks.

Web Developers and Tech Enthusiasts

For developers and tech enthusiasts, GovTrack.us is a vast experiment in the political application of structured information. By converting the THOMAS versions of bills into RDF and XML, the data gains new uses, more easily mixed with other data sources. For example, OpenCongress.org takes GovTrack’s bill data and combines it with news feeds and blog coverage from technorati to provide further cultural context for legislative action. Whereabill.org uses GovTrack data to physically locate the bill in the Capitol complex, and highlight it on a google map.

APIs (Application Programming Interfaces) are the techincal mechanism by which database proprietors permit other servers to access their information. A well known example would probably be google maps. If you’ve ever used a “store locator” when shopping, and had the location appear in a google map, then the site where you entered the search also sent a query to the google maps database, and then combined your entry with the results from the google maps result.

GovTrack accesses APIs to function, and also creates APIs to empower other sites to build on the data sets that Josh is creating. For example, the zoomable district maps page is built on an API, and is then available as a new API for others to use.

Political and legislative information are becoming more useful as they become both more standardized, and also more decentralized (and easily distributed). These developments only happen when new data standards are developed and utilized, and the information released in its most useful form, without copyright restriction. GovTrack is a huge leap in that direction, implementing well conceived data standards and empowering web designers to integrate congressional data into their projects.

Written by John Wonderlich and Paul Blumenthal