Data in the City

On Monday of this week I attended a hearing in New York City organized by the Technology and Government Committee of the New York City Council. On the agenda was a proposal (Int. No. 991) regarding the use of open standards for publishing New York city government data. I picked up a printed copy of the proposal and a summary when I walked into the hearing. To my surprise the handout referred to W3C by name (the online proposal does not) and included a reference to the recent publication of the eGovernment Interest Group Improving Access to Government through Better Use of the Web.

So I filled out a form requesting to speak. To my surprise, the Chair invited me to testify early in the hearing.

Before I spoke, however, a representative from the Mayor's Office voiced opposition to some specifics of the proposal. Earlier that day, at the Personal Democracy Forum elsewhere in the city, the Mayor himself announced several initiatives regarding publishing government data. This had generated some excitement, and a number of people who had been attending the conference (I had not) were present at the hearing.

The Mayor's Office cited 5 or 6 reasons why it opposed the particular proposal (which I trust will appear in the public record that I've not yet located) but the main ones I recall were cost and burden. I would paraphrase some of the exchange between the city council committee and the Mayor's office as follows:

• City Council: Please put raw data on the Web.
• Mayor's Office: We prefer publishing information that is less raw and more citizen-friendly.
• City Council: Citizens won't know what they are missing unless you put it up there.
• Mayor's Office: That will cost too much (e.g., scanning old documents). We have lots and lots of documents.
• City Council: By choosing what to provide and massaging the data, you are not letting people make better use of it.
• Mayor's Office: See the initiatives we just announced. We think that we are meeting customer needs (which we hear through surveys, complaints, etc.)
• City Council: You shouldn't decide what people want. Let them decide.

W3C's eGovernment Interest Group has been working with a growing number of agencies to gather information that will help address these sorts of concerns. Now they will develop best practices and guidelines for publishing government data. This is not an area I know well, so I look forward to being able to refer to the eGov IG's findings. However, I'm sure New York City is not the first government to wrestle with the technology, the cultural issues ("why should I publish my data?"), and how to use taxpayer money to do this.

When my turn came to speak, I said something like this:

• Thanks for using open standards.
• Use W3C Semantic Web Standards to publish data. As a starting point, I referred to Tim Berners-Lee's recent draft of Putting Government Data online
• Don't try to do everything at once. Start with what is already available electronically, for example.
• Don't require agencies to coordinate through a single portal. Let them publish data at their own speed. Then aggregate (through a single portal if you wish and if people find that easy to use).
• Participate in the eGovernment Interest Group.

I hope my summary here is backed up by the public record.