Published using Google Docs
Maryland Council on Open Data - testimony Sept 3
Updated automatically every 5 minutes

Thank you members of the committee for allowing me to speak today.

I started SpotCrime 8 years ago shortly after moving to Baltimore.  As a transplant, I was aware of the crime reading the papers, but not aware of their locations because I was unfamiliar with the city.

Since then, Spotcrime has grown to the most visited public facing crime mapping website in the world.  We are arguably the largest crime alerting system in the US by providing over 140 million email alerts annually.

I’ve been fortunate to accomplish this feat for two main reasons.  The move towards transparency by many municipalities and police agencies around the country, and the drastic reduction in computing costs and hosting over the last 10 years.

Our journey has not been without headwinds, and SpotCrime is only able to access a fraction of the public crime data in the US.

As a state, Maryland’s crime data transparency ranks high in relation to many other states.  Recent open data legislation has been instrumental in increasing access.  

What I’d like to do for the next few minutes is share a success and some troubles we are having.  I’d also like to explain how I see procurement and vendor control to be key issues in creating open data success in the future.  And how full agency participation in an open data standard could improve internal quality of policing and reporting.

As provided in the presentation, we’ve listed all the agencies we have been able to identify that provide some type of open data and their frequency.

One problem, as you see from the spreadsheet, each agency uses different technologies and different approaches to what is released and when.

One success as result of the recent legislation is that we are now receiving data again from Anne Arundel County.   After a two year hiatus of providing data, they were very responsive to our request earlier this year and now provide data in a machine readable format on a weekly basis.  I think it is important to note that when the data went down, SpotCrime reached out almost quarterly to check up on the data feed.   It wasn’t until the legislation had passed that were were able to get data.  

Unfortunately, we were not so successful with the City of Frederick.  Each request has been met with a different response, and quality of response has declined with each requests.  And finally, charges were added to each subsequent request.

Specifically, we were first provided with an excel file, then a pdf and then a pdf that was spread over two pages.  When we inquired about the changes, the response was that “we do not want (to allow) the data manipulated in a excel spreadsheet”.  

But my making the data more difficult to process, the time and error rate goes up.  The chance the data is incorrectly manipulated goes up, not down.

This is doubly confounding because the crime mapping vendor the city has employed for years sells the data in spreadsheet format to anyone who pays.   However, local media and press are restricted.  So, you have a marketing companies around the country getting full access and the Frederick News Post getting limited or no access.

For the City of Frederick, the quality of responses and data have only partially improved since the new legislation.  We are however getting data.  It is at a snail's pace and increasingly complex format.  We don’t knows what new format will be invented on our next request.

In general, we’ve found over the years that many police agencies end up with records systems that don’t have basic functionality of simple reporting.  Recently, we reached out to the city of Denton, TX.  They were very willing to provide a public data feed, but upon inquiry found that their vendor had access locked up. This particular vendor has a billion dollar enterprise selling risk management solutions to industry and has an incentive to make the data less public and their product more valuable.

In some cases the vendors offer free software in exchange for control over the data. The vendor gets to limit access to the public and the press while selling enhanced products to their commercial clients.  What seemed like a good deal for the agency actually increases long term costs and harms the public by restricting access to vital crime data.

Thankfully, we are seeing sophisticated larger agencies avoid this pitfall by retaining control over their data.  If you look across the US, the less technically competent agencies end up taking the free software deal without factoring the public costs of limited access.   For the most part, these agencies are smaller and more rural.

From a State perspective, we’d like to see standardization of the data for public consumption.   I believe that this will influence internal reporting through quality and speed for the whole state.   I understand that State Police have done a great job in coordinating the state agencies data, however, if we can get the public to get near real time, high quality crime data, then it is likely the State will better tactical intelligence for fighting crime.

By improving public facing crime data, we will likely have the positive unintended consequence of better internal reporting.

Each agency does not need to buy the same database product, but at a minimum, they should control their public feed and have a standard output.

My suggestion for future open data success is to incentivize software and database procurement that provides simple export and reporting features for public consumption.

We suggest that all future GOCCP grants for technology with databases require the technology to meet the State’s open access standards.