How should the success of open data be measured?


#1

What should be measured to determine if an open data policy is successful?

How should these results be shared?


#2

I hear regular stories about Government’s own internal use of shared and open data to create new ways of doing things - which is resulting in anecdotal stories of cost savings and efficiency gains. As a minimum, it would be excellent to see those stories become better communicated internally and shared, so as to inspire others. Ultimately, it would be even better to endeavour to quantify the actual cost savings and efficiency gains and promote those outcomes.


#3

It’s all about the purpose, or why data is made open.
One possible purpose is to help businesses deliver their services. In such cases, I agree with maree that having open data users share their stories is already a potential good measure of success.
For many businesses, open data would be just one of many factors contributing to their success. Often very hard to quantify. It could be a bit like trying to ask a business what was the impact of new street lighting in front of their office on their operations. At best, they would say: “thanks to it our customers visit us more often”, but it could be tricky to quantify the change.

But there are other purposes for releasing data:

  • transparency (success measure: increased trust in government)
  • promoting entrepreneurship (success measure: number of startups that exist because they could use open data to create new products)
  • education (children using open data to learn more about the world)
  • research (new findings resulting from data that was previously not available to scientists)
  • safety (data used to inform citizens about potential risks/threats etc.)
  • wealth of citizens (citizens using open data to make better informed financial decisions)
  • etc. etc. etc.
    And for each of those, the measures would be different.

#4

My day job is to help clients with their data management.

We suggest a KPI model that measures user interactions with the system (e.g. downloads of open datasets) and with the open data team (e.g. questions to publish further data, data quality issues), engagement data (e.g. surveying the ease of use, the ability so solve innovative questions) and system information (uptime, timeliness) just to name a few.

Overall the KPI model needs to cover people, process and technology.

Open data can be measured just the same.


#5

Agree with Marek recommendations, specially the need to focus on the purpose. And there are some good examples on some internationally leading open data initiatives (check data.gov) that go with the list you made (also on theodi.org).

I would like also to highlight an important point:

Stories should be made keeping in mind we communicate with a wide range of audience: the public, businesses, academia etc… so not only the “open data community” . This way stories can reach further ans also encourage more people & organizations to use open data and create further success stories.


#6