Category Archives: Open government

Setting the agenda: how the open data community entered the radar of European political leadership

On July 17th 2014, addressing the Open Knowledge Fest crowd in Berlin, EU Commissioner Neelie Kroes made the following statement:

We want to work with you, and see you work together across borders and languages. We have set up Erasmus for Open Data to support this. Starting with an event in Nantes, France, in September. But if you have an idea for what more we could do – then let us know! – source

I normally don’t pay too much attention to announcements, but hey: this was fast. To the best of my knowledge, the idea of an Erasmus for open data did not even exist before April this year. For a concept to go from first-time appearance in a private blog to candidate policy of the first economy in the world in three months is unprecedented. Just what is going on?

A short recap:

Back in April, in the wake of the Spaghetti Open Data gathering, I wrote a post that argued for a proposal for an Erasmus-like program for open data. The idea was to build those all-important horizontal ties that can connect today’s open data communities, largely national, into a European-level one. The post sparked a small debate with some of my fellow activists in other European countries, notably some that are or have been involved with EPSI (the European initiative on public sector information), like Ton Zijlstra in the Netherlands and Martìn Alvarez in Spain.
In early July I got an email from French NGO LiberTIC: they have gotten the city of Nantes to get behind a conference seen as the launchpad of a future Erasmus Open Data initiative. EPSI platform is fully involved. The European Commission is going to show up, probably represented by EPSI platform’s project officer. To show they mean business, LiberTIC have even allocated a small budget for funding at least some data geeks to fly to Nantes. With the usual generosity, Spaghetti Open Data are rising up to the challenge – Italy will be well represented in Nantes, I can promise you that.
Now, the Commissioner has joined the front line. That she even knows about this means somebody in DG CNECT (the EPSI team?) must have done a really good job of getting the idea up through the hierarchy.

We might get our Erasmus for open data. Or not. Whatever the outcome, one thing is clear to me: if a random guy like me can spawn an idea on his personal blog in April and hear a European Commissioner throwing her full support behind it in July, it means the open data community is setting the agenda. We are on the ground, we are doing interesting things with data, and everybody acknowledges that. We are talking to government agencies who are supposed to set the standards and write the guidelines, and at times doing quite a bit of the work for them – and who else can those guys talk to? We can mobilize quickly and effectively – just look at how LiberTIC threw together an international conference in two months. And everyone knows you just can’t do open data without a strong open data grassroots community.

So Europe is listening; and the signs are there that the Italian state, amidst the usual drama, handwaving and short-termism, is listening too. It would be a shame to waste this opportunity to build some data-powered transparency and knowledge sharing into our societies. But I think the community is ready, and the opportunity will not be wasted.

Meanwhile, Ms. Kroes, thanks for your support. There is a small factual mistake in that part of your speech (“we have set up…”): Erasmus open data, as of now, is a community initiative, not a EU one. No big deal; we are not territorial, now the idea is out for everyone to improve on. You are welcome to Nantes, just like any other data geek, within or without your official capacity. If you come as a private person, drop us a line: there will be code to write, and datasets to cleanup, and pizza and war stories about data to trade. We’ll sit down together and scrape some EU website. It will be fun.

Join us in Nantes for the Erasmus Open Data conference

You did WHAT? The Italian revenue agency infringes OpenStreetMap’s copyright

2 Replies

A couple of months ago, Simone Cortesi, deputy president of Wikimedia Italia and the primus inter mappers of Italy’s geohackers, noticed an oddity in the maps of the Revenue Agency’s property market dataset. How could they know about the walkways in his own garden? He realized he himself had uploaded those data, not into any government dataset but into the “wikipedia of maps”, OpenStreetMap. Since the maps did not credit OSM as the data source, the Revenue Agency was technically infringing on OSM’s intellectual property rights. OSM maps are free to use for all, but if you do use them you must respect the terms of the Open Database License protecting the data. If Simone’s allegations proved to be correct, this would be the largest ever copyright infringement against OpenStreetMap. And done by the tax authority of a G8 country, no less.

A group of Italian expert contributors to OSM coded a website exposing the problem and containing a tool for comparing the Revenue Agency’s “proprietary” maps with OpenStreetMap. Hundreds of eyeballs were put on the case, and sure enough, the data are the same, and the copyright infringement was there.

On July 8th 2014, after the Italian Twittersphere had put the word out, the Revenue Agency tweeted back that it had “demanded an explanation” from its technology provider, a company called Sogei. This is an in-house company, 100% owned by the Ministry of Treasury. Later in the day, Sogei complied with the terms of the OpenStreetMap license and issued a statement of apologies. With this, the generous Italian mappers declared themselves vindicated. Simone, bless him, rose to the occasion to demand the Agency opens up its own data, specifically those of the real estate registrar, as he and many of us in the Italian open data community have been advocating for years.

Over and above the embarassment, there is a deeper lesson to learn here. Sogei is a monopolist: the Revenue Agency had no choice but to get its tech from them. Sogei, in turn, ostensibly acquired their geodata from a company called Navteq, (source, in Italian), owned by Nokia (wikipedia), that appears since to have changed its name into Here.

So what happened, really? Did Navteq repackage free and open data and sell them as proprietary to Sogei, who resold them back to the Italian state? How much money was spent on this procurement process? Was there financial damage to the public purse, and was it intentional (hence an offence)? How much money could we have saved, and keep saving, if smart communities like the OSM, open source and open data communities were involved in public procurement?

Open data comes of age

Leave a reply

If you live in Italy and are curious about your local authority’s pattern of spending and taxing, you are in luck. Since last week, OpenBilanci publishes on the web detailed financial data from all the 8,092 Italian local authorities for the past ten years. Both budgets and closed ex post accounts are available, along with a galore of indicators like financial autonomy or spending velocity. Not only are all data downloadable and open: OpenBilanci sports a nifty web interface for preliminary data exploration. The latter is a feature found also on other highly successful Italian open data projects like the mighty OpenCoesione, that released spending data on 749,112 projects funded by the country’s cohesion policy. And no surprise: though OpenCoesione is a government initiative and OpenBilanci is not-for-profit one, the same team of visionary coders stand behind both projects, through both a non profit and a for profit arm.

In the space of only a few years, open data have become a formidable force for openness, transparency and even data literacy in a country that badly needs all three. Forward-thinking civil servants and political leaders in some of Italy’s 20 regions (and some cities) have been working together with civic hackers for years now: Lazio has funded OpenBilanci through its SME-centred innovation policy, whereas Emilia Romagna has successfully built a partnership with the largest Italian open data community, Spaghetti Open Data. In a veritable stroke of genius, the city of Matera has decided to host on its own open data portal any open dataset produced by the local community.

When public authorities do not play ball, Italian civic hackers simply proceed to open up government data anyway. One of my favourite projects in this sense is Confiscati bene, started during an epic Spaghetti Open Data hackathon. The group wrote a crawler to extract data from the (non-open) website of ANBSC, a government agency tasked with reallocating assets confiscated to Mafia bosses and other assorted mobsters (the Italian police is doing a sterling job there, since ANBSC is juggling over 11,000 such assets). It cleaned them up, geocoded them, made them downloadable, built the customary sleek interface for web exploration, embedded them into a brand new website and released everything as a gift to ANBSC. OpenBilanci itself entailed scraping over two million web pages.

I know Italy’s scene best, but exciting open data projects are appearing everywhere. My absolute favourite one is British: OpenCorporates gathers data on over 60 million corporations all over the planet. Using unique identifiers and information on ownership structure, OpenCorporates shines a light on the corporate world, that has far less tight legal requirements on transparency than government. This OpenCorporates-based visualization, for example, will teach you much about Goldman Sachs.

It looks like the open data movement has come of age. It was surprisingly fast: in less than four years we went from a small cadre of nerds obsessing on Tim Berners-Lee famous “raw data now” speech to a strong community (there are almost 1,000 subscribers to the Spaghetti Open Data mailing list, churning out twenty messages a day 365 days a year) and a phalanx of young decision makers that understand the issue and are plugged into the community. I am proud of you all, my sisters and brothers in arms. And the best is yet to come – especially as we come together all across Europe, as I am sure we will soon since the times are ripe for this to happen. Who knows, data culture might even be able to shift European politics away from populism and onto evidence-based debate.