Open data: what if we used Wolfram Alpha?

In several countries governments and local authorities are starting to release their data for citizen reuse and remix. This is important in itself, but clearly these data will generat more impact the easier and intuitive using them is. I am seeing very different usability strategies being tried out.

At one end of the spectrum, some authorities go for “in-house” visualization. An example is the OECD with their eXplorer: it comes with a sophisticated visualization interface that supports animations, multilayer maps, Google Maps integration. The only problem is that the data themselves are locked inside, as are citizen-created visualization (“stories” in eXplorer parlance). Just about the pnly thing you can do is export them as XML files to share with your friends, but they need to go to the eXplorer website to see it. In general, the system is complicated and lacks flexibility; plus, it’s quite unfriendly for beginners (the instructions files is more than 30 pages long).

At the opposite end we find experience like the Italian State Accounting Service’s. The databases are downloadable, and there are instructions to generate tables summarizing them. Unfortunately, they are very specific, assuming as they do that all citizen use a specific function (pivot tables) of a specific software, proprietary and expensive on top of that (Microsoft Excel. It would probably have been more tasteful to refer the tutorial to Open Office). Except for expert users of Excel, this system is “all or nothing”: either you are looking at enormous, unmanageable disaggregated tables or you invest several hours to follow the tutorial and try a few ways to crunch the numbers so that they make sense. It’s ok for researchers, but it does not create an interest for citizens to play around with data.

Maybe a good middle ground is the World Bank’s strategy. World Databank enables the creation of simple reports (charts and maps included) directly on the site, and it also allows users to download the data in several formats. So a citizen can start by exploring directly on the website in less than a minute she can be looking at a simple chart. Then, if she so chooses, she can download the data and explore them more thoroughly with the software of her choice.

I think open data policies are just as meaningful as the community of citizens that know how to interpret, reuse and explain to others public data is wide. On this ground, I would strongly advocate to public authorities contemplating them to incorporate in their websites rapid preview features, as the World Bank did.

Developing those feature can be expensive (eXplorer certainly does not look cheap). A low cost possibility is probably to write a widget for Wolfram Alph, the computational engine of the man who gave us Mathematica. Its computational capacity is more than adequate to any reasonable open data use (it does some amazing stuff: try typing “compare a mouse and an elephant” as in the compute field): the problem is rather interfacing it with government databases. If that can be done, it becomes easy to write intuitive widgets like the one above – with the added benefit that the data then become accessible to anyone using Wolfram Alpha anywhere in the world, even if they are not even aware of the existence of this database in particular. More open data than that…

Wolfram Alpha could provide public authorities with a fast, cheap preview mode for citizens to develop a taste for open data; this would also increase the value of Wolfram Alpha itself, as the mass of data accessible to the engine is increased. It’s not for free, though: Wolfram sells subscriptions that cover a certain number of monthly queries. I wrote them to find out whether they have solutions for this kind of client, curious to see what they get back to me with. The real solution would be an open source version of Wolfram Alpha, but I am not aware of anything remotely like that.

6 thoughts on “Open data: what if we used Wolfram Alpha?

  1. Antenore

    Oh my God no!!! Wolfram is the worst example of openness. It’s enough to read their http://www.wolframalpha.com/termsofuse.html and their Privacy Policy or do some research. Wolfram wants to own all the data (not the index of the data, but the data itelf).

    Public domain data have to be owned by the public and not by a public company.

    Is this an advertisement of Wolfram (apologizes if I’m wrong)?

    Reply
    1. Alberto Post author

      Antenore: no, I am absolutely not advertising for anyone. I don’t even have adsense on my blog. 🙂

      Legalese is not my strong point, but in the terms of use I do not really recognize anything about ownership of the data. Anyway, as I said, I contacted them to ask the direct question: would like a public authority to give people access to their data through WA? Of course, said public authority cannot give data ownership away: data belong to the people. No response so far.

      What I would really like is for the European Union to make some infrastructural investment, creating the equivalent of Ning, Second Life, WA e what-have-you and making them free and open source. This would abate dramatically ICT costs for the public sector in Europe, and we would not have to depend on American companies and their TOS.

      Reply
  2. Stefano Durì

    anche secondo me Wolfram Alpha rischia di essere un buco nero. Vediamo cosa ti rispondono. Per quanto riguarda la possibilità di visualizzazione di set di dati, ti consiglio di farti un giro (anche se probabilmente li conosci già) sui siti di cui si parla qui http://infosthetics.com/archives/2010/06/review_big_three_online_viz_tool_benchmark.html
    Particolarmente interessante è Tableau Public, che ti consente (nella versione free) di creare ed editare localmente grafici basati su tuoi set di dati con un pacchetto scaricabile dal sito e caricarli sul tuo spazio personale, per poi condividerli e embeddarli. A titolo di esempio, ho creato un grafico basato sui dati della spesa pubblica (fonte: Ragioneria Generale dello Stato). Il risultato è bruttino, ma solo per fretta e scarsa perizia. I dati rimangono pubblici, nel senso che possono essere scaricati (v. toolbar in basso nella pagina)
    http://public.tableausoftware.com/views/Spesapubblica/Sheet1?:embed=yes&:toolbar=yes

    Reply
    1. Alberto Post author

      David, vero? È un mio pallino da tempo: uno stack di soluzioni open source per l’informatica “semplice” pubblica. Una specie di Ning, una specie di WA, una specie di Second Life… 🙂

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.