Lections from Egypt: moving on from prediction to early warning

Daniel Kaufmann had some fun compiling a list of the authoritative commentators that predicted that – unilke in Tunisia – in Egypt the disgruntled population would not take to the street, or anyway not in such a way to threaten the regime. Everybody seems to have fallen for it, from Foreign Policy to the BBC, from Time magazine to the Economist.

Forecasting was always tricky business, and is getting more so. In a society as complex as ours, even the best analyst are lousy at prediction. In an entirely different context, David Lane and others (yours truly included) are suggesting that in some cases prediction might be replaced by a system of early warning, that spots emergent social dynamics in its early stages, when correction is still possible. This would be done by combining and filtering large masses of data, many of which collected on the web. The idea — which might ring familiar to those who use the Internet as a social filtering device for information, is that the global conversation is an entity that exists at a level superior to ours, and as such might know things that none of us, mere participants, know.

To describe this hypothetical system, David likes to quote post-marketing surveillance on pharmaceuticals after the Thalidomide scandal. This drug used to be prescribed to pregnant women in the 1950s, and it could induce terrible deformities in newborn children, but only combining in exquisitely nonlinear ways with other agents: it was cleared for rollout because the lab tests did not allow to discover the problem. It was the doctors treating the mothers of deformed or sick children that discovered, in the ocean of statistical noise, the weak signal of taking Thalidomide during pregnancy. As a consequence of this story, pharmaceutical companies now work with physicians to spot correlations to weak to be spotted in the lab, but that might be revealed by processing the mass of data obtaiend by tapping all doctors.

It is a fascinating topic, at least for me. And — going back to Egypt — it leads to an unexpected conclusion: it suggests another way that Wikileaks might be a good thing. Laks feed the global conversation, and thereby increase the probability that bloggers, citizens and activists poolf their knowledge and discover emergent trends. It has been argued that Wikileaks is nefarious, because it might hinder the work of diplomacy: but without better analysis, diplomacy cannot do an acceptable job anyway.

Share this:

4 thoughts on “Lections from Egypt: moving on from prediction to early warning

  1. Freddy

    Spero non sia un commento troppo OT…Qui Google mette a disposizione grafici di richiesta connessioni ai suoi servizi filtrati per Paese. E’ interessante guardare Egitto e Libia di questi giorni. Inoltre vi sono dati relativi alle richieste da parte degli stati di rimozione contenuti. Interessante.

  2. giacomo

    Some stories you might find interesting:
    There’s a venture I find particularly fascinating that works like a crystal sphere, you can sign up for 150$/month, I didn’t try it yet, but still…it’s already out and running…

    Also, related to this theme: at the end of this month there’ll be a, quite creepy, workshop promoted by DARPA to kickoff a new line of research on how storytelling can be used to change shape political unrests (more or less like the 1984 thought police)

  3. Alberto Post author

    @Freddy: anzi, è molto interessante! Considerando anche un Googler è uno dei leaders della protesta e che si fa il suo nome per le presidenziali… 🙂

    @Giacomo: thanks for the tips. Recorded Future is prediction markets. It’s a cool tool, but quite well known. The DARPA link is superinteresting!


Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.