Tag Archives: software

Zen and the art of website procurement: why bureaucrats should get their hands dirty with technology

In the past few years I worked for several public sector agencies. Much of my work consists of thinking up and delivering projects that happen mostly through Internet channels. This is a good time to take a step back and muse on what I learned. As always, the most valuable lessons come from mistakes made – so it’s a good thing I made a lot of them.

  • Software-as-service is a bad idea, though there are exceptions. My team and I made this mistake with Kublai, as we decided to deploy our platform on Ning. That allowed us to be up and running in half an hour, no small advantage; but we paid for it by sequestering our own database, procured and paid for by the Italian government, and handing it over to an American private company forever. A year later, Ning changed its CEO and business model: it moved its platform from the open source to the full copyright domain, disabled APIs and blocked migration tools. Just to do a network analysis, Ruggero Rossi had to write a web crawler – a bit like picking the lock to the door of our own home. It could have been worse: we were using a free service (that was before Ning rolled out pricing plans). If the company had simply shut down business, formatted the hard drives and walked away we could not have stopped them, since we were not in any contractual agreement. They would not even answer our emails. I am never going to even consider again rolling out a public sector project in which my agency does not have root access to the server hosting the database.

  • Using proprietary software is not a good idea either, again with some exceptions. It is expensive and it amounts to a open-ended commitment to your supplier. If a large software house develops custom software for you and then sells you the license, no one, except that same supplier, is ever going to be able to tweak that code. You risk finding yourself disempowered and stuck in a situation in which changing the color of the background or the font is expensive (as in billable hours expensive) and involves a lot of administrative friction. Furthermore, it is politically questionable: proprietary software is not reusable for free by other administrations, and that is not good – especially in a time of budget cuts and of (justified) skepticism vis-a-vis the effectiveness of administrations in spending taxpayer money.

  • That leaves free/open source software. I have been using WordPress in public sector projects since 2007; for the Edgeryders platform, more or less finished as of this week, my team ventured into Drupal. Working with open source software can be hard and frustrating. Features that are supposed to work simply installing a module or a plug-in turn out to have horrible bugs in practice; everything takes longer that you think; most of the work is not developing, but debugging. Meanwhile, the rest of the projects activities are stalled. It feels horrible. I think experience can mitigate the problem, but never really solve it. Free software is by definition organic and gritty: it works by hacks and duct tape as well as by elegant, rational solutions.

Despite all the problems, my experience of Drupal procurement is going to be positive in the end, as with WordPress before. The reason is this: these platforms allow, and even require, a hybrid figure of “power admin” to emerge, somebody who is less skilled than a developer but more so than a normal user. This happens because the admin interfaces of WordPress and Drupal are intuitive and very powerful; Drupal, especially, allows fine-grained control over your website. You can query the database, format the return of the query and send it to a page, a block or even an email message; you can tell the website to execute instructions of the kind IF [condition] THEN [action], not quite programming but on the border. Furthermore – and here I am thinking about my standing love affair with WordPress – when the admin interface is not enough, it is easy to find online resources and tutorials to get your hands into non-core parts of the code. I am technically incompetent, but still I have been able to teach myself to tweak the CSS in a blog’s style sheet, and even the PHP code for very simple tasks, like assigning different headers to different page or inserting a line of Javascript. That required a small-ish investment, to which the proliferation of “For Dummies” books in my library is testimony. This gives you an incredibly important freedom: that of developing in a quick-and-dirty fashion, launching, and then just keep tweaking as your project evolves. Trust me, you will feel the need from day one.

Here’s the trick: the hackerish power admin role is perfect for a public servant that needs to procure software. Getting to know the architecture of these platforms well and to take full advantage of their scope for customization does not make you developer, but it does mean being able to have a constructive conversation with your developers, get real on what can and can’t be done, how long it takes and how much it costs. Furthermore, a power admin can rethink her goals in terms of the software, and so come up with highly sophisticated terms of service for the procurement effort. For example, on Edgeryders we need to constantly reinvolve users in the conversation: this is done through email notifications and the recent activity feed. In Drupal, these functionalities are carried out by certain non-core modules. If the public servant knows this, she can procure not “a website that feels buzzing”, but “a website in which the activity stream module logs activities that are not logged out-of-the-box”, that is much clearer

When I got into motorcycle riding, I read the obligatory Zen and the Art of Motorcycle Maintenance. The lesson of that book is the following: the act of driving a motorbike is not really separable from that of doing its maintenance. “Romantic” bikers, who do not enjoy getting their hands dirty, don’t accept this, and delegate to professional mechanics even the simplest maintenance operation. But they pay the price of disempowerment, when their machines stop by the roadside and won’t get started again, and they don’t have a clue what’s wrong and how to fix it. This system failure can be disastrous in public policy: in the projects I manage technology typically accounts for less than 10% of the budget, yet if the technology is not there the entire project grinds to a halt.

Summing up, high quality procurement is impossible until you know what you are buying. In my experience the free/open source software community is up for sharing its knowledge; corporates producing proprietary software much less so. If, like me, you find yourself in the position of procuring a simple technological solution for the public sector, I recommend you turn to this community, arm yourself with patience and get your hands dirty with the technology the developers intend to use. Install and configure sandbox sites, add functionalities, tweak their look and feel. Spend time with hackers, show that you are eager to learn, an grill them with questions. Above all, don’t yield to the temptation of going “this is not my job, just make it work and send me the invoice”. It doesn’t work like that. This is very time consuming, but you will save that time, with interests, once you are in production. I know it’s not a perfect system, but it is still better than available alternatives. Truth be told, I think it would be really useful if somebody started a course of website procurement for public servants. Anybody out there is interested? I would certainly sign up.

Thanks Freddy Mascheretti, Ivan Vaghi, Paolo Mainardi and Claudio Beatrice for their patience and suggestions

Dragon training: computer-aided online community management

In my book Wikicrazia I claim that the public sector, society’s system to pursue the common good, can be made smarter by mobilizing the citizenry’s collective intelligence. Accessing collective intelligence entails enabling a large number of individuals to coordinate on some common goal. Normally, this is done by means of online commmunities, that use the Internet as their technological infrastructure and where interaction is mediated by some kind of social bargain, with somebody to resolve conflicts and keep the group focused on the goal.

There’s a problem here. On the one hand, online communities cannot be run by top-down command and control: it is exactly the free action of their different participants that make online communities so incredibly effective in processing large amounts of information. On the other hand, public policies have by definition a goal which is set exogenously with respect to the community itself: whereas Facebook users are on Facebook to hang out, and it does not really matter what they do with it, the users of Peer to Patent are there to process patent application; those of Kublai to write up creative business plans; those of Wikipedia (not a public policy, but similar in this respect) to write an encyclopedia. Community managers, myself included, are trapped in this dilemma: practically the only way we have to figure out the social dynamics in our communities is to spend an unreasonable amount of time participating in them, and we try to steer them by rhetoric and persuasion. We end up navigating pretty much by gut feelings. And as communities scale – even to just a few thousand participants – it gets really hard to understand what is really going on.

I thought our work would improve a lot if we could augment our ability to read social dynamics of online communities by using software. In essence, a policy community is a social network, and as such it can be represented by a graph with nodes and links, and studied mathematically. The community’s social dynamics should be encoded into the mathematical characteristics of the graph that represents it: for example, the creation of a cohesive group of senior users in Kublai in 2009 was picked up by the crystallization of a structure called k-core. If we managed to build some sort of dictionary that maps social dynamics onto mathematical characteristics of the graph, we could use network analysis to detect community dynamics that are invisible to the eye, because they happen at a scale too large for human participants: and this would work even for very large communities, at least in principle.

I intend to develop this software as my Ph.D. thesis. Colleagues at University of Alicante and the European Center of Living Technology will help. I call it Dragon Trainer, because doing policy through an online community is like training a dragon, an animal too large and dangerous to order around. If you are interested in learning how we plan to do this, you can watch the video above (12 mins).

Dragon training: gestione di comunità online assistita dal computer

Nel mio libro Wikicrazia sostengo che il settore pubblico, il pezzo della società deputato al perseguimento dell’interesse comune, possa essere reso più intelligente mobilitando l’intelligenza collettiva dei cittadini. Ricorrere all’intelligenza collettiva vuole dire abilitare un gran numero di individui a lavorare in modo coordinato su obiettivi comuni. Questo in genere si traduce in comunità online, che usano Internet come infrastruttura tecnologica e in cui si interagisce sulla base di un patto sociale e di qualcuno che media i conflitti e fa in modo che non si perda di vista l’obiettivo.

Qui però si pone un problema. Da una parte, le comunità online non si possono gestire con il comando top-down: è proprio l’azione libera delle tante persone che le compongono a produrre la loro straordinaria efficienza nell’elaborare grandi quantità di informazione. Dall’altra, le politiche pubbliche hanno per definizione una missione da compiere che viene dall’esterno della comunità che le attua: mentre gli utenti di Facebook sono su Facebook per stare insieme, e non importa quello che poi faranno usando quella piattaforma, quelli di Peer to Patent sono lì per valutare domande di brevetto; quelli di Kublai per elaborare progetti di impresa creativa; quelli di Wikipedia (non è una politica pubblica, ma ne ha alcune caratteristiche) per scrivere un enciclopedia. I community managers, me compreso, si dibattono in questo dilemma come possono: quasi l’unico modo che hanno per interpretare le dinamiche sociali delle loro comunità è passare una quantità spropositata di ore online, e cercano di influenzarle con la persuasione, l’esempio, la retorica. Ma si lavora molto a istinto, questo è chiaro. E quando le comunità diventano relativamente grandi — anche solo qualche migliaio di persone – è davvero difficile capire cosa sta succedendo.

Ho pensato che il nostro lavoro migliorerebbe molto se avessimo un software che accresce le nostre capacità di lettura delle dinamiche sociali online. In essenza, una comunità di policy è una rete sociale, e quindi può essere rappresentata con un grafo di nodi e link e studiata matematicamente. Le dinamiche sociali della comunità si dovrebbero riflettere sulle caratteristiche matematiche del grafo che la rappresenta: per esempio, la creazione di un gruppo coeso di utenti senior in Kublai nel 2009 veniva segnalata dalla formazione di una struttura che si chiama k-core. Se riusciamo a costruire una specie di vocabolario che traduca le dinamiche sociali in cambiamenti nelle caratteristiche matematiche del grafo, possiamo usare l’analisi di rete per individuare le dinamiche di comunità che a occhio nudo non si vedono, perché sono “troppo macro”: e questo funziona anche per comunità molto grandi, almeno in linea di principio.

Sviluppare questo software sarà il lavoro della mia tesi di Ph.D. Mi aiuteranno i colleghi dell’Università di Alicante e dell’European Center for Living Technology. Per ora si chiama Dragon Trainer, perché gestire una comunità online che deve svolgere un compito esogeno è come ammaestrare un drago: che è troppo grosso e pericoloso per essere costretto a fare quello che vuoi, e quindi va sedotto o convinto. Se ti interessa capire come sarà fatto, guarda il video qui sopra (12 minuti).I intend to develop this software as my Ph.D. thesis. Colleagues at University of Alicante and the European Center of Living Technology will help. I call it Dragon Trainer, because doing policy through an online community is like training a dragon, an animal too large and dangerous to order around. If you are interested in learning how we plan to do this, you can watch the video above (12 mins).