Tag Archives: qualitative research

Augmented ethnography: processing qualitative data from massive conversations

I am working on a project called Edgeryders, a massively collaborative exercise to reassess and redesign public policy towards youth. The idea is to get participants to share their experiences on policy-relevant topics, like how we make a living or how we participate in public life. As the project starts to pick up speed it does what crowdsourcing exercises do: it spews up a torrent of experiential data. In my book and elsewhere I have claimed that respectful conversations converge: a consensus is achieved, and we can just move on. I still think that’s true, and fairly obvious to the participants. The problem is how you convey it in a verifiable form to external observers – European governments and the European Commission in the case in question. Demanding that they go through even a small part of the raw data is simply not realistic. So what do we do?

My best guess is ethnography. Ethnographic methods are particularly well suited to this kind of investigation, because they are designed to embed the point of view of the people they study. For the same reason, I would argue that they are well suited to Internet ethics: we are not the lab rats, we are the lab itself, just as we are not the users, but the protagonists of our online meeting places. Modern ethnography employs software like Atlas.ti or its open source counterpart Weft QDA to annotate interview transcripts.

The advantage of social networks-based data collection methods like Edgeryders is twofold.

  • The data come already in written form. A major cost of ethnographic analysis, trascription of interviews (in 2006 they were talking about €100 per hour of recording), is therefore avoided.
  • Crucially, the data are really fragments of a conversation. Participants comment, contradict, praise each other. What might appear like different “interviews” (see this great example) are really linked to each other by a web of social ties, which are encoded into a database, and amenable to quantitative analysis.

In order to take advantage of these features, we are trying to develop a methodology that I have been calling augmented ethnography. It should work more or less like this:

  1. first, organize the material by participant. In Edgeryders, this means gathering all of my mission reports (a sort of blog posts), my user profile, and my comments, and filing them under my online identity. This produces a sort of uber-interview of a participant (me), that extends over several topics. Repeat over all participants. Annotate this material with Weft or Atlas.it.
  2. next, specify a network to represent the conversation. To a first approximation, I would start by considering the all participants are nodes of the same network. A link is created between participants Alice and Bob according to some interaction encoded in the database: the most intuitive one in Edgeryders is that Alice and Bob are connected if at least one of them has commented one of the other’s mission reports, or if both have commented a third participant’s. Edgeryders was intentionally designed with redundance in the relationship modes, so that different relationships could, in principle, evolve by exaptation to signify different things.
  3. crunch the network data and look for structure that could help you. One of the first things I would try is to compute measures of network centrality for each individual. This would help solve the classic ethnographers’s problem: the researcher arrives in some remote island to investigate a community whose culture he does not yet understand. A person sits with her and provides a lot of information. How can the researcher use it? That depends whether she has been talking to a respected member of that community or to the village idiot – and the researcher may have no easy way of knowing that. Network analysis can help: in respectful, fact-oriented conversation, the village idiot will almost never enjoy a high degree of centrality.

Of course, this is only a rough sketch. I don’t doubt that many clever tricks can be devised that will work on ethnography-on-a-database. Now, the problem is finding researchers that can take full advantage of that: competent ethnoghraphers that can take in the results of network analysys too. Readers: do you know any? Is there anyone interested in picking up this conversation, and maybe give my team and myself a hand?