Observing the Web as a way of making sense of a world of “wicked” problems


(Image courtesy of Thanassis Tiropanis in his “Web Observatory” presentations).

In a previous post I talked about Socrates and notion of the “examined life”. Countless words have been written on human attempts to examine the psyche, and the challenge of more effectively understanding how we can live together more harmoniously for the betterment of all.

This is what is at the heart of most governments, and during the ANZSOG Master Class earlier this year I asked John Alford what he felt was the greatest challenge which currently faces public sector managers. He felt that it was need to simultaneously manage the demands of “business as usual” with the complexity presented by the so-called “wicked problems”.

A “wicked problem” is one that “resists resolution”, and, all too often, through its own solution there emerge further “wicked problems”. In discussing this with other colleagues an interesting distinction emerged: the difference between problems that are “complex” and those that are “complicated”. When it comes to dealing with “wicked problems” many are complicated but not complex. For example the NASA Space programme. Whenever problems arose during the race to the Moon the Director asked his team three questions:

  1. What is the problem?
  2. Do we still want to do this?
  3. Get on and solve the problem.

Much of what was required was to uncomplicate the seemingly complicated, to simplify and retain the essence without getting overwhelmed, and the results speak for themselves. Something that is complicated can seem to be complex, but, when it is analysed in terms of systems and processes it can be simplified and solutions can be found.

The challenges of twenty first century government and governance are both complicated and complex and, as The Economist outlined in a recent article, globalization and digital technologies are making many of the traditional systems and processes seem outdated, adding another layer of complexity. In order to even begin to deal with these issues we first need to be able to observe what is going on, and to then apply tools to those observations in order to make sense of them.

This is at the core of the Web Science “Web Observatory” initiative.

A “Web Observatory” is, in essence, a “Social Machine to observe Social Machines” (see an Overview paper here). If we consider the world of astronomy, then what the physicists do is utilse a range of telescopes (such as the “Square Kilometer Array”) to focus on a different part of the sky. From these divergent observations they construct a picture of the universe.  The Web Observatory (or “Web of Observatories”) consists of a range of Web “telescopes” which focus on particular parts of the Web (be they certain Social Machines, such as Twitter or Wikipedia or others which utilise Open Data such as maps or transport applications).


Given the vast amount of open data that is now available we are not short of telescopes, but the trick is to make sense of it, and that requires a range of tools to both visualise and analyse what is going on.

Making sense is not only the purview of scientists however; it is absolutely at the core of twenty first century policy development, and we are now contributing to this with a new ANZSOG research project “Governance in the age of Social Machines: Web Informed policy making” (an overview of which can be found as both a Overview Paper and Presentation).

This project is being undertaken in partnership with the Web Science Trust, University of Southampton, University of South Australia and Government of South Australia, and our objectives are:

  1. To develop the data publishing and governance structures which enable the SA Government to publish its data on the Observatory;
  2. To develop a methodology to utilise that data to inform policy making, and
  3. To develop cases which underpin a “digital literacy” education programme to be developed by ANZSOG together with the SA Government for delivery to other jurisdictions.

The project will address three key research questions:

  1. How do we build a “Social Machine” to better observe the workings of government?
  2. How can this Government Web Observatory better inform the creation of public policy?
  3. What are some of the key challenges, which governments will face as a result of being armed with a Web Observatory?

It is both exciting and refreshing to see organisations such as ANZSOG and others supporting initiatives such as this which focus on the core of our digital literacy and competencies. The reality is that we can have all the data in the world, but if we cannot learn to “read” it – and from there turn it into information and knowledge – then we are merely flailing about in the dark.

The UK has recognised this with it’s “teaching coding” in schools, and The House of Lords has just called for evidence for its Committee on Digital Skills stating that its goal is to examine “the digital capability of the nation”.

In his recent presentation at the National Archives “Information:  The Currency of the Digital Economy” Conference in Canberra, Minister for Communications Malcolm Turnbull stated that

“In reality data is not an ice-cold world of algorithms and automatons. There is an essential role for people, with all the limitations, inaccuracies, subjectivity and humanity that come with being human … It is now the responsibly of us all, government, industry and the community more broadly to unleash the latent power of today’s modern information for a common social and economic good.”

I couldn’t agree more. We need to learn to design our world for “data” as much as we need to design for humans, because with the emergence of Social Machines humans and data are becoming inextricably bound.