A Design Epistemology for Studying the Social Impact of New Technologies

This is the text of a proposal I was working on a few years ago. I thought it might be of interest to others, and that I may get some useful feedback. I'm releasing this under a creative commons Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.

New technologies offer endless opportunities for empowering individuals and communities through access to knowledge, means of expression, and coordinated action. Mobile and social technologies are penetrating every aspect of our life; personal, social and professional. Every day we witness new initiatives to leverage the potential of such technologies in education, government and civic society. In her recent speech, Secretary of State Clinton highlighted the important role these technologies play in promoting civil rights and good governance, and in supporting humanitarian and civic actionⁱ. Yet many of these initiatives fail, and those which succeed are often hard to replicate. The proposed research aims to develop an epistemological and methodological infrastructure for the study of the social impact of technology, apparent as well as potential, in a manner that will promote robust and sustainable innovation.

My previous research explored a similar theme in the context of technology-enhanced mathematics education, driven by a perception of education as institutionalised social action, directed at empowering individuals and communities. Yet I see the consequences of that work as applicable to a much broader domain, as illustrated by the following examples.

In the wake of the 2008 post-election violence in Kenya, kenyanpundit.com emerged as a vital information hub. What began as the personal blog of Ms. Ory Okolloh, a young activist and lawyer, filled a gap left by elusive government sources and self-censored mainstream media. Okolloh harnessed her social and professional networks, asking peers to send her reports of incidents across the country. Before long the flood of information became overwhelming, and Okolloh asked her network for help in building a dedicated site for aggregating, processing and displaying citizen reports. A few scores of volunteers responded to the call, and within a week Ushahidi ('testimony' in Kiswahili) was launched (Okolloh, 2009; Makinen and Kuira, 2008). Ushahidi is a web-platform which allows people in a crisis affected area to file reports via SMS, email, and web. These reports can be validated, aggregated, and listed by category or displayed on a map. Consequently, they can be used to raise awareness, assess the situation and direct responses. Since 2008 Ushahidi has been used to monitor the elections in India, track xenophobia in South Africa, visualise the effects of the war in Gaza and most recently to coordinate relief efforts in Haiti. However, the picture is not always so rosy. Mobile phones and social media (Twitter, YouTube, and FaceBook) played a significant national and international role during the events following the 2009 post-election crisis in Iran. But while these tools helped to organise protests and rally international support, they were also used by the regime to oppress, hunt down and in some cases kill dissidents (Burns & Eltham, 2009).

These are two extreme examples of how mobile and social technologies are creating new opportunities as well as new risks for individuals, communities and institutions. Yet the effects of such technologies are deep and wide-ranging. An estimated four Billion mobile phones were in use in 2008ⁱⁱ - roughly three for every five members of the world population. Social networking sites boasted close to 600 Million users in 2008, over half of all internet users and an annual growth rate of 25%ⁱⁱⁱ. Mobile phone penetration has been shown to be strongly correlated with economic growth, most significantly in developing countries (Waverman, Meschi & Fuss, 2003). A recent UN report identifies important roles for mobile and social technologies in the context of large-scale emergencies, from prevention and early warning to rebuilding (Coyle & Meier, 2009).

Coming back to education, where research is making genuine efforts to keep abreast of technological developments, the rapid pace of technological innovation and deployment poses a challenge to systematic analysis. The fluidity afforded by these technologies, which allow their users to blend school, family and leisure contexts, blurs the boundaries between education and other forms of social action. Several reports, such as Childnet International (2008) survey the technological landscape and consider the potentials and risks for young people. Special issues of journals (e.g., Selwyn & Grant, 2009), provide a multi-faceted view on key issues. Most of the publications include either an analytical framework or a set of anecdotes, or both. Sharples et al (2009) propose a theory of mobile learning which highlights context and sees learning as occurring through exploration and conversation. They then review a selection of innovative exemplars. Luckin et al propose the notion of ecology of resources as an organising principle, and explore a range of cases in its light.

The dichotomy between high theory and anecdotes creates a critical gap from a pragmatic viewpoint. As Kelly et al (2008, p. 3) argue, "a central question for educational research is how to design interventions that move beyond 'what is?' or confirming 'what works?' to designing 'what strategy or intervention might work better?'" This argument reflects a design science perspective, which has gathered momentum in educational research over the last few decades (Mor and Winters, 2007). Steming from the seminal work of Herbert Simon (1969), design science takes an expansive view: "everyone designs who devises courses of action aimed at changing existing situations into desired ones" (Simon, 1969, p 129). It is an inherently interdisciplinary paradigm, rooted in a pragmatic stance. Asking what works better links the value of scientific knowledge to the action it engenders. In order to answer such a question, design science needs to refer to the social science for description of what is, consult the humanities to define what is desirable, and harness the products of the natural sciences to get from one to the other.

Design science seeks a rigorous discussion of the complexities of human actions and their products. This discussion is simultaneously prescriptive and analytical: it seeks both to understand and to guide the subjects of its enquiry. Such an investigation is radically different than that of the natural world, not least because we cannot detach ourselves from the object of our study. It implies a value-laden scientific agenda, a change of methods and awareness of subjective issues such as representation.

The last decade has witnessed a growing trend towards design based research in education and, in particular, on the use of technology in education (Mor & Winters, 2007). Design based approaches focus on the process of developing innovative tools and activities as means of understanding the dynamics of learning and advancing educational practice. As a young discipline, it draws on multiple roots and is evolving simultaneously in multiple locations. Nevertheless, some shared practices and pockets of expertise are emerging. Among the common methodological characteristics are a dual focus on practical and theoretical contributions, a highly interventionist and agile attitude, and a cycle of iterative research. This cycle includes phases of theory, design, implementation, execution (experiment / practice), articulation of experience, interpretation, evaluation and analysis, and feedback to both theory and design. The products of this cycle are validation or critique of existing theory, evidence regarding the effectiveness of artefacts and practices in well-defined settings, and innovations in practice and theory. A frequent by-product of research is the synthesis of multiple frameworks.

Design science offers a sophisticated response to complex challenges, but sophistication makes it hard to communicate. This difficulty is amplified by the fact that as a young discipline it has not yet established consensual methodological frameworks, as noted by many leaders in the field (Bell, 2004; Middleton at al., 2008; Bannan-Ritland and Baek 2008; Cobb and Gravemeijer, 2008; Kelly, 2004). Such frameworks need to be based on a coherent epistemic infrastructure: a set of rules and assumptions which bound the discourse of a scientific community, and a logical system by which claims are presented and justified.

The primary aim of the proposed research is to identify a candidate epistemic infrastructure for design research in the social effects of mobile and social technologies. Such an infrastructure should ensure that arguments made by researchers are readable by the scientific community, in both the immediate and the broader scope of neighbouring fields, as well as practitioners and policy makers. All these parties should be able to judge the validity of claims and interpret and utilize the results to their needs. It needs to specify the full cycle of a design study – from theory to conjecture through experience and back to theory – so that it is observable by an external reviewer. The forms it offers for communicating design research should allow for the articulation of all that is needed to support the above requirements. They need to be able to capture process and product, connecting personal experience and generic abstractions. The mechanisms used to organise and communicate knowledge need to be aligned with the nature of this knowledge. Given the pragmatist foundations of design science, the research community needs means for organising this knowledge accordingly. For example, indexing findings by the problems they solve rather than by the means they use, by the conditions under which they are relevant more than by their academic heritage. Finally, the forms of presenting claims and arguments need to afford easy aggregation of knowledge, building new results on the basis of prior art. At the same time, this demand needs to accomodate individual and local voice, creativity and the uniqueness of any given human situation.

Several forms have been suggested for capturing abstractions of design knowledge in educational research, among them design principles (Kali, Spitulnik and Linn, 2004; Kali and Ronen, 2005; Kali, 2008; 2005; Kali, Levin-Peled and Dori, 2009), scripts (Miao et al., 2005; Kobbe et at., 2007), design narratives (Bell, Hoadley and Linn, 2004; Hoadley 2002; Barab et al, 2008), design patterns (Mor & Winters, 2007; Retalis et al, 2006; Haberman, 2006; Derntl and Motschnig-Pitrik, 2005; Goodyear et al. 2004) and sequences (Dalziel, 2006). McAndrew, Goodyear and Dalziel (2006) compare a few of these. These and other forms will be reviewed and mapped into a clear taxonomy. Two of these forms – design narratives and design patterns will be inspected in detail.

Design Narratives provide an account of the history and evolution of a design over time, including the research context, the tools and activities designed, and the results of users’ interactions with these. Narrative is a fundamental mechanism we use to make sense of our experiences (Bruner and Lucariello, 1989; Bruner, 1991). Loosely defined, a narrative recounts something happening to someone under some circumstances. It is a sequence of events with temporal, and implied casual links. Design narratives are accounts of critical events from a personal, phenomenological perspective. They focus on design in the sense of problem solving, describing a problem in the chosen domain, the actions taken to resolve it and their unfolding effects. . They portray the complete path leading to an innovation, not just its final form – including failed attempts and the modifications they provoked. Narrative, notes Hoadley (2002:454), “is only one way of making sense of design-based research” but “to really convey what happened, though, requires a story.” They provide a “thick description” of the design experiment, allowing critics to assess the validity of the researchers’ claims, and trace them back to evidence. At the same time, design narratives provide sufficient contextual information for those who wish to conduct a similar experiment in proximal settings, be they fellow researchers or practitioners wishing to apply the research findings.

Design patterns (Alexander et al, 1977) were developed as a form of design language within architecture. This was done with the explicit aim of articulating tacit knowledge to allow the accumulation and of solutions and to allow all members of a community or design group to participate in discussions relating to design. These patterns were organized into coherent systems called pattern languages where patterns are related to each other. The use of design patterns never achieved a large following among professional architects, but the idea has been embraced in several other disciplines, starting with software engineering, following Gamma et al.'s (1995) seminal book. Design patterns have been appropriated for education more than a decade ago (Eckstein, Manns and Voelter, 2001).

The quest for an epistemological infrastructure touches upon questions of philosophy of science and epistemology, and these disciplines too will therefore be consulted. Some current debates in these fields would appear to be relevant, for example the question of the relationship between knowledge-that and knowledge-how (Ryle, 1949; Stanley and Williamson, 2001), the relation between knowledge and actions (Hawthorne and Stanley, 2008), the dependence of knowledge on the practical consequences for the knower (Stanley, 2007) and the value dimension of scientific enquiry (Kitcher, 2001). The core of a design pattern can be seen as a local functional statement: “for problem P, under circumstances C, solution S has been known to work”. Such a structure reads like a direct generalisation of the form of a design narrative. By forefronting the problem, the structure of a design pattern acknowledges the functional axis of decomposition and the value dimension. These features are further expressed in the links between patterns, inherent to the pattern format. The design patterns approach is sensitive to complexity and context-dependence, and reflects them by restricting solution statements to compact classes of problems in clearly delineated contexts.

An epistemological infrastructure needs to be operationalised by domain-specific methodological frameworks. Whereas the epistemological elements are generic to the design research discipline, their methodological implementation is context dependent.

The secondary aim of the proposed research is to explore the methodological implications of the proposed epistemological infrastructure through a series of small and medium scale design studies. These studies, ranging from several weeks to several years each, will examine the current and potential effects of mobile and social technologies in education, public health, development, and civic action. Each study will define a clear and well-contained goal, but together they will be committed to a “triple-bottom line”: to expand the underlying theories, to produce practical guidelines for design and action in the respective domains, and to contribute towards a repertoire of methodological frameworks.

Such studies will build on and extend past and current initiatives I am involved in. These interleave three streams of activity: learner-centred design of games for learning mathematics; participatory design of mobile and social technologies for development and civic action; collaborative reflection workshops for educational practitioners.

Alongside developing the theoretical arguments regarding the foundations for a design enquiry of the social impact of new technologies and exploring the practical implications of these arguments, I will strive to disseminate these ideas through a series of public appearances and publications. These will include journal papers, edited books and the initiation of national and international symposia.

i http://www.state.gov/secretary/rm/2010/01/135519.htm

ii CIA world factbook, https://www.cia.gov/library/publications/the-world-factbook/geos/xx.html

iii ComScore market research, http://www.comscore.com/Press_Events/Press_Releases/2008/08/Social_Networking_World_Wide