The ontology of (digital) testing

My notes on Thompson, G., & Sellar, S. (2018). Datafication, testing events and the outside of thought. Learning, Media and Technology, 43(2), 139-151.

In this paper Thompson and Sellar cast a Deleuzian lens upon the data hungry character of contemporary educational institutions. As they put it on 139, “Education institutions, and the people who work and learn in them, are constantly generating and using more and more data”, much of which is digital. This proliferation of digital data reflects and in turn encourages complex forms of software, in turn driving the development of interoperability systems to ensure they can work together. This interoperability makes ‘big data’ techniques possible in a way they otherwise wouldn’t be. This is an important reminder that data analytic possibilities don’t unfold naturally from new technology but rather require institutional work with influential sponsorship in order to make them happen. They caution that the “the conceptual tools that we have for thinking through these new technologies in assessment and learning, particularly in relation to implications for politics, policy and practice, remain caught between dystopian visions of technology undermining human values, culture and subjectivity, or utopian fantasies about continual improvements to the efficiency and effectiveness of learning.” (139-140).

Deleuze and Guattari help that revolutions in institutions such as education proceed through challenges to orthodox images of thought (“the set of implicit presuppositions about what it means to think”) which “create new possibilities for thought that can bring about new activities, behaviours, organisations and connections” (140). However “thought that is formed in digital learning environments such as CATs ultimately conforms to a model: an information ontology” and cannot be revolutionary in this sense (141). The production of ever more data produces events, in so far as the data leads things to happen which otherwise wouldn’t, but it is constitutively unable to produce a new image of thought. This runs contrary to the disruptive rhetoric of educational technology which promises innovation and change.

Data can be produced in educational settings through a variety of means: deliberate production in assessment and management procedures, volunteered through the use of digital technology or produced through covert monitoring. It assumes a form as information through models encoded within hardware and software architectures at work in its production, as well as the processes at both levels at work in its generation. Informational ontologies are crucial to the operation of computing processes and facilitating interoperability between processes.

But where does the desire for data come from? Their Deleuzian approach sees desire as primary, productive of subjectivity. But this is where their philosophical approach seems weak to me, in comparison with the digital sociology approach they distinguish it from. It makes it difficult to think systematically about who desires it, how they desire it and how the context frustrates or facilitates it. The turn towards the para-subjective makes it hard to pin down subjects in any concrete way. They reflect on how “Groups and individuals come to believe that they need something to solve their problems and this lack is felt as a lack of data” (143). But their concern for the “abstract essence of this desire” immediately leads them away from this empirical specificity towards a ‘bargain with modernity’ in which data helps fortify people against the uncertainty which is endemic in modernity. Well sure but how does this specifically play out in educational systems? Or is this dynamic manifest uniformly across all spheres of society? The philosophical heterogeneity transmutes into an analytical monism and this is a problem for me. I’m not sure how it helps us make sense of education at all, as opposed to providing a vast panorama through which to sketch the questions about education we have in much grander terms.

I find the analysis much more useful when it comes to standardised tests, consisting of sample tests (a representative sample of pupils) and census tests (all within a population). These are adminsitered uniformly and scored in a predetermined way. They predate neoliberalism and the authors observe that many critics of these ‘neoliberal instruments’ fail to pay attention to the process by which they are constructed:

  • The domain is “the specific area of interest that is being measured, whether it is a body of knowledge, skills, abilities or attributes” (144). Sub domains have a relationship with each other. What are tested are constructs which are designed to track unobservable characteristics through evaluating observable behaviours.
  • This involved sampling from within the domains, drawing on a selection of potential questions within a domain in order to construct a test which can be done in a finite amount of time while still facilitating inference in the way described above. This is guided by a test specification plan.
  • Inference involves using test scores to assess achievement within a domain. Standardisation means this can be undertaken at the level of the individual, group, school or nation. Not all inferences are valid.

The computerisation of these tests promises capture transactional data, facilitating big data analytics. Selection can be built into the test itself, allowing branching or pathways depending on the student’s response and aptitude. But what sort of event is this? They suggest three concepts to classify these:

  1. Break-events involve the movement from one pre-existing category to another. As they put it, “Labelling a student as above-average, a school as failing, or a system as excellent based on aggregated data, are all examples of break-events that express potentiality as information” (146). These categories matter and they bring some possibilities to the fore while suppressing others.
  2. Crack-events are forms of change that lack the perceptibility of moving between pre-constituted categories. They occur all the time (“such as feelings of bewilderment, anxiety or elation when sitting a test”) but often don’t manifest in a recognisable way.
  3. Rupture events occur when the coordinates of thought are lost, as cracks aggregate into a potentially transformative moment of change. Revolutionary technologies could only be such if they produce rupture events.

Standardised testing can generate breaks and cracks but not ruptures becauset they are “created according to an image of thought that limits (a) what tests should be and (b) what the purposes of schooling are” (148). They cannot break with the past because they are premised on a sense of correct answers and simple errors, moving people between discrete categories on that basis.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.