Big data and the future of sociological theory
My notes on Mohr, J.W. (2015) Big Data/Big Theory – Part 1. Perspectives 37(2) and Mohr, J.W. (2015) Big Data/Big Theory – Part 2. Perspectives 38(1)
It’s often assumed that big data poses a threat to sociological theorising in the traditional sense, with the infamous Wired article on the ‘end of theory’ standing as emblematic of the belief that sufficiently large datasets mean theory can be dispensed with. While recognising the limitations which attend to transactional data, such as the irreducible material which surrounds the digital and the limited range of features of social life which can be digitised, John W. Mohr strikes an optimistic note about what this means for the future of sociological theory:
I think Big Data does have the potential to produce digitally accessib le information that is far richer than anything social scientists have ever had or known before, and that some part of that richness will come from the fact that much of that data is produced within the very flow and practice of daily life itself. Instead of gathering answers retrospectively from standardized survey questions, Big Data can provide texts from spontaneous tweets, posts, or messages that are wound into dynamic conversations between friends or communities, thus allowing social scientists to capture social life in its natural richness as it unfolds in real time. High quality data could mean data that was created authentically, with complete textual (and visual or audio?) content recorded, all types of relational signatures captured, and precise temporal and geo-stamping included.
There are technical objections which can be raised to much of this. The language of spontaneity obscures the role of platform architectures in shaping behaviour. The accuracy of geo-stamping can’t be assumed and it’s hard to imagine how it could be improved much beyond its current unreliable form. There are different degrees to the sense in which digital engagement exists ‘within’ social life. But these shouldn’t blind us to the epistemic gain which can emerge from transactional data and I found it valuable to read this from someone theoretically inclined who sees immense value in this data without believing it means that the sociology should be refounded as social physics. It also has a nuanced sense of the cultural ontology of those domains of social life which transactional data provide a new way of knowing:
Data can allow us to strategically examine different types and forms of meanings, from simple sentiments to complex thoughts, from immediate reactions to deliberative reflections.
I share his interest in “how Big Data is going to have an impact on the intellectual subfield of sociological theory over the next generation or so” and it seems to me this question remains under explored. He argues that the explanatory challenge provided by Big Data will mean that sociologists will need to call upon Big Theory. He offers three reasons for this claim: the paradigm effect, the data effect and the culture effect.
- The Paradigm Effect: he frames paradigms in Kuhnian terms as reducing the labour involved in scientific investigation by eliminating the need to continually reexamine first principles and instead focus one ver more specialised issues. While sociology lacks a paradigm in the Kuhnian sense, it does have paradigmatic components which link assumptions and technologies together in ways that regularise the measurement of the social world. If I understand him correctly, he means the survey research apparatus and content analysis as examples of such components, which have in turn generated all manner of innovations as people seek to address their limitations and aporias. Both reflected a fundamental problem of information scarcity, seeking to find ways of drawing conclusions about an intractably large whole from a carefully selected and methodologically measured part. Both involve the “careful and calculated leveraging of scarce information” (3). Therefore the paradigmatic components struggle when we enter circumstances where the assumed problems no longer hold true e.g. statistical measures of significance become useless. The absence of such a paradigmatic component to draw upon means that “research scientists working with Big Data will need help in theorizing where to look, how to look, what to look for, and what to make of what they are looking at”. Better theory is needed, encompassing the theorisation of measurement but also “new (and reinvigorated) theories of the social world, theories which can now find a new and possibly more illuminating empirical footing in the plenitude of information which has begun to come our way” (5).
- The data effect: even though ‘big data’ is associated with the size of datasets, their variety (different kinds of information) and what the tools, techniques and epistemologies they are giving rise to mean for the data we already had access to is just as significant. His discussion here concerns advancement in the digital humanities and what these offer for the emerging era of computational hermeneutics including the tools that are being built within DH and within the tech world that can be applied to this undertaking. It heralds a move from a methodological orientation towards throws away nuance, focusing on one question in a small sample, towards one which tries to find as much nuance as possible. Scarcity based methodological approaches have broken down and researchers need theoretical resources which can help the make sense of the interpretation of existing data.
- The culture effect: the character of culture is being changed as we are “inundated with textual data, visual data, audio data, and other kinds of highly nuanced cultural data, as the social world continues to digitize its subjective experience of selfness” (6). This creates the opportunity re-invent the study of culture and rethink the distinction between interpretation and quantification in how we seek to understand the social world.
In a way these all seem like iterations of the same point: new methodological challenges raise questions which have amongst others theoretical answers, creating an opportunity to ensure we can ‘upgrade’ social theory and ensure its relevance for these changing circumstances. The point I find really interesting is his observation that “theory tended to race ahead of our methods” and that now “methods have raced ahead of our theory, begging for more effective engagement” (7). This is a much clearer expression of something I’ve tried to say in multiple forums. He links to the disciplinary threat which failure implies, saying on pg 7 that “When social scientists don’t step up, physicists and engineers have no incentive to wait”. These give rise to ad hoc responses to the aforementioned question (often calling the ad hoc response a theory in the process).
Reply