Raiding the inarticulate since 2010

accelerated academy acceleration agency AI Algorithmic Authoritarianism and Digital Repression archer Archive Archiving artificial intelligence automation Becoming Who We Are Between Post-Capitalism and Techno-Fascism big data blogging capitalism ChatGPT claude Cognitive Triage: Practice, Culture and Strategies Communicative Escalation and Cultural Abundance: How Do We Cope? Corporate Culture, Elites and Their Self-Understandings craft creativity critical realism data science Defensive Elites desire Digital Capitalism and Digital Social Science Digital Distraction, Personal Agency and The Reflexive Imperative Digital Elections, Party Politics and Diplomacy digital elites Digital Inequalities Digital Social Science Digital Sociology digital sociology Digital Universities elites Fragile Movements and Their Politics Cultures generative AI higher education Interested labour Lacan Listening LLMs margaret archer Organising personal morphogenesis Philosophy of Technology platform capitalism platforms populism Post-Democracy, Depoliticisation and Technocracy post-truth psychoanalysis public engagement public sociology publishing Reading realism reflexivity scholarship Shadow Mobilization, Astroturfing and Manipulation Social Media Social Media for Academics social media for academics social ontology social theory sociology technology The Content Ecosystem The Intensification of Work The Political Economy of Digital Capitalism The Technological History of Digital Capitalism Thinking trump twitter Uncategorized work writing zizek

What are the judgements which students are making as they use AI?

I enjoyed this paper by Walton et al about the role of judgement in how students use AI. There’s currently a startling lack of rich data about the judgements students actually make in their use of chatbots chatbots, as they summarise on pg 2:

There remains little information beyond decontextualised self-reports about
what these judgements might be and how they influence what students learn (or fail to learn) from completing their assessments. Thus, educators and institutions continue to design assessments and set policy to take account of GenAI use without understanding how their choices will affect students and learning. Exploring how students make judgements with—and about—GenAI will therefore provide a much-needed perspective on how students are coming to learn with, rely on, and dissemble with GenAI.

Through a nicely designed walk through method they identify six categories of what they call judgement events: time bound occurrences where the student where a student evaluates AI and its outputs as they worked on an assessment. What I particularly like about this framing is how it enables us to distinguish between:

  1. The occurance and sequencing of judgement events.
  2. The (epistemically) better or worse judgement events which make up that sequence

It does what Milan and I describe in The Platform Learns To Speak as opening the blackbox of AI use in order to look at the process which underpins it. The obvious lesson to take from the notion of judgement events is to ask three questions of assessment design:

  1. What is the process? Where and when are judgements called for?
  2. What kind of judgement events are desirable for constructive alignment?
  3. How does the logic of the design ideally knit together these judgement events?
  4. How does the embedding of the assessment support or hinder this ambition?

A crucial point they make in this paper concerns student’s ability to distinguish their own epistemic contribution to the output. I’ve been prone in the last year to saying that we need to help students understand what it feels like to be learning*. I stand by this but I realise it’s a precarious achievement rather than something we can rely on. This is why I like their two points on pg 13 so much:

This makes two points: firstly, GenAI use can enhance or hinder learning,
depending on circumstance. Secondly, students’ own views of what they learnt or how they worked with GenAI use does not distinguish between these cases.

*Thanks to David Meechan for setting me off on this, when he visited us.

Fediverse Reactions
, , , , , , ,
,