Raiding the inarticulate since 2010

accelerated academy acceleration agency AI Algorithmic Authoritarianism and Digital Repression archer Archive Archiving artificial intelligence automation Becoming Who We Are Between Post-Capitalism and Techno-Fascism big data blogging capitalism ChatGPT claude Cognitive Triage: Practice, Culture and Strategies Communicative Escalation and Cultural Abundance: How Do We Cope? Corporate Culture, Elites and Their Self-Understandings craft creativity critical realism data science Defensive Elites desire Digital Capitalism and Digital Social Science Digital Distraction, Personal Agency and The Reflexive Imperative Digital Elections, Party Politics and Diplomacy digital elites Digital Inequalities Digital Social Science Digital Sociology digital sociology Digital Universities elites Fragile Movements and Their Politics Cultures generative AI higher education Interested labour Lacan Listening LLMs margaret archer Organising personal morphogenesis Philosophy of Technology platform capitalism platforms Post-Democracy, Depoliticisation and Technocracy post-truth psychoanalysis public engagement public sociology publishing Reading realism reflexivity scholarship sexuality Shadow Mobilization, Astroturfing and Manipulation Social Media Social Media for Academics social media for academics social ontology social theory sociology technology The Content Ecosystem The Intensification of Work The Political Economy of Digital Capitalism The Technological History of Digital Capitalism Thinking trump twitter Uncategorized work writing zizek

There is no solution to the AI and assessment problem

This is the core message of a surprisingly upbeat paper. There is no solution to the AI and assessment problem because it’s a classic example of a wicked problem. This means that, as they put it on pg 2:

Wicked problems, as opposed to ‘tame’ problems, do not have ‘correct’ or ‘incorrect’ solutions (Rittel and Webber 1973). This does not mean there are no ways forward, nor does it mean that all ways forward are equally valuable. However, it does mean that responses must look very different. For one, they require a shift from seeking definitive answers to engaging in ongoing, adaptive work shaped by competing priorities and evolving conditions.

There are a number of reasons they claim it is a wicked problem:

  • It cannot be clearly or conclusively defined
  • There is no clear criteria for knowing when ‘the solution’ has been reached
  • There are only better or worse options involving trade offs
  • There is a lack of clear metrics to adjudicate between these better or worse options
  • They cannot be studied through trial and error because every trial has real world consequences which means decision makers are on the line for them
  • The range of putative solutions and potential approaches is pretty much limitless
  • They exist because of deeper structural issues and reflect these issues
  • The framing determines which approaches show up for us as relevant

This means academics are “put in the position of needing to make continuous professional judgments in conditions of permanent uncertainty” (pg 12). This is not a good position to be in and it’s not going away. Rather than a council of despair, recognising the character of wicked problems is necessary for helping us cope with being placed in that position:

  • “First, it lifts the impossible burden on teachers and institutions to immediately get things right once and for all. When problems are unsolvable and ever-changing, missteps and course corrections are not failures. They are part of doing the work well.” (pg 12)
  • “Alternatively, a wicked problem frame suggests that trade-off are necessary and there is no optimal balance nor solution. The teacher who wondered ‘Have I struck the right balance? I don’t know’ (T6) was describing the uncertainty inherent in weighing pedagogical goals against workload, security against authenticity, current needs against future preparation.” (pg 13)
  • “Permission to diverge recognizes that in wicked problems, context determines every- thing. What transforms learning in a 20-student philosophy seminar becomes logis- tically impossible with 250 business students. What prepares future lawyers for AI-integrated practice might undermine the clinical skills nurses need.” (pg 13)

It means we can accept there is no fix but rather iterative and evaluative design work which is necessary because the environment has shifted in a fundamental sense. What matters is that we are moving in the ‘right’ direction while ensuring that we build up a variegated (and always provisional) sense of what ‘right’ is that reflects the range of different practices and imperatives within a multidisciplinary university.

Fediverse Reactions
, , ,
,