In Generative AI for Academics I drew the distinction between conversational agents and templated responses. In the former you interact in natural language with a chatbot through a dialogue, whereas in the latter you select buttons to enact pre-defined transformations to text. This is how I talked about it in a webinar a couple of years ago:
Kim et al draw a similar distinction: “Two interaction techniques currently dominate: dialogue (e.g., OpenAI’s ChatGPT and Google’s Gemini) and predictive text completion (e.g., GitHub Copilot)”. I’d suggest that inline automation as found in Microsoft Copilot 365 or Gemini in Google Docs is a development of predictive text completion, in the sense that a user can complete a transformation of text by pushing a button. In both cases the system produces content with the difference being how this production happens: with chatbots it happens through dialogue and with inline automation it happens through pressing a button. If you’re a confident writer who uses chatbots in a purely dialogical way, it’s easy to forget how much chatbot use can be geared towards the production of content. I’d say it’s better educationally in the sense that some articulation is necessary to produce anything from a chatbot. But it’s still outsourcing a task to the machine in a radical way.
The approach developed by Kim et al tries for a third approach which “motivates users to reflect on their text with LLM-generated summaries, questions, and advice on writing (which we refer to as LLM views), helping them discover opportunities for improvement or elaboration”. They offer a useful definition of two stages involved in this process (my emphasis):
Revision means critically examining and evaluating, which we refer to as reflection, and identifying any opportunities for improvement or further development, which we refer to as discovery, and then making the appropriate changes. This can occur at any stage of the writing journey
https://hai-gen.github.io/2024/papers/9904-Kim.pdf
The purpose of their interface is to offer perspectives on the text which scaffold reflection and discovery while avoiding the language model merely producing text directly for the user. They identified three outgrowths in their pilot of the project which are educationally relevant: (1) discovering underdeveloped ideas, (2) catering to their audience, and (3) identifying opportunities to improve clarity.
If we’re going to be using inline automation with students, this suggests how we can do it in a pedagogically responsible way. The purpose is to offer perspectives, taking advantage of how the model is embedded in the office software, rather than predictive text completion. Copilot offers these perspectives which present themselves for educational use. The problem is that (1) they are integrated seamlessly into an interface which is built around inline automation (2) they are not fine tuned for the specific domain so their advice can often be questionable.
But they are nonetheless there, which offers pedagogical opportunities. The key I think is to encourage students to gravitate towards dialogue (using the prompt windows where they are available) and perspectives (using the reflection tools built in) while discouraging them from using the push-button forms of automation. Where these are used, they need to be engaged with thoughtfully: to reflect on why you are selecting this button and to review the consequences of pressing it. The challenge is introducing moments of friction and reflection into software which is fundamentally designed to be frictionless, but it’s not insurmountable I think. Engaging with it educationally in this way though presupposes a lot of critical AI literacy in general and familiarity with Copilot 365 in particular.
Mark Carrigan 