Justin Lewis and Ted Wayland
Olympic College
This assignment challenges students to use LLMs to map the counterarguments to their main claims through a process of counterclaiming. By writing prompts that position the AI as argumentative adversary, students can refine, extend and evolve their thesis over the course of the research, drafting and revision processes. This lesson is focused on 100- level composition courses but could be adopted to any undergraduate or graduate course focused on argument and claim making.
Learning Goals:
Materials Needed: Large language model with linguistic analysis feature such as GPT-3.5 or GPT-4
Original Assignment Context: First-year writing course
Timeframe: ~2 course sessions
Overview
AI chatbots that rely on LLMs, such as ChatGPT, provide a wealth of opportunities for instructors working in first year writing. When directed appropriately, a chatbot provides a collaborator for students in numerous invention activities (Anson 42). In our composition courses, we’ve used ChatGPT and other LLM wrappers to map the stakeholders or relevant audience(s) for a given argument, summarize key points from short course texts, provide alternative language to (re)shape thesis statements, and restructure drafts through the tool’s imposition of outlines on draft-stage freewriting activities. In this short teaching reflection, we’ll describe how we’ve used ChatGPT as a method of counterclaiming, or the process of mapping the complex counterarguments that work against the initial claim-making of students in first year composition. Inviting students to sketch counterclaims with ChatGPT not only provides them novel pathways to new research foci, it also invites them to resist overcommitting to a central claim too early, instead encouraging them to consider how their claims may evolve through a series of successive complications in the research process (Rosenwasser and Stephens 139; Nussbaum and Schraw 59).
Context
Both authors teach first year writing courses at a mid-sized community college in the Pacific Northwest. Like most community college student populations, our demographics vary widely and include a diversity of ages, backgrounds, and abilities. Through direct transfer agreements, many of our students move on to four-year universities after completing their Associates; however, students in our courses also pursue certificate and degree pathways in skilled trades or are completing pre-college requirements before applying to other schools. Because of the college’s proximity to US military installations, we also provide educational opportunities to many active duty and recently discharged military personnel.
The counterclaiming activities that follow were developed and integrated into two first year writing courses. For Ted, the tool was integrated into his Winter 2023 section of ENGL101, a class dedicated to introducing students to academic literacy in reading and writing. For Justin, the tool was first utilized in Fall quarter 2022 but further refined in his Winter 2023 ENGL102, a course that emphasizes academic research and the development of source-supported academic arguments. While the broad course outcomes for ENGL101 and ENGL102 are similar, the major difference between the two courses is 102’s emphasis on finding, evaluating, and integrating sources into student writing. Both courses focus on critical reading practices, rhetorical choices in academic writing, and the use of a recursive writing process.
Justin used the counterclaiming activities during the second week of Unit 1 (4 weeks total). The first project is an “Inquiry Memo” that invites students to submit a proposal for the research topic they aim to pursue over the course of the term. The main activities of the inquiry memo include defining research topics, defining researchable questions, and posing a main claim to guide research practices in Unit 2 (annotated bibliography – 2 weeks) before composing a researched argument in Unit 3 (4 weeks). The activities of this assignment occupied two class periods, or the third week of the first unit. After completing these activities, students will be introduced to questions of source credibility in the final week of the unit.
On the first day of his ENGL101 course, Ted showed students a short video of Jacques Droz’s 18th century writer automaton, sparking discussions on our collective definition of the act of writing and its relationship to cognition. With the memory of that uncanny, clockwork writer in mind, the course’s final essay assignment introduced the use of ChatGPT as a digital interlocutor. The final assignment asked students to write a thesis-driven essay on a topic related to memory, synthesizing information from a film and essays relevant to the topic. Students were asked to use ChatGPT at two moments in the writing process for the assignment: when working on defining their topics, students used multi-stage queries with the bot to create more specific, narrow subtopics, and later, when working on crafting theses, students used the bots to generate counterclaims to test their arguments and refine their theses in response. Finally, students were asked to reflect on their experience using AI in the writing process and to revisit their ideas about the intersection of writing, technology, and thought.
Assignment Goals
Over time, we’ve both attempted numerous invention activities to encourage students to map counterclaims to the initial positions they adopt early in the writing process. We advocate this kind of work because we want students to avoid overcommitting to a particular position until their research process is complete. Ideally, we want students to evolve their thesis over the course of their reading and writing activities. Unfortunately, as many students remind us, “We don’t know what we don’t know.” While keyword association and topic relationship mapping tools provided some pathways for students to discover counterclaims, the authors found these technologies cumbersome and tied to library subscription services. Because ChatGPT’s UI is intuitive and reflects a search-based mental model, we were hopeful that developing counterclaim activities in this interface would encourage higher student participation due to usability and familiarity.
The goals for our use of ChatGPT to counterclaim include:
The specific learning outcomes of our use of ChatGPT to counterclaim include:
Required Materials
At present, this lesson relies on the free preview of ChatGPT; however, we expect access to this model will change and likely require a fee that could be paid by the student, instructor or the institution. Another prompt-based, generative AI interface would also work for this assignment. In addition to ChatGPT access, students were required to use the prompts.chat Github repo to shape language around defining the subject position of the AI. This lesson could be customized/extended by integrating additional subject roles for the AI using the prompts builder at prompts.chat.
Assignment Results
Ted’s ENGL101 Course
ENGL101 students showed a great deal of enthusiasm and interest with using ChatGPT in the writing classroom. The majority of the students in Ted’s two classes had not used an AI bot for writing before our activities in February 2023, and many had not even heard of ChatGPT. During the first activity, students were wowed by the capabilities of the AI, one student noting that it was “interesting how it imitates lifelike conversation.” Their first activity centered on generating subtopics for an essay assignment on the theme of memory in a film they had selected, and the general feedback from students was that it was helpful to query the bot as a brainstorming tool, quickly generating ideas that suggested new possible topics for their essays; there was largely a consensus that the subtopics generated by ChatGPT were reflecting their own ideas back to them, albeit in a more focused manner. A student who fully embraced the tools said “it’s hard to come up with a good topic, but with this, you put in your idea and it gets right to your point!” It was remarkable to them how well ChatGPT could generate subtopics with some degree of specificity on a particular film: it might mention, without a prompt, that Rashomon’s director is Kurosawa, or seem to possess a knowledge of the disjointed narrative in Memento. On the other hand, students observed moments where the AI offered up fuzzy, incorrect, or incomplete information about films, especially less well known ones; it seems not to have “seen” Chris Marker’s La Jetée, for example.
On this first day of working with ChatGPT, our discussions of the process had a consistent theme: amazement with what the AI could produce, mixed with some uneasiness with the technology; one student explained that “it felt like cheating” to use AI this way, while another expressed a feeling that the AI “took the art out of writing.” In response, another student approvingly described the AI as offering up a skeleton that he would be able to flesh out with his own writing.
The second ENGL101 activity took students from topics to theses, and this time ChatGPT’s role was providing counterclaims to students’ first drafts of a thesis. Interestingly, on just the second day of using the bot, students were already past the novelty of AI writing and were more intent on stretching its capabilities and looking at it with a more critical eye. Students wrote a reflective note at the end of their assignments, and the general feedback was that using the AI was helpful and something they could imagine incorporating into their writing process—with, however, significant caveats. In class discussions and the students’ reflections, students returned again and again to the generic, boring, non-committal quality of the writing ChatGPT produced, and by the end of the activity, students were balancing their fascination with the possibilities of the technology with sharp observations on its limitations. Ultimately, this led us back to our class mascot, Droz’s writer automaton, and a recognition that thinking and writing remain, for now, in the sphere of the human mind.
Justin’s ENGL102 Course
Overall, students enrolled in the research focused ENGL102 course responded positively to the use of ChatGPT in the invention process. Beyond students’ initial excitement over the “magic” of AI text generation, students noted that ChatGPT provided them with new pathways for research and discovery, leading them to question their initial reaction to a broad topic. For students with claims that were far too broad, ChatGPT tended to narrow through complication by identifying counterarguments that supported the counterposition to their own. For students whose initial claims were too narrow, the AI occasionally provided new vocabulary and new lines of research related to the counterposition. These new counterclaim subtopics tended toward less specificity, allowing students whose initial claim-making was too granular to zoom back out to identify a more appropriately scoped focus.
Yet, not all students reported that the counterclaiming activity was generative. For students with topics that were of the appropriate scope and specificity, the AI sometimes provided generic counterpositions that restated their own or that they had already identified in their own research and writing. For still others, the counterclaiming activity complexified their research questions and claim-making in negative ways by diverting the research process and creating too many opportunities for further exploration, especially in the context of a quarter-long research project.
Without equivocation, when challenged with evaluating the sourcing credibility of the AI, students found that ChatGPT fabricated evidence by inventing credible sounding sources. Interestingly, this didn’t invalidate the counterclaims themselves; rather, as one student put it, “The sources are bullshit, but the ideas seem legit.” In this way, we view ChatGPT – and other tools like it – in much the same way as Anson and Straume who see these AI as “’writing assistants’ requiring some supervision of outputs” to ensure that the ideas and the sources aren’t both ‘bullshit’ (5).
Works Cited
Anson, C.M. “AI-Based Text Generation and the Social Construction of Fraudulent Authorship: A Revisitation.” Composition Studies, vol. 50, no. 1, 2022, pp. 37-46.
Anson, C. M., and I. Straume. “Amazement and Trepidation: Implications of AI-Based Natural Language Production for the Teaching of Writing.” Journal of Academic Writing, vol. 12, no. 1, Dec. 2022, pp. 1-9, doi:10.18552/joaw.v12i1.820.
Nussbaum, E. Michael, and Gregory Schraw. “Promoting Argument-Counterargument Integration in Students’ Writing.” The Journal of Experimental Education, vol. 76, no. 1, 2007, pp. 59–92. JSTOR, http://www.jstor.org/stable/20157471. Accessed 13 Feb. 2023.
Rosenwasser David and Jill Stephen. Writing Analytically. 5th ed. Thomson Wadsworth 2009.
Over the course of class this week we’ll be using artificial intelligence to assist us with discovering new pathways and avenues for research. During the first two weeks of the unit, you identified a main idea, or topic, that you’re invested in and that can sustain your research activities for the term. Further, you’ve also evaluated what’s arguable about that topic and developed an initial arguable main claim to guide your research. Now that you’ve found some stability around your topic, we’ll be conducting some activities to destabilize that topic. My hope is that in completing this process, you’ll reassess your existing previous work by accounting for the intricate nature of your initial claim. In addition to refining and evolving your claim, we’ll also use these activities to introduce source credibility as an integral component of the research process.
Day 1: Writing Good Prompts for Chat GPT & Initial Queries
Overview
Today’s class will be dedicated to securing a ChatGPT account, crafting effective prompts for the AI and executing counterclaim queries.
Learning Objectives
After today’s class, students:
Activities
I want you to act as a debater. I will provide you with some topics related to current events and your task is to research both sides of the debates, present valid arguments for each side, refute opposing points of view, and draw persuasive conclusions based on evidence. Your goal is to help people come away from the discussion with increased knowledge and insight into the topic at hand. My first request is “I want an opinion piece about Deno.”
Your first writing assignment in this counterclaim activity is to rewrite this prompt so that it best fits your own research project and main claim/thesis. So, for example, if my topic is “Gifted programs for students in K-12” and my main claim was, “Gifted programs provide students with positive opportunities to excel beyond their peers,” I would rewrite the prompt above this way:
I want you to act as a debater. I will provide you with a main topic and my claim about a topic. Your task is to research the claims that go against my topic and highlight the best arguments against my main claim. Your goal is to help me better understand alternative viewpoints on my topic and other claims that could be used to argue against my main claim. My main topic is: “Gifted programs for students in K-12.” My main claim is, “Gifted programs for students in K12 provide students with positive opportunities to excel beyond their peers.”
So, to begin, identify the persona you want to use to develop your counterclaims. Next, rewrite the prompt provided for that persona in a way that you believe will generate claims and/or arguments against your claim.
Homework
Continue to refine your prompt and generate responses to your queries. For homework, refine your prompt at least three times, run the subsequent queries as new chats and copy+paste that information into a document to refer to in our next class meeting.
Day 2: Mapping Research Opportunities and Evaluating AI Credibility
Overview
Today’s class will be dedicated to mapping the counterclaims provided by your ChatGPT queries to complicate your initial claim and identify new research opportunities. We’ll also spend a bit of time considering the reliability and credibility of the AI responses.
Learning Objectives
After today’s class, students:
Activities
Homework
Read “Assessing Source Credibility for Crafting a Well-Formed Argument” in Writing Spaces Volume 3.