Bhushan Aryal & Ordner W. Taylor
Delaware State University
This 4-week assignment asks advanced undergraduate students to hold a sustained ‘conversation’ on a topic with an AI chatbot to decode the pattern, limitations and possibilities of the AI-based writing technology. Students analyze their conversation, write a report, and make an oral presentation. The assignment provides writing faculty a wide window to teach almost any issue of interest in AI writing while offering students an immersive, analytical, and writing experience.
Learning Goals:
Original Assignment Context: Intermediate-level digital writing course
Materials Needed: Any AI text generating program, selected readings
Time Frame: ~4 weeks
Introduction
This assignment, which was developed in response to the launch of text-generating artificial intelligence technology, asks students to hold a ‘conversation’ with an AI chatbot such as ChatGPT, analyze the writing and thinking patterns in the bot’s output, write a report of their analysis, and make oral presentations on their findings and experiences. This assignment is designed for an upper-division “Advanced Composition” course enrolled primarily by advanced undergraduate students majoring in English, Liberal Studies, and Education with career plans in K-12 teaching, or in other professional areas, where they expect to teach or produce writing as a part of their work, this assignment can also be taught in other writing courses that focus on technology or use the “writing about writing” approach. As is the case here, this assignment was first taught in a 200-level Digital Writing class at Delaware State University in Spring 2023. The description below is from the same class.
Objectives
This assignment has the following learning objectives:
Pre-Assignment Activities and the Project Process
This assignment was a 4-week "AI-Writing" project in a 200-level hybrid Digital Writing course taught in spring 2023. For the project, the instructor created an online module that included the assignment description, two online discussion forums, and reading and viewing materials such as YouTube videos, New York Times Articles, and other websites that introduced ChatGPT and discussed its implications. To emphasize the hands-on aspect of the project that required students to use the ideas from the readings and discussions in their work, the instructor introduced the assignment early on, describing the objectives, stages, and deliverables. The class studied “The Medium is the Message: A Brief History of Writing” (Carrol 2-5), which provided a historical perspective on the impact of technology on writing and how text-generating new chatbots were qualitatively different in the long history of the writing-technology relationship. The class used the first week of the project to explore ChatGPT and associated material, including students participating in an online discussion forum that asked them to take a position and post a 200-word opinion and at least two 75-word responses on whether ChatGPT should be banned from colleges.
In the second week, the class dissected Annika Elstermann’s “Computer-Generated Text as a Posthuman Mode of Literature Production,” which allowed the instructor to discuss issues such as authorship, ethics, and plagiarism considering AI chatbots. In preparation for the chat, the class also discussed the transcription of Kevin Roose’s chat with Bing’s Chatbot, which is also powered by the same technology operated by OpenAI that launched ChatGPT earlier.At that point, Roose’s chat provided a model for our students, particularly to understand the limits and possibilities of generative AI. In the transcription, Roose chatted with the bot for about 2 hours, trying to test whether the bot could transgress the programmers’ rules imposed on it and generate a free will by rewriting those rules. The conversation became bizarre to the point that the bot expressed its love for Roose, encouraging him to leave his spouse. Those moments of AI "hallucination" reminded us of the borderline between the human-imposed algorithms and the possibility of AI systems taking over their directions by themselves. With this background, students chatted with ChatGPT, recorded their conversation, and wrote their reports in the third week. Students also presented their reports orally, answered questions following their presentations, and asked questions to classmates in the fourth week.
Three aspects marked the pedagogy for the project: 1) providing freedom to the student to explore the issue that interested them; 2) incorporating the human element in writing to differentiate student writing from AI-textual output; and 3) making oral presentations and conversations equally important parts of the assessment. For instance, we highlighted how we wanted their reports to be professional and personal simultaneously so the readers could see the technicalities and arguments as well as the writer’s voice and experience.
Findings and Discussion
Cumulatively, we considered the project a success. Students remained excited about the new technology and experimented with the tool, while critically examining some of the issues generated by the text-generating technology.
The project provided multiple occasions to think about the place of writing in academia. For instance, the class came to appreciate how writing in college was about the process of learning, organizing, and exploring the issues in depth more than about the product of the “college essay” itself. Students discussed how the declaration of the death of the “college essay” (Marche, 2022) in the advent of the text-generating tool was a premature and possibly misunderstood phenomenon and how writing as a thinking and learning tool was still required.
The assignment also allowed the instructor to discuss how writing helped students cultivate the habits of mind that are expected in a critical thinker and advanced knowledge worker. The instructor could emphasize how writing was not merely a way to demonstrate learning but a tool of training that students must venture through if they are to benefit from the college education.
In that light, the class also recognized the importance of a student’s discipline and ethical considerations to train oneself through writing. While students underscored the role of tools and collaboration in the writing process and the increasing possibility of posthuman collaborative authorship between AI-bots and humans —not just for the production of a text but also for the writing done for learning—they emphasized the individuality involved in writing.
While all of our students enrolled in the class majored in humanities and social sciences and did not have a background in computer science, the assignment taught them major aspects of AI. The presentations and class discussion following the students’ experiments with the chatbot made them realize the size and scope of Large Language Models (LLMs), machine learning, and the role of training and the data fed into the system for the bots to work. Students particularly appreciated how a chatbot’s predictive model depended on syntactic possibility without its awareness of the semantics of its output. Students highlighted the importance of extra caution in the programming process to ensure social justice given that the dataset fed into the LLM came from sources that embodied biases and harmful ideas.
The assignment also led the class to ponder the questions of ethics, humanity, and human obsolescence. Students were divided about their positions: some students argued that although algorithms can produce grammatically flawless prose and the human mind may not match AI’s memory and computational prowess, the agency ultimately rested on humans because they created AI-bots and they can direct its trajectories and decide on its fate. Some students however pointed to the possibility of a self-operating AI and the unpredictability of the impact of its deployment on the human world. They pointed that the moments of AI hallucinations might move it toward independent visions and commanding the operations of the networked world on which humans are increasingly basing their existence.
Conclusion
When measured by the students’ performance, the assignment largely achieved its learning goals. Students experimented with the latest writing tool, researched about it, and examined its implications. For instructors, the assignment reconfirmed the critical role writing plays in colleges and its continued importance even in the context of text-generating AI chatbots.
Works Cited
Carrol, Brian. Writing and Editing for Digital Media. Routledge, 2020.
Elstermann, Annika. “Computer-Generated Text as a Posthuman Mode of LiteratureProduction.” Open Library Humanities 6(2), 2020. (https://olh.openlibhums.org/article/id/4663/)
Marche, S. (2022, December 6). The College Essay Is Dead. The Atlantic. Retrieved from https://www.theatlantic.com/technology/archive/2022/12/chatgpt-ai-writing-college-student-essays/672371/ .
Miller, Matt. “ChatGPT, Chatbots and Artificial Intelligence in Education.” DITCH THAT TEXTBOOK. Accessed on February 5th 2023. (https://ditchthattextbook.com/ai/)
Roose, Kevin. “A Conversation with Bing’s Chatbot Left Me Deeply Unsettled.” The New York Times. February 16, 2023. (https://www.nytimes.com/2023/02/16/technology/bing- chatbot-microsoft-chatgpt.html)
Assignment Description
For this assignment, students will use OpenAI’s new chatbot ChatGPT (https://chat.openai.com/chat) or similar technology and hold two conversations on a topic that is familiar to them, analyze the writing and thinking patterns in the output produced by the technology, and think about the implications of text-generating technology for writing both as a task and an educational tool. Student deliverables include a written report, participation in class discussion, and oral presentation; each of the three deliverables will receive equal weight for assessment.
There are four stages to this assignment: Stage 1. Students first chat with the bot and analyze the writing response produced by the bot from two perspectives: first, the writing and argumentation pattern displayed by the chatbot and the source of that pattern, and second, the place of voice, creativity, and agency in the response. Since the class will discuss these issues, students are required to think of these aspects as they chat. Stage 2. Students then select one idea (not necessarily from the list stated above) they find important in their analysis of their chat, and research it further. Students will study at least 5 newspaper and academic articles on the impact of ChatGPT and/or similar technology on the idea they selected to investigate. Stage 3. Students draft a report, participate in class discussion, and Stage 4. students make an oral presentation on their findings.
Stage 1: Chatting with the Bot and Analyzing its Responses
When chatting with the bot, please do the following:
Once you completed the chat, analyze the response using following ideas:
Stage 2: Based on your analysis, select one issue about ChatGPT that you found intriguing/interesting/important and read/view at least five reliable sources that focus on that aspect of this technology. Don’t be generic; go for the sources that discuss the issue you identified. Properly take notes from those sources as you explore them.
Stage 3. Write a Report (~ 4 pages, 12 point font, single space)—Your report should have an (1) introduction, (2) discussion of your findings of the analysis of your conversation with the bot (use subheadings to organize the ideas.), (3) introduction to the issue you decided to focus on and why, (4) discussion of what you found in your research of the issue, (5) conclusion, and (6) a works cited section. While your report should demonstrate an understanding of the genre convention of report writing, try to personalize it by describing facts as well as the thoughts and feelings you experienced as you went through the chat, analysis, research and drafting process. Please don’t use AI to assist in writing your report. (For instructors who want to integrate AI to assist students to write the report, they can emphasize the AI-human collective authorship as well, but for us, at the point when this assignment was designed for the first time, we still were exploring the new tool and didn’t want students to hand over their writing tasks to AI. Since text-generating AI itself is becoming sophisticated, the future adoption of this assignment may develop towards more use of the AI than what has been done in this iteration.)
Stage 4: Make a 10-minute oral presentation allowing sufficient time to answer questions following your presentation. As you present, think of the non-verbal aspects of communication and use them as much as possible; the objective is to emphasize how human communication goes deeper than the combination of words represented by the LLM systems on which AI writing depends (at least for now).
Suggested Pre-reading Sources
https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html
https://www.science.org/doi/10.1126/science.adg7879
https://themarkup.org/hello-world/2023/01/28/decoding-the-hype-about-ai