Daniel Hutchinson (History) and Erin Jensen (English)
Belmont Abbey College
This assignment asks first-year undergraduate history and English students to use AI writing models to aid in accessing and understanding readings on specific topics. Students used AI to understand the texts they were reading including the Declaration of Independence and rhetorical analysis readings. Students asked AI questions about the texts and evaluated how AI created academic citations. Students used AI to understand the readings, but also engaged in critical thinking about using AI.
Learning Goals:
Original Assignment Context: Multiples sections of 100-level undergraduate core curriculum courses typically taken by first semester students (Freshman Writing and Western Civilization I).
Materials Needed: Accessible AI text generator (ChatGPT and Explainpaper used)
Time Frame: ~1 week
Introduction
We are a history professor and an English professor, and we have many reading and writing assignments in our classes. We wanted to create an assignment where students would use AI writing models to aid in accessing and understanding readings on specific topics. We used this assignment in several sections of 100-level undergraduate core curriculum courses typically taken by first semester students (Freshman Writing and Western Civilization I).
We successfully implemented the assignment in both classes but experienced a few challenges. The week that Jensen had her classes use AI was when the college wifi blocked both of the AI programs she was planning on using (Explainpaper and ChatGPT). Through some creative decisions, the students discovered the college's front lawn had the best reception for using cell phone hotspots, which is how they could access and use the AI applications. Fortunately, when Hutchinson tried a week later, he did not run into this situation; his students could access the AI apps from their classroom. We specifically used Explainpaper and ChatGPT, but instructors can choose from an ever-increasing number of AI applications for this lesson plan.
We sought to foster student engagement and comprehension of assigned texts by reading, summarizing, analyzing, and writing. Jensen had her students use texts focused on learning about rhetorical strategies found in arguments. Many students struggled with these concepts and unfamiliar vocabulary words such as rhetorical devices, allusions, Aristotelian appeals of argument, imagery, tone, etc.The students successfully used the following assignment and AI apps to better understand the texts they were reading and to apply their learning to other class assignments. Hutchinson likewise directed students to experiment with the AI's facility to explain challenging passages of a historical document (the Declaration of Independence) in plain language. He also encouraged them to test the AI's ability to explain concepts through familiar frames of reference. Students could choose to have the AI explain a passage through characters from their favorite works of popular culture. For example, one student sought clarification on the following passage:
[King George III] has called together legislative bodies at places unusual, uncomfortable, and distant from the depository of their public Records, for the sole purpose of fatiguing them into compliance with his measures.”
She then asked explainpaper to translate this passage from the perspective of an invented scene from Jane Austen’s Pride and Prejudice:
In Pride and Prejudice, Mr. Darcy has called a meeting of the local gentry to discuss his plans for improving the estate. He has chosen an inconvenient location far away from their usual gathering place in order to make it difficult for them to attend and tire them out so that they are more likely to agree with him.
Students reported success in having the characters from Big Bang Theory, Friends, Star Wars, Game of Thrones, and Spiderman explain different aspects of the Declaration of Independence.
The assignment consists of prompts for students to explore how an AI engages with the nuances of an assigned reading and how this tool can aid students in developing their own ideas and arguments through revision and dialogue. We believe these technologies have real potential to serve as AI writing tutors.
We explored with students the potential strengths and weaknesses of AI as an academic tool. Hutchinsons’ students began by summarizing key points of the Declaration of Independence and then comparing their summaries with the AI. Jensen had her students complete a similar activity, only they based their summaries on articles they had found that focused on rhetorical analysis strategies. Many students found the AI's summary accurate, informative, and in some cases, more comprehensive than their summarizations. However, others found the AI summaries too superficial or repetitive and preferred their own approach.
Students in both classes then probed the AI's accuracy. Hutchinson had his students select a person, place, or event mentioned in the Declaration and then asked the AI a factual question about their choice. Jensen had her students look at specific mentions of facts or concepts in the articles they were using and had them ask AI a factual question. All students then conducted basic fact-checking of the AI’s output via online research via Google and Wikipedia. For future iterations, such fact-checking will be paired with broader forms of training on informational literacy. The AI excelled on queries about well-known topics, such as the life of figures like George Washington or King George III or why pathos is an appeal of argument often found in song lyrics. Yet more specialized questions, such as the number of children born to an obscure Founding Father or why rhetorical questions were used in a specific song, resulted in incorrect answers or what is commonly known as "hallucinations.”
Students then examined the AI's ability to provide academic citations for its statements. The AI provided sources drawn from Wikipedia, the U.S. National Archives, some journal articles, and other generally trustworthy sources. However, on more narrow topics, the AI tended to generate citations that the students could not trace or, more likely, never existed in the first place. We discussed with our students how AIs like ChatGPT "learn" about the world through training on internet data and the risks of AIs misinterpreting and promoting inaccurate knowledge.
Regardless of their prior experiences, students were almost all impressed with the ease of use of the AI, yet some also expressed pointed concerns about the broader implications of these technologies. A frequent theme in the comments was excitement about the potential of AI to make reading more efficient, effective, and interactive. One student reported, "This AI is something I've never seen before, and it's AWESOME. As someone who rarely gets out and socializes, having a private tutor in the form of an AI at no cost is insanely innovative and amazing." However, these positive comments were balanced by a more mixed reaction when students considered AI's larger social and ethical implications. Many students acknowledged the real temptation of substituting AI generated-prose for their own work. Others expressed concerns about whether such technologies undermined the core function of education itself. As one student noted: "This technology can potentially assist students in grasping difficult material, however, it also limits the critical thinking and developmental growth fostered by interacting [with the text].... I personally believe that at this tender stage of AI development, it is better off not commercially available to students." Nonetheless, students agreed that this technology was here to stay, and that both teachers and students needed to understand AI better. As another student reflected: "The ethical considerations of this technology are yet to be determined because of how relatively new the technology is. AI won't be going away anytime soon and will only continue to improve with time and its ability to machine learn."
Goals and Outcomes
The primary goals of the assignment were to enhance student engagement, improve comprehension of complex texts, and foster critical thinking about the implications of AI in education. By analyzing the AI's responses to different prompts, students were encouraged to reflect on their own learning processes and become more aware of the potential strengths and weaknesses of AI as an academic tool.
Based on the students' feedback and our observations, several outcomes emerged from the assignment:
Materials
Students in this study used ChatGPT and Explainpaper (which uses ChatGPT and GPT-4). When we conducted our assignments only GPT 3.5 Turbo was available for ChatGPT. For future iterations of this assignment, we could see great value in experimenting with the different outputs provided by GPT 3.5 vs GPT-4, different ChatGPT plugins, or even comparing different LLM apps. Other AI applications could also be used with the same methods. Students accessing these technologies also require access to the internet, a laptop computer, or smartphone.
The background knowledge and skills of students in our classes in using AI was limited. A few students expressed that they had already tried AI programs for completing their homework. Most other students expressed some hesitation at wanting to use an AI program, but appreciated being guided through the use of AI and gaining greater familiarity and ease after the assignment.
Acknowledgments
Many thanks to the developers and co-founders of Explainpaper, Aman Jha and Jade Asmus, and to the developers of OpenAI for ChatGPT.
Students followed this assignment for both their history and English classes. This type of assignment can be adapted across subjects and disciplines and implemented using various plain language translation AI programs.
Students are provided with the following information:
1. Text Selection: Choose a short text (this can be an article, a news story, or any other written material) that you think is interesting or relevant to your field of study.
2. Summarization: Create a brief summary of the text's main points, noting the text's major arguments or ideas.
3. AI Summarization: Upload that text to an AI of your choosing (we often use explainpaper or ChatGPT). Then ask the AI to generate its own summary of the text. Compare your summary with the AIs. What do you observe about the accuracy and completeness of the AI summary versus your summary?
4. Text Explanation: Select a section of the text that you find difficult to understand or that you think could be explained in simpler terms. Prompt the AI to explain that section of the text. Does the explanation make sense? Is it accurate? Bonus: Have the AI explain that section of the text in the form of a scene from your favorite work of popular culture.
5. Fact-Checking: Select a specific person, event, place, or concept mentioned in the text. Ask the AI a factual question about your choice. Such questions could include when or where someone was born, who participated in a particular event, or the appearance of a particular place. Do your own research to double-check the information given. How accurate and reliable was the information provided by the AI?
6. Source Citation: Ask Explainpaper or ChatGPT to provide citations on the topic you are investigating. Then conduct research online to see if you can find the source listed in the citation. How accurate and reliable was the citation?
7. Reflection: Based on your exploration, write a paragraph reflecting on your experience using AI to explore a text. How do you perceive the potential benefits and difficulties of this technology? How do you perceive the ethical considerations of this technology?