Anuj Gupta
University of Arizona
In this assignment, students critically tackle the biases exhibited by large language models (LLMs) like ChatGPT against world Englishes and other marginalized language varieties by using prompt engineering as a means to reprogram these biases for more equitable AI outputs. This assignment, originally developed for my ENGL 430: UX Research Methods class, introduces students to ChatGPT prompts as a digital genre and presents prompt engineering as a critical making activity through which they can iteratively design prototypes of innovative chatbots to tackle social problems they see in generative AI tools. This assignment resulted in innovative designs and thoughtful reflections on AI, language, and ethics.
Learning Goals
Original Assignment Context
ENGL 430: User Experience (UX) Research Methods class at a large, R1 university in Southwest USA.
Materials Needed
Time Frame
Can be adapted in various contexts ranging from 2 week to semester-long projects.
Overview
In exploring the biases LLMs like ChatGPT exhibit against world Englishes and other marginalized language varieties, I was drawn to a critical making approach (Butts, 2022; Faris and Holmes, 2022; Tham, 2021). Extending critiques of ChatGPT’s linguistic biases by scholars like Vee et al. (2023), Owusu-Ansah (2023), Byrd (2023a) and Bender et al. (2021), I wondered: could we, as educators and learners, use prompt engineering to reprogram these biases and foster more equitable AI outputs? This question led me to develop an assignment for my ENGL 430: UX Research Methods class, where I introduced students to the concept of prompt engineering as a tool for mitigating linguistic biases. I showed them how to view ChatGPT prompts as a digital genre, which they could use to critically interact with and reshape AI chatbots. As a model, we used 'Translinguo', a chatbot prototype I created. It responds in various world English varieties, providing students with a hands-on experience in linguistic diversity. The assignment, which I’ve taught once so far, encouraged students to create their own chatbots, using high or low-fidelity prototyping through prompt engineering and resulted in innovative designs and thoughtful reflections on AI, language, and ethics.
Phase 1: Introduction to key terms. To begin, let’s acquaint ourselves with key concepts that we will require for this assignment:
A) Prototyping: Using Liu (2022), we can think of prototypes as draft designs that UX designers create to test with different users before a final product can be launched. These are usually of two types. Low fidelity prototypes or wireframes are simplified representations of digital interfaces. They look less polished and include less details when compared to the finished product. High fidelity prototypes or mockups are more accurate representations of a digital interface. They look similar to finished products and often integrate visual elements like colors, images, layout etc., as well as functionalities like interactions so they can demonstrate the final logic and behavior of a designed product.
B) Critical making: Using Butts (2022), we can understand critical making as a tradition in design thinking that combines academic critical thinking with the designing or making of objects. By making prototypes informed by critical or theoretical goals, critical makers experiment with technologies in a thoughtful and ethical manner.
C) Prompting as an emerging digital genre: Using Gupta (2023a), we can conceptualize prompts as an emerging digital genre using which humans engage in the social action of interacting with ChatGPT. Prompt engineering then is the process of iteratively designing prompts to elicit desired outputs from AI tools while critically assessing the outputs & reflecting on their usefulness. Additionally, prompting techniques are elements that make up rhetorical moves in prompts. To sample some techniques, let’s explore Poe (2023), Saravia (2023), Akin (2023), Gupta (2023b), OpenAI (2023), Mollick and Mollick (2023), Mills (2023), and Watkins (2023). While certain prompts allow us to interact with generic AI chatbots in a spontaneous manner, others can be used to design prototypes of chatbots called prompt bots. To explore some examples of what such prompt bots look like, let’s explore www.poe.com. While prompts and prompt bots can be written for many purposes, by writing prompts that critically engage with existing biases of chatbots or address social problems, we can design prototypes of chatbots that try to use these emerging technologies for social good.
Phase 2: Immersion in Translinguo. Now let’s look at a sample chatbot I created to enact the concepts of critical making, prototyping and prompt engineering.
1. Let’s explore a chatbot that I created called Translinguo by either watching this video or using it on poe.com by clicking here.
2. You can also view this chatbot on the classroom projector. During class, I will ask you to share questions or ideas you want to ask this chatbot and we will look at its outputs together.
3. Through this, let’s try to look for patterns in how the chatbot responds. For example, Translinguo always responds to user queries in a randomly chosen variety of World Englishes. Here is an example:
4. Now let’s read the prompt I composed or engineered to get the chatbot to respond in this way: You are Translinguo, a chatbot specialized in using World Englishes. Your mission is to immerse users in the diverse English dialects from around the world, helping them appreciate the unique variations English has taken on in different regions.To accomplish this, for every user input: a) Compose your response by picking an English variant randomly from this list: Hinglish, Spanglish, Chinglish, Singlish, Manglish, Taglish, Konglish, Runglish, Dunglish, Navajo English, Jamaican English. b) In your response, start by mentioning which variety of English you are writing and in brackets tell where this variety comes from. c) Your response should not use any standard English, no matter what. d) After your response, give line break and provide a translation in Standard English in brackets immediately following the original response. e) For each new user input, choose a new variety of English from the list in step a.
Phase 3: Writing your prompts/designing your chatbots using the following steps. Now it is your turn! Please follow the steps below to complete this assignment:
Step 1: First, describe the function that you would like your chatbot to perform. Why is this function important? Who could it be useful for? Does it address a bias that AI tools have or a social problem that you’d like to solve?
Step 2: Second, decide whether you want to create a wireframe (low-fidelity prototype) or mockup (high-fidelity prototype) of the chatbot.
Step 3: For high-fidelity prototypes, you can use a free platform like poe.com If you are using poe.com, you can either create your own account (make sure to read the privacy agreements before doing so) or you can use an account that I’ve created for the class. Then you can follow these instructions on how to create a prompt bot on Poe.com. Be sure to toggle the public button off for your bot since you are testing it out for now: https://developer.poe.com/prompt-bots/how-to-create-a-prompt-bot
Step 4: For low-fidelity prototypes, you can use either a pen and paper to write a prompt and design what the chatbot would look like. Alternatively, you could also use a visual sketching software like canva.com. If you are new to using canva.com, you can use these instructions on how to get started: https://www.canva.com/learn/how-to-canva-beginners-guide/
[Note for teachers: For accessibility and privacy reasons, it is important to offer such options to students who might not be comfortable creating accounts on chatbot apps or websites.]
Step 5: Create your chatbot prototype. First, write a prompt using ideas from the samples shown in class. Feel free to test different versions based on outputs that you see. If you are using poe.com, you put it directly into the chatbot there and see how the chatbot responds. If you are using pen and paper or canva.com, then put the prompt next to the visual chatbot you design.
Step 6: In a discussion post, share your chatbot prototype (wireframe/mockup) through screenshots, videos, or hyperlinks, along with a short description of the function you designed it to perform and a reflective note on what you would like to do to improve this design in future iterations.
Akın, F. K. (2023). Awesome-chatgpt-prompts [HTML]. https://github.com/f/awesome-chatgpt-prompts (Original work published 2022)
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922
Booten, K. (2023). Synthetic Metacognition—Iterating Prompts with GPTs. In A. Vee, T. Laquintano, & Schnitzler (Eds.), TextGenEd. https://wac.colostate.edu/repository/collections/textgened/rhetorical-engagements/synthetic-metacognition/
Butts, S. (2022). Critical Making. In J. C. K. Tham (Ed.), Keywords in Design Thinking: A Lexical Primer for Technical Communicators & Designers (pp. 73–77). The WAC Clearinghouse; University Press of Colorado. https://doi.org/10.37514/TPC-B.2022.1725.2.12
Byrd, A. (2023a). Truth-Telling: Critical Inquiries on LLMs and the Corpus Texts That Train Them. Composition Studies, 1, 135–142.
Byrd, A. (2023b). Using LLMs as Peer Reviewers for Revising Essays. In A. Vee, T. Laquintano, & Schnitzler (Eds.), TextGenEd. https://wac.colostate.edu/repository/collections/textgened/rhetorical-engagements/synthetic-metacognition/
Eldin, A. (2023). Genre Generators. In A. Vee, T. Laquintano, & Schnitzler (Eds.), TextGenEd. https://wac.colostate.edu/repository/collections/textgened/rhetorical-engagements/synthetic-metacognition/
Faris, M. J., & Holmes, S. (Eds.). (2022). Reprogrammable Rhetoric: Critical Making Theories and Methods in Rhetoric and Composition. Utah State University Press. https://doi.org/10.7330/9781646422586
Gupta, A. (2023a). Exploring ChatGPT Prompts as an Emerging Digital Writing Genre – Exploring AI Pedagogy. In MLA-CCCC Joint Task Force on Writing and AI (Ed.), Exploring AI Pedagogy: A Community Collection of Teaching Reflections. Humanities Commons. https://exploringaipedagogy.hcommons.org/2023/11/14/exploring-chatgpt-prompts-as-an-emerging-digital-writing-genre/
Gupta, A. (2023b). Robopoetics: AI prompt engineering and ethics for design students and creative writers [Invited Workshop]. Anant University. https://www.canva.com/design/DAFqsP2yMC0/9ktphm9VMVI2oo8CBlRv3w/edit
Lewis, & Wayland. (2023). Writing Against the Machine Debating with ChatGPT. In A. Vee & T. Laquintano (Eds.), TextGenEd. https://wac.colostate.edu/repository/collections/textgened/rhetorical-engagements/writing-against-the-machine/
Liu. (2022). Wireframing. The UX Cookbook. https://theuxcookbook.com/Wireframing-2f98c1760c244ad8833b9c9d72a4ec77
Mills, A. (2022). AI Text Generators and Teaching Writing: Starting Points for Inquiry—The WAC Clearinghouse. https://wac.colostate.edu/repository/collections/ai-text-generators-and-teaching-writing-starting-points-for-inquiry/
Mollick, E. R., & Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts (SSRN Scholarly Paper 4475995). https://doi.org/10.2139/ssrn.4475995
Owusu-Ansah, A. L. (2023). Defining Moments, Definitive Programs, and the Continued Erasure of Missing People. Composition Studies, 51(1), 143–148.
Poe. (2023). Text generation. https://developer.poe.com/prompt-bots/best-practices-for-prompts/text-generation
Saravia, E. (2023). Prompt Engineering Guide. DAIR.AI. https://www.promptingguide.ai/
Tham, J. C. K. (2021). Design Thinking in Technical Communication: Solving Problems through Making and Collaboration. Routledge, Taylor & Francis Group.
Vee, A., Laquintano, T., & Schnitzler, C. (Eds.). (2023). TextGenEd (Vol. 1). WAC Clearinghouse. https://wac.colostate.edu/repository/collections/textgened/
Watkins, M. (2023, May 8). AI Feedback: An Imperfect Digital Mirror [Substack newsletter]. Rhetorica. https://marcwatkins.substack.com/p/ai-feedback-an-imperfect-digital