WAC Clearinghouse home page
About ATD   Submissions   Advanced Search

Writing Technologies and WAC: Current Lessons and Future Trends

Seduction or Productivity: Repurposing the Promise of Technology

Abstract: In tight financial climates, emerging technologies loom as enticing options to administrators: efficient and cost-effective solutions to both operational and pedagogical problems. Disciplinary knowledge, frequently hailed as the pinnacle of cultural capital in the academy, does not hold absolute power in rhetorical situations in which the material conditions, competing research paradigms, conflicting needs, institutional history, and the promise of technology come into play. This article illustrates just such a conflicted rhetorical situation and shows how writing faculty used the promise of research, faculty development, and teacher education, all important currencies in academe, to cooperate with administrators' goals, and to derive benefits for the writing program and writing faculty. The project described tested two highly-promoted commercially-available writing technologies in local classrom settings via a small, manageable research project to give us systematically collected and interpreted evidence with which we could counterbalance anecdotal evidence about student performance, recitations from product literature about the promise of technology, and national distress about hte most recent incarnation of a literacy crisis.

Mid-level academic administrators are charged not just with the task of managing daily operations of their academic units (a task with enough challenges to fill anyone's day), but also with taking the lead on new initiatives, with shaping, guiding, and changing the direction of their units to improve efficiency and effectiveness, and to do it all as they trim costs to fit their ever-shrinking budgets. In such a financial climate, emerging technologies can loom as enticing options to administrators, as efficient, cost-effective solutions to both operational and pedagogical problems. Even when technologies are largely untested in situ, they can be quite alluring, especially if the need is great and if the technologies' development and design come certified by plausible research. For-profit producers of academic technologies certainly recognize a vast economic opportunity in the higher education market, and they are quite adept at portraying their products as sleek problem-solvers with evidence that speaks from a discourse that academics understand and value. They market their products well.

But what happens if administrators in their honest efforts to solve problems turn to technological solutions that cannot fulfill their promise, such as automated essay scoring software? Initiatives that fail can damage trust in programs, as well as in the individuals who lead them. The viability of broad-reaching academic programs (like WAC and WID) depends greatly on the trust and confidence that both the faculty and the programs inspire in others across campus. Similarly, the careers of those faculty members and administrators involved can be tarnished by failed initiatives, and as their credibility is diminished, so too can be their ability to get things done on campus and in their own professional lives. Most importantly, failed initiatives waste the time and resources of students, faculty, and administrators and risk both the viability and value of students' education.

Yet, refusing to cooperate with initiatives can be equally devastating to a program or career, particularly when those initiatives come from above. Deans, provosts, and chancellors can make even the most securely tenured professor's life difficult by blocking promotions, diverting funding for professional development, staff, or facilities, or by reassigning him or her to undesirable positions, units, or projects when faculty are perceived as antagonistic. When faculty members find themselves caught in such contexts, they must be able to take a deep breath, step back, and evaluate the larger picture, the larger rhetorical situation in which the initiatives occur, in order to find the opportunities that will let them address the needs as they've been identified and lead to solutions that the varied constituencies can buy into. In late spring 2007, we found ourselves in just such a conflicted rhetorical situation, and, through these pages, we wish to portray it in enough particularity to help others analyze similar situations they might find themselves in to sort out the rhetorical, political, and pedagogical complexities more easily.

This kind of particularity is valuable, according to Robert E. Stake, a scholar of qualitative research methods. Well-known in education communities, Stake (1978) argues, "Truth in the field of human affairs is better approximated by statements that are rich with the sense of human encounter. . . . Generalization may not be all that despicable, but particularization does deserve praise" (p. 6). Particularization is "what becomes useful understanding" especially as one "recognize[es the particular] . . . in new and foreign contexts. That knowledge is a form of generalization too, not scientific induction but naturalistic generalization, arrived at by recognizing the similarities of objects and issues in and out of context and by sensing the natural covariations of happenings. To generalize this way is to be both intuitive and empirical, and not idiotic" (Stake, 1978, p.6).

A Rhetorical Situation

In the bustle of the final week of the 2007 spring semester, our College Dean and Associate Dean approached the Writing Program Administrator with their three-year plan to improve writing instruction and student writing performance in the University, a plan in which writing technologies figured heavily. Already fully formed in broad strokes, the plan was in urgent need of fleshing out because the deadline for a grant they had decided to seek was just weeks away. Thus ensued a flurry of meetings with the deans, the WPA, the assistant WPA, and others. Since the WPA was in only her second semester on campus in a newly lined position, she was in the process of carving out her responsibilities and getting to know the campus systems and people and had not yet time to assess with any rigor the state of writing on campus. But for years, the deans had listened to complaints from faculty members who claimed that students could not write. They were driven by that history, and something had to be done about it, immediately.

Not surprisingly, these complaints pointed to errors in the surface features of writing, and writing with such errors was characterized as "poor." The complaints, as well as the deans' experience as faculty who assigned writing, contributed to their perception of the problems on campus. For example, the Associate Dean, who used writing assignments faithfully in her upper division biology classes, found it difficult to focus on higher order concerns in her students' writing until it was virtually error-free. Aware, experientially, of how much time this grading took, she was frustrated and confident that she was not alone. While both deans wanted to encourage faculty to use more writing assignments in their classes and thereby help improve the quality of student writing on campus—goals we as writing program administrators value—they also wanted to reduce the burden of grading on faculty. They saw automated essay scoring software as a promising solution.

Disciplinary Background

Complaints about faculty workload and student writing are familiar to compositionists. David Russell (2002), Mike Rose (2005), and others have demonstrated that, historically, public outcries about "illiteracy" almost always produce the same complaints about student weakness. These scholars note that literacy crises are cyclical, that they have a history that reaches at least back into the 19th century, and that they typically have followed significant historical or political events and/or socio-economic changes, such as the sudden influx of students that occurred when the first state universities opened in the Midwest following the Morrill Act of 1862, or when women began entering universities, or when the GIs flooded colleges and universities after the Second World War, or when veterans returned to college after the Viet Nam War in the 1970s. With every such change, the demographic make-up of student populations shifts in a new direction, and the social, economic, educational, and textual conventions that had marked literacy for the earlier generation no longer fit the new one. In 2007, the echo of "Why Johnny Can't Write" from 1975 sounded louder than ever, as the deans knew from current national reports.

The Neglected "R": The Need for a Writing Revolution, a 2003 report from the College Board, indicated we are in the midst of another crisis. It claimed that school reform in general had neglected writing instruction and needed to reverse that trend and begin promoting writing as an "essential" skill for success in school. In its 2004 report on surveys of leaders in American corporations and state governments, the Commission again urged school reform to promote writing instruction. The Commission's 2005 survey of state officials across the country claims that many college-educated state employees can barely write sentences that make sense and that "English composition seems to have fallen off the list of things that count in college" (22). Views about the cause of this crisis vary. Some believe that faculty (and not just writing faculty) have failed to challenge students to write, and so students do not write and read as much or as often in school as they used to (Musgrove, 2006). Others believe that writing programs, their courses and faculty, have been simply undervalued and ignored, and the consequences are insufficient curricular reform and an inability to draw the best teachers to the teaching of writing (Bartlett, 2003). Others believe that writing instructors have let "standards" slide (Rochester, 2004). And others of us see how literacy crises cycle through our national consciousness and wonder, with good reason, if the current cries of alarm have targeted causation accurately.

That the problems the College Board names are much the same as those decried by our deans and faculty colleagues is not surprising. The surface of writing has been always easier to point to as problematic, and the assumption that these surface features should follow codified rules (like "natural laws") has been always easy to make. Trained composition teachers realize, of course, what goes on "below" the surface of student writing—such as the development of critical thinking processes, rhetorical processes, and ethical processes—but to the untrained eye seeing below the surface of student writing is difficult and can only be done after surface error is corrected.

The attention and importance our society gives to the surface features of writing actually arises out of complex socio-political circumstances. Robert J. Connors (1985) explains that the concern with grammatical correctness and the surface features of writing in the United States can be traced back to the 19th century when America was undergoing transition out of its state as a cultural backwater. By mid-century, the literate intellectual elite in the Northeast felt it was the equal of any in the world, and it exerted its influence on the growing, poorly educated population that was moving west. During this period, practice exercises came to dominate more and more textbooks, and these exercises aimed at the perfect replication of spelling, capitalization, punctuation, and grammar (Connors, 1985, p. 62). In fact, as Connors (1985) points out, the history of college composition instruction in America can be seen, at least until perhaps the last 50 or 60 years, as the history of a dominant focus on mechanical correctness (p. 65).

Since the second half of the 20th century that emphasis has more or less changed. Research in composition and linguistics, the reprise of rhetoric in its key role in composition, and most certainly the process movement of the late '60s and '70s, has taught compositionists to emphasize other components of writing (e.g., invention, audience) and thus contextualize the function of mechanical correctness much differently. Some research even suggests that too early a focus on mechanical correctness in writing instruction can be detrimental to invention and critical thinking processes (Rose, 2005).

A Rhetorical Situation, Continued

Disciplinary knowledge — historical, pedagogical, scholarly — was allowed no place in our rhetorical situation. Any attempt to bring to our discussions with the deans the disciplinary knowledge we knew — about changing pedagogical practices, our commitment to writing-as-learning, writing-as-inquiry, and writing as a tool of critical thinking and cognitive development, and situating error correction within broader textual, contextual, and rhetorical discussions — seemed to indicate that we were resisting their efforts and resisting change, a resistance they thought the English department had displayed for many, many years. They were inclined to stick to their plan for wholesale course redesign that would implement computer-based writing instruction in three University writing courses: the required first- and third-year writing courses and a yet-to-be-designed basic writing course, their template for which resembled the programmed learning texts of the 1970s, this time with computer programs of drill and practice replacing the print texts.

Those of us who administer writing programs know the complexity of issues such widespread change would entail, and those who have studied the use of technologies in writing instruction know the limitations of automated essay scoring software, but "change without any new costs" was the deans' agenda, and automated instruction seemed to offer them a financially expedient way to meet their goals. Bolstering their arguments was evidence from several sources.

The deans' redesign was based on a promising and attractive model taken from Carol Twigg's (2005) National Center for Academic Transformation "Program in Course Redesign." The report concluded from its study of 30 institutions that technology-based instruction enabled those institutions to reduce costs at an average of 37% while at the same time improve student understanding of course content, an appeal hard for anyone to resist—particularly for our deans because of similar successes they had achieved in revamping a required math course and lower-level foreign language courses (as we discuss below) and because of the budget deficits they faced in the college. The promise of technology as a solution was (seemingly) too well supported for us to persuade the deans to discuss problems we anticipated or to consider alternatives.

Supporting the deans' position further was the fact that the NCAT's report had been lauded in the influential Spellings Commission Report of 2006, which made two relevant recommendations: first, to increase the amount of writing in the curriculum, and second, to take advantage of innovations in technology to improve the quality and access of higher education. The Commission's (2006) report also asserted that the rising costs of higher education "have outpaced inflation for the past two decades and have made affordability an ever-growing worry for students, families, and policymakers. Too many students are either discouraged from attending college by rising costs, or take on worrisome debt burdens in order to do so" (p. 18). With backing from national reports and models, the deans were well-supported in their planned course redesign.

Their confidence was buoyed, as well, by their considerable success when they had redesigned the curriculum of College Algebra. In 2002, the student success rate (based on a C- passing grade) hovered at 55% on our campus. In 2003, the deans began a wholesale course redesign using a model from the NCAT report that emphasized the use of technology to enhance instruction. The model encouraged continuous assessment of student progress and techniques, like practice and instant feedback loops, to keep students focused on completing their assignments successfully. The course structure of lectures was abandoned and replaced by students' independent work with problems delivered via this technology and the offer of tutorial support in a math lab. The drill-and-practice approach worked very well: by 2006 success rates in college algebra rose to over 75%. This kind of success would make any administrator confident to try it again.

So the deans turned to the Foreign Language Department to improve language acquisition in the required courses it offered. The plan to redesign the introductory level courses met with staunch resistance from tenured and tenure-line faculty. Eventually, the department chair left and the department began to fall apart, and the Dean moved the tenure-track faculty into other departments where they could be protected and earn tenure. Then, he created a Center for Languages and Cultures comprised of the non-tenure-track Foreign Language faculty who agreed to concentrate on redesigned introductory language courses. It was among these non-tenure line faculty members that the deans urged and supported the use of new technologies for language instruction. Since then, considerable curricular changes have resulted, exciting ones. Faculty members teaching the required language courses have become rejuvenated, and since its creation, the Center has become one of the most innovative academic units in the College. Although changes in Foreign Languages and math were enacted with the best of intentions, they made us uneasy, particularly in light of the deans' conviction that the English department was "out of touch." Even though both deans acknowledged that writing was "different" from math and foreign language, any resort to disciplinary expertise in response to those differences seemed nothing but resistance to change.

How could we respond? We perceived distrust, disregard, and deep misunderstandings on both sides. Quickly, we realized that settling into a standoff (even based on disciplinary knowledge, the cultural capital of academe) would help no one, deans, faculty, students, or budget exigencies. Pressed as we were, we wanted to address the deans' interests proactively and constructively, and that meant stepping away from the emotional pressure of the moment. We reasoned that the rhetorical situation required us to abandon any attempt at resistance and to demonstrate good will and cooperation with the deans' plans. We thus repurposed the promise of technology. We reasoned that because the campus had recently reaffirmed its commitment to research and because the Dean himself was a strong and constant proponent of faculty research, the likelihood of his support for a research project that tested the technology was good, so we recast our predicament as a research opportunity that could serve our own interests: We would implement the essay scoring software on a very small (and controllable) scale. This quick implementation would precede the course redesign (if funded) and would allow us to collect evidence of the software's effectiveness, evidence that would be persuasive to the deans (regardless of the results) in ways that we were absolutely unable to be at that moment. We knew we had the opportunity to demonstrate the professionalism and research-based nature of composition, which we hoped would build credence for any future claims we made about writing instruction on campus. We thus built our ethos via two forms of cooperation, the quick (and responsive) small research project and further participation in producing the grant proposal.

We foresaw as well the potential for other benefits. For years, our writing courses have been taught by a core of non-tenure line faculty who have been teaching the same writing classes over and over for years and to whom we wanted to offer avenues for professional regeneration. While they were not trained in composition and rhetoric pedagogy, we speculated that we might help them see their professional identities a bit differently if we conducted and publicized the research project effectively and perhaps eventually involved them in it. We thought a significant research project in a writing course could also augment the program rejuvenation we anticipated in response to the hiring of three bright new teaching professors who would join us the following fall.

Technology Repurposed

We knew from experience and from the considerable body of literature on computer-assisted writing instruction that the drill-and-practice pedagogical approach that had been so successful in College Algebra was not effective in teaching writing. We also knew that the technologies the Associate Dean had settled on, the ACT's Compass and ETS's Criterion, were dressed in highly seductive rhetoric. Our understanding of Natural Language Processing and Artificial Intelligence technologies indicated that both software systems were still somewhat primitive and limited, but far more advanced than the software discussed in the literature from studies in the '80s and '90s. We reasoned that even if Compass and Criterion failed to achieve what they promised about scoring effectiveness and the savings in faculty time, which seemed likely, a small, manageable research project would give us systematically collected and interpreted evidence, to counterbalance anecdotal evidence about student performance, recitations from product literature about the promise of technology, and national distress about the most recent incarnation of a literacy crisis. This evidence had the potential to facilitate more productive future campus dialogues. Our problem-solving strategies borrowed heavily from the strategies scholars in Computers and Writing have used for years to merge technology into the writing curriculum. They have long recognized that computing software is constrained by the specific uses it is designed to address and that adapting it to better suit writing pedagogies requires teachers to subvert (or re-purpose) that software for local use. We took the same approach to the software recommended by the deans' plan.

Our small study would observe, evaluate, and assess the pedagogical and logistical effectiveness of Compass and Criterion in a summer FYC course taught by one of our advanced and more technology-oriented graduate students. Joining her was another advanced graduate student who would act as a classroom observer, advisor, and data collector. The teacher would learn how to integrate the technology into her curriculum and identify what parts of it were useful and the ways in which they could be used (we expected her to re-purpose or subvert, as necessary). She was also to identify any difficulties she or her students had with the technologies and how or if she could overcome them. The second graduate student was to observe each class meeting, keep detailed notes, consult closely with the teacher, and do extensive reviews of the literature on machine scoring of essays, the technology product literature, and the literature on automated essay scoring systems.

As suggested above, the Dean's office had not asked us to conduct this preliminary research nor envisioned such a corollary to the grant design. We invented this project out of our own needs, in proactive response to the rhetorical and pragmatic exigencies we faced, and it changed the rhetorical situation dramatically. By taking a leadership role in the initiative, we regained a degree of control over the instructional and critical ways technology would be used in our writing courses, over the extent to which an instructor and students would use it, and over the making of knowledge in our discipline. Importantly, we inserted this project into the immediate time and space—while the deans' grant proposal was under review. Its results, which would be available by the time the grants were awarded, would help support and direct us if in fact we moved into the stage of technological redesign the grant proposed. We also demonstrated that whatever recalcitrance to change the English department might have shown (or seemed to have shown) in the past was lessening, if not over. Moreover, we were able to turn the situation into an educational experience for our graduate students who were learning first hand and in deep context the goals and methods of composition scholarship and the influence political conditions can have upon research.

The Outcomes

First, we should note that the grant was not funded, so the primary writing program changes came from our self-initiated project, one we'd likely not have undertaken at that time without pressure from the Dean's office. In addition, we were generally encouraged by our attempts to implement the software products we tested. While they are, as we predicted, still quite primitive and limited, our teacher found that with some adaptations the software might be shaped and used to promote good writing habits (namely, proofreading). In the next semesters, we expanded the project to include other writing faculty members and additional classes (at the junior level). This expansion has enabled us to offer professional development opportunities to faculty members that would not have been possible otherwise. Moreover, the project gave occasion for extended conversation among writing program members and thus encouraged a focused kind of community reflection about teaching. This secondary project has given us reason to be hopeful as we look forward to expanding the project even more widely to include professors and instructors from other disciplines.

Conclusion

It's not difficult to imagine how our rhetorical situation might have developed without the problem-solving strategies we put into motion in a very short time under considerable duress. Our campus, like many other campuses, is rife with stories of battles over power and who had it, who lost it, and who wanted it, with stories of individuals standing toe-to-toe in mutual disagreement, threatening and parrying and countering. This situation could easily have become one of those, and we could easily see ourselves in a situation of heightened conflict had we merely resisted the deans. We believe, however, that the predicament we faced in the summer of 2007 was not simply about power, though it was about making change and about who and what would be the agents of change. It was also a frustrated, but honest, demand for improved student writing with which we could not disagree – even though our definitions of "writing" and "improvement" were not completely consonant with the deans'. By responding to the deans' plan to address that demand with a two-pronged strategy of cooperation, we were able to create a different rhetorical situation, one that will serve us well as we turn this initiative in a great many positive directions. By restructuring that rhetorical situation, we were able to see more clearly that what looked like a personal threat was not really personal at all. Had we not done so, we might easily have found ourselves in a far less desirable place.

References

Bartlett, Thomas. (2003, January 3). Why Johnny can't write, even though he went to Princeton. The Chronicle of Higher Education, 49(17), p. A39.

Connors, Robert J. (1985). Mechanical correctness as a focus in composition instruction. College Composition and Communication, 36(1), 61-72.

College Board, National Commission on Writing. (2003). The neglected "R": The need for a writing revolution. Retrieved December 10, 2008, from http://www.writingcommission.org/prod_downloads/writingcom/neglectedr.pdf

College Board, National Commission on Writing. (2004). Writing, a ticket to work . . . or a ticket out: A survey of business leaders. Retrieved December 10, 2008, from http://www.writingcommission.org/prod_downloads/writingcom/writing-ticket-to-work.pdf

College Board, National Commission on Writing. (2005). Writing: A powerful message from state government. Retrieved December 10, 2008, from http://www.writingcommission.org/prod_downloads/writingcom/powerful-message-from-state.pdf

Musgrove, Laurence. (2006, April 28). The real reasons students can't write. Inside Higher Ed. Retrieved December 10, 2008, from http://www.insidehighered.com/views/2006/04/28/musgrove

Rose, Mike. (2005). Lives on the boundary: A moving account of the struggles and achievements of America's educationally underprepared. New York, NY: Viking Penguin Group.

Rochester, J. Martin. (2004). Class warfare: Besieged schools, bewildered parents, betrayed kids and the attack on excellence. San Francisco: Encounter Books.

Russell, David R. (2002). Writing in the academic disciplines: A curricular history (2nd ed.). Carbondale: Southern Illinois University Press.

Sheils, Merrill. (1975, December 8). Why Johnny can't write. Newsweek, 92, 58-65.

Stake, Robert. (1978). The case study method in social inquiry. Educational Researcher, 7, 5-8.

Twigg, Carol A. (2005). Increasing success for underserved students: Redesigning introductory courses. Report from the National Center for Academic Transformation. Retrieved December 18, 2008, from http://www.thencat.org/Monographs/IncSuccess.htm

United States Department of Education, Spellings Commission. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, DC: U.S. Government Printing Office. Retrieved December 18, 2008, from http://www.ed.gov/about/bdscomm/list/hiedfuture/reports/final-report.pdf

Contact Information


Complete APA Citation

Klein, William, & Duffey, Suellynn. (2009, January 19). Seduction or productivity: Repurposing the promise of technology. [Special issue on Writing Technologies and Writing Across the Curriculum] Across the Disciplines, 6. Retrieved from https://wac.colostate.edu/atd/technologies/klein_duffey.cfm