Abstract: This essay presents results from a comprehensive study of writing in the undergraduate curriculum at the University of Pittsburgh. It is in two sections. The first section reports on interviews with faculty from across the disciplines in the School of Arts and Sciences; the second reports finding from student focus groups and from an extensive student survey.
In Spring 2009 we presented the Pittsburgh Study of Writing to the faculty at the University of Pittsburgh. This was a study of writing in the undergraduate curriculum on our campus and, in particular, in the School of Arts and Sciences. The project began in 2003; we learned much along the way—much about undergraduate writing on our campus and much about the pleasures and pitfalls of a campus-wide research project. And there is much in what we learned, we believe, that may be useful to faculty, students, and program directors on other campuses. With that goal in mind, we have prepared two sections of the report for Across the Disciplines—a section based on faculty interviews and a section based on student surveys and focus groups. (The full report can be found at: http://www.academic.pitt.edu/assessment/pdf/Pitt-writing-study.pdf.)
What follows is not the kind of research article usually found in Across the Disciplines. It is different, for example, from the essays presented in the special issue, Writing Across the Curriculum and Assessment: Activities, Programs, and Insights at the Intersection (Across the Disciplines, Vol 6, 2009). We are presenting the results of an assessment project rather than an account of how it was done or an argument for its methods. Although this was a local study, prepared for a local audience, we believe that it can be generally useful for the following reasons:
This project began several years ago, in Fall 2003, before our campus had fully committed to outcomes assessment, but when assessment was much in the air. Our Vice Provost called the English department chair (Bartholomae) to say that while the Provost knew we had a strong and highly regarded writing program, he wondered what measures we had to test and to demonstrate its effectiveness. And, he asked, "was I at all interested in an assessment project?" The Vice Provost, Jack Daniel, was an old friend and made it clear that this was a loaded question, that a wise chair would say "yes, of course," that an assessment of student "outcomes" was on its way, and that my department had a chance to get out ahead of this juggernaut as it made its way to our campus.
I said that I was and I wasn't interested in an assessment project. I said that in the Arts and Sciences we had developed a curriculum that promoted student writing across the four years of undergraduate study. The curriculum produced these outcomes—students were writing regularly and with close instruction in a first year writing course and then in two writing intensive courses, one of them in the major. This writing would not necessarily be produced otherwise, and certainly not in the forms that had been developed through our WID program (which made revision the primary point of instruction and sequencing the basic format for a writing course). Outcomes assessment, then, could be descriptive.
I said that we would not learn much from an SAT-like test of general writing ability, an instrument that was being peddled across the nation at the time; I said that what we needed was detailed knowledge of what was happening on the ground. We needed something like an ethnographic study of the "culture of writing" on our campus. I said, "I have been a member of this faculty for 30 years and for 30 years directly engaged with student writing, and yet (with a few exceptions) I could not provide a detailed, documented, or comprehensive account of what goes on in the courses taught outside the English department—in History or Economics or Chemistry." Nor did I know in detail what the faculty, in general, think about these courses or their students or their students' writing. And so, with Beth Matway, who leads our WID program, we proposed a study designed to learn what we could about the culture of writing in the undergraduate curriculum. To make this manageable, we decided to focus for the time being on undergraduate writing in the School of Arts and Sciences. Our Dean, John Cooper, agreed to the plan and we were off and running. (I should quickly add that we were fortunate to have a Dean who understood the importance of writing as a part of intellectual work at any level, and who was truly interested in a detailed account of writing in Arts and Sciences.)
In announcing the project, the Vice-Provost said that the study should document the "core values and practices associated with writing at Pitt." Our goal was to bring forward the experiences and expectations of students and faculty, to hear what they had to say in their own terms and their own voices. And this, we believe, was the great strength and pleasure of the report. It included, verbatim, comments from a wide variety of students—some pleased with their courses, some not so pleased. Their comments were pointed, thoughtful, eloquent, sometimes critical, always useful. The report also included, verbatim, comments from faculty colleagues across the disciplines, who provided detailed accounts of their courses—their writing assignments; their methods for responding to student writing; their expectations; their frank assessments of what their students do with ease and where they struggle. We had said at the outset that one goal of the study was to give voice to students and to faculty and to make their daily practice as writers and teachers visible.
And so we were pleased to see that, once the report was released, what travelled most widely and most quickly were particular statements by students and their teachers. Actually there were two areas of the report that travelled widely and quickly across campus: these statements by students and faculty, but also lists of faculty members and their courses. The project included a survey, and in this survey we asked students to list the courses that had been most useful to them as writers and to list faculty who had made a difference. We organized both lists by department, and these were quickly picked up by department chairs (and by the Dean), slipped under doors and passed from hand to hand as colleagues looked to see how they and their departments fared in comparison with others.
But we are getting ahead of ourselves. Let us briefly describe the project. Then we will present some selected findings.
The study had four primary areas of inquiry, each with its own instruments.
We did an inventory of existing courses and requirements. From the record of course descriptions across Arts and Sciences, we wanted to learn what we could about writing beyond the two required writing intensive courses (w-courses). We also wanted to look closely at the existing writing intensive courses, to document the various ways departments made sense of the requirement by developing w-courses as part of advanced study in their disciplines. We summarized other assessments of writing at Pitt.
We conducted a series of focus group meetings with students drawn from upper-division, writing intensive courses. We asked the participants what they had learned, as writers, in their courses at Pitt; how their writing had changed; how they understood faculty expectations; we asked about their best and worst experiences. We learned a lot. And we used what we learned to design a survey.
We designed and administered an on-line survey in the Spring Term, 2005. (For this, we relied on substantial support from our University Center for Social and Urban Research.) 1000 juniors and 1000 seniors were invited to take part. The response rate was 32%, a relatively high rate of response for surveys of this type. We had representative participation across the three primary areas in the Arts and Sciences: Humanities, Social Sciences, Natural Sciences. The survey was designed to give us a detailed account of the genres of academic writing in use on campus, and of the ways our students' understood and valued these genres. We gathered information about writing and teaching both in and out of writing intensive courses. And we asked students, in general, about the importance of writing to their education and future careers. 90% said that writing was important, very important or extremely important to their education at the University of Pittsburgh; over 1/3 said it was extremely important.
We also had a section of open ended questions. In the report, for example, we provided a list of courses and teachers that students said "made a significant difference" to their education as writers. And we asked, in more general terms, for comment on the courses they had taken or for advice they would offer the faculty--and the report features what we heard. (One student said, "You should have asked us if we thought long papers due at the end of the semester are useful—they aren't!!") These student comments were thoughtful, interesting and useful. We were impressed with the time they took to tell us what was on their minds.
And we interviewed 27 members of the faculty, nine from each of the three divisions: Humanities, Social Sciences and Natural Sciences. We developed the pool by writing to Department Chairs and asking for a list of faculty from across the ranks whose courses included writing and/or who thought of themselves as particularly interested in or concerned about student writing. The report summarizes the interviews by highlighting common themes or issues. Our report, then, provided a rich menu of best practices from our colleagues across the disciplines.
Below, we will provide a more detailed account of what we learned from the student focus groups, from the student survey, and from the faculty interviews. These are excerpts from our report to the campus community.
In Fall 2004 we wrote to department chairs in A&S asking them to identify up to 5 members of their faculty from across the ranks (including Lecturer) whose courses included writing (in whatever forms). We were looking for people who thought of themselves as writing teachers. From this list we created a pool of candidates representing departments across the disciplines. We planned for 30 and we completed interviews with 27 members of the faculty equally distributed across the three areas: Humanities, Social Sciences, Natural Sciences. 24 of the 27 were tenure track faculty (some very prominent professionally and on campus); the other three were lecturers, one in each of the three divisions.
Although we had a core set of questions, the interviews took their own shape and direction; they lasted from 30 minutes to 90 minutes. The interviewers had course materials on hand. We wrote up each individual interview and returned the transcript to the faculty colleague for comments or corrections. As we began to think about the full range of interviews, the study became a study of key words, and so we then organized our findings according to the primary themes or concerns that emerged across the group, and we created subdivisions as they seemed useful and/or appropriate. We will list those, with commentary, below. The text is drawn from the final report. The language we use, and the key terms, all came to us from the interviews themselves. We used colleagues' names, of course, when we released the report on campus. We were eager to identify best practices across departments. Here we use initials.
Most instructors we interviewed agreed that the quality of student writing, and the quality of students' preparation for a course with writing, have improved over the last decade. Most felt that A&S students came to their advanced courses with appropriate skills, while acknowledging that students come (as people come) with a range of abilities, some writing with apparent ease and others struggling. A. O. (Religious Studies) reflected on this range in remarking that with some "naturally gifted" writers, he can work on "subtle improvements in style," while with others he teaches at a "more basic level." In general, however, faculty members agreed that most students in advanced courses are ready to make progress in their writing. With faculty support, students can move beyond prepared forms (such as, they said, the 5-paragraph theme, the Term Paper, the Report, the 5-page critical essay), and overcome common bad habits (such as "empty prose" or the broad generalization that "supports all claims and requires no evidence").
Some teachers spoke about the necessity of requiring students to attend to the details of proofreading and correction. M. G. (History) proposed an "institutional policy promoting the use of Standard English."
Almost everyone we interviewed mentioned "clarity" and "coherence" as important qualities in student writing, and almost everyone said that these qualities were often lacking. Thinking about what students need to learn, K. S. (History of Art and Architecture) remarked, "I find that they're not writing as precisely as they need to be." Several faculty members noted that students struggle to accurately describe and represent what they see, study or read. They have not, that is, yet learned to use the lenses or optics of a particular discipline; they work, rather, through a more general cognitive/intellectual lens–often "describing" in terms of expectation, habit or cliché.
Many faculty members saw the lack of clarity and coherence in student writing as an intellectual problem: students, they say, need to learn how to "focus." Faculty commented that students do not get to the point; they have too many ideas working at one time; they are overwhelmed by all that they have read or learned; they do not know how to eliminate extraneous information. They need to learn to choose and select and focus on a single issue or question. They need to frame a problem, to summarize and to justify their findings. And they need to do this economically–that is, they need to know what can be left out.
Faculty members used a variety of phrases to describe what constitutes coherence in student writing. They spoke of the need for students in their discipline to "construct an argument" or "develop a narrative," to "organize" or "structure" a whole piece of writing, or to move from beginning to end in a "logical" manner. It appears that although faculty colleagues agree on the value of coherence, it takes different forms in different disciplines. In the context of advanced courses and the intellectual demands they make on students, the virtues of clarity and coherence do not travel well; they are not easily portable or generalizable. For example, a "clear and coherent" narrative in a history class is different from a "clear and coherent" account of an experiment or a "clear and coherent" analysis of a set of readings in political theory. What was most often represented in faculty concern for "clarity" and for "coherence" was the desire to see students master specific materials and represent those materials appropriately within the expectations of the field.
As students work with complex materials in advanced courses, the lack of clarity and/or coherence in their writing may represent an uncertainty about disciplinary methods and expectations. It is also evidence of students struggling to get intellectual purchase on complicated material.
Alongside their concern for clarity and coherence, some faculty asserted the value of complexity in student writing. They want students to develop the ability to handle multiple sources, ideas, or points of view in a single piece of writing. Perhaps the most difficult task for student writers is to negotiate what appear to be competing demands for "clarity" and "control" on the one hand, and "complexity" and "exploration" on the other. As they move beyond the simpler texts they have learned to control, they find themselves struggling to manage complexity.
A common thread across the interviews was a concern to make writing matter, to make it more than a routine and predictable classroom exercise, to present a writing assignment as something other than one more hoop to jump through en route to graduation.
Many we interviewed use writing as a way to compel students to extend their thinking. Short paper assignments may ask students to prepare an argument or develop a thesis not explicitly developed in class; to articulate what "they think" about a particular argument or issue; to apply a theory taught in class to materials that have not been discussed; or, as P.W. (French and Italian) puts it, "to enter into a dialogue with the text they are reading."
Often, in senior seminars or advanced courses, assignments are designed to help students move from one level of thinking to another—to take the next step necessitated by (and valued by) the discipline. These assignments move students from description to analysis, for example, from summary to interpretation or from report to theory.
For some of the faculty we interviewed, the long paper assignment models the process of writing an article for a professional journal. In J. N.'s Senior Seminar in History and Philosophy of Science, for example, students are writing articles directed toward the journal, Philosophy of Science. This practice was most common, however, in the Natural Sciences.
For other faculty, the senior seminar (or writing practicum) is conceived as an introduction to writing in professional business or industrial settings.
I. F. (Psychology). For her course, "Psychological Aspects of Human Sexuality," the research project, staged out over the semester, is designed, as she says, "to provide the student with firsthand experience in doing professional writing and research on a topic relating to the psychological aspects of human sexuality." The students write a two-part paper on some aspect of sex education. Her instructions tell them:
In order to make this project similar to the type of professional writing and research a college graduate might be expected to do, each student should imagine that he or she is working for a company that markets sex education materials or a government agency that will be implementing a program for sex education. Develop a research question that might be of interest to this company or agency.
Many instructors include some form of oral presentation in their courses, to give students practice in a kind of writing they will use in their professions: "Being able to give a talk is almost more important" in Physics than being able to write a report. Faculty also regard oral presentations as opportunities for students to learn to prepare abstracts or short-forms of longer written work, or as occasions for students to feel the pressure of audience and the demand for clear organization.
Most of the faculty we interviewed have students writing short (1-3 page) papers, often informally (that is, without grades or commentary). These serve a variety of functions. They are designed to exercise particular skills (often without concern for evaluation)—preparing graphs and charts, writing an introduction or conclusion, writing a summary or précis, generating ideas from data sets. In large lecture courses, short informal assignments are used to engage students but also to assess what they know.
R. L. (Political Science) has created a useful taxonomy for the short assignments in his courses.
In class writing assignments, at least one a week, sometimes several a week. These are short and directive; some are unsigned, not all are graded. They are designed to assess prior knowledge; identify prior assumptions; summarize key points in the readings or lectures (one-minute papers); and help students identify key differences in time periods, approaches or concepts ("structured matrix comparisons").
Out of class writing assignments (four per semester, two of them using draft and revision). These are linked so they build toward a medium length paper. Students are encouraged (but not required) to choose a subject they can pursue through the four short papers. The assignments represent different genres, each providing students with a critical perspective on a text or topic: WDWWWWHW (Who Does/Did What to Whom, When, Where, How, Why?); assessment of a news source (comparing the treatment of a single topic across media, according to criteria provided to by the instructor); book review; and analytic paper.
K. S. (History of Art & Architecture) assigns six low-stakes informal writing exercises in his introductory courses, each designed to help students generate and develop ideas. He wants his students to learn that "writing is a tool for them, rather than purely performance—it can help focus their minds and clarify their thoughts."
Many faculty members we interviewed organize their students' writing through a sequence of smaller assignments that lead to a larger project.
The faculty colleagues we interviewed all provide extensive written commentary in response to the student writing they receive. Many also meet with students in individual conferences. Most assert the importance of providing not only evaluation, but also instruction—in forms of feedback that are directed toward the next piece of writing. "In the next assignment (in the next draft), here is what you need to work on…."
All of the faculty we interviewed spoke about organizing their courses so that revision becomes part of the required work and one of the crucial methods students are given for working on their writing. (All officially certified writing intensive courses on our campus are expected to make revision part of the required work of the course.) Some described revision as a way of cleaning up or tightening up a draft. In many courses, however, instructors use the revision process to open a first draft up to question, to provide the context for additional research and new lines of argument, to raise the problem of alternative points of view, to provide the occasion for attention to audience.
Most of the faculty we interviewed provide task or assignment-specific handouts (or on-line guides)—detailed supports that anticipate problems students might encounter in their writing.
M. G. (History) provides students with a packet of writing materials, including a carefully elaborated set of guidelines on writing "argument-driven essays." She outlines a sequence of procedures students may follow to develop a paper based on "well-honed middle-range conceptualizations that can be supported or refuted with evidence" rather than inflated generalizations.
J. N. (History & Philosophy of Science) provides a rich set of on-line materials that offer pointed advice about problems particular to writing (and power-point presentation) in his field. Here, for example, is what he says about Voice:
Voice. In both textbooks and research articles, scientists are encouraged to write in a passive anonymous voice. The fiction is of disembodied scientific consciousness that is the repository of scientific knowledge. "It was known that . . . ." New discoveries are stripped as much as possible of human form and motivation. "It was observed that . . . ." This locution suppresses the human beings who made the discoveries, where and when they were done, the reasons they thought to observe where they did, their passions and aversions, the rivalries and feuds and the many dead ends. Writing in this style makes it very hard to pay proper attention to context.
D. F. (Music) sent his senior seminar students additional instructions by email when they were struggling to move from description (of a musical work) to criticism. In his message, he tries to find another language (different from that in the writing assignment he had prepared for them), a language that might connect with the students, and at the same time he strives to represent the project as a writing project—something to be done in sentences and paragraphs. Here are sections of the e-mail:
Many of those we interviewed use models in their teaching—either examples of student papers or examples of professional writing—in order to give students a point of reference for genre, format, and style. With models, students learn that writing comes from within a community rather than out of the blue (or through divine inspiration). In some cases, the models are provided only to those students who are struggling, who don't have a sense of what is expected of them or who need help in imagining "good" writing.
The use of published models also prepares students to read the professional literature—not simply for information but as a demonstration of thought and method.
For some of the faculty we interviewed, using journal articles as a model for student writing was a new idea. Some expressed an interest in having their students read scholars' work as a model of writing within the discipline. Others prefer that students not read the professional literature. One colleague (Anthropology) said that he wants to demystify research and believes that journal articles would make that task difficult, by distracting students from their own decisions about substance and method. Another (History) avoids professional models so that students will attend to primary materials and work from "inside" the problem of narrating a particular history.
A number of faculty members use student writing as models—in addition to, or instead of, professional writing.
Most of the faculty members we interviewed provide handouts or lessons on the use of sources. Their particular concerns go beyond students' understanding of plagiarism. Students need to learn to evaluate sources, to read them critically and to use the material as the basis for their own thinking and argument (rather than stringing together quotations in lieu of thought or argument). Students have to find a way of using the material and they have to find a position from which to speak, as writers and thinkers, in relation to the experts or the professionals. They need to learn to understand what is new and ground-breaking, what is controversial, and even how to identify a fact or a conclusion.
In several of the courses discussed in the interviews, faculty restrict the range of materials students can work with as they write—often limiting "research" to materials in a course-pack. Instructors gave several reasons for this restriction. Much time is taken, and much back-tracking is required, when students head off to the library to try to find appropriate sources. A limited set of sources and a limited topic is also a hedge against plagiarism. Limiting the source materials also allows for comparisons across a class and increased attention to the intellectual or academic task—that is, what students can make from the materials at hand.
In the survey and in the focus groups, students expressed a variety of concerns about essay exams. The History Department faculty we interviewed, however, were the only ones to focus attention on this genre of academic writing. So much rests on these exams, they noted, and yet students are often unprepared for the task.
Bartholomae, David, & Matway, Beth. (2010, October 4). The Pittsburgh study of writing. Across the Disciplines, 7. Retrieved from https://wac.colostate.edu/atd/articles/bartholomae_matway2010/index.cfm