Abstract: Although teacher commentary on student papers and students' revisions have been investigated separately, the relationship between commentary and revision has not been much studied. Such a gap in the research leaves the nature of the progression from effective commentary to substantive revision obscure. This article describes a mixed method study, using quantitative and qualitative data, that grew out of a WAC assessment project in which multiple drafts of 64 student papers from first-year composition and writing-in-the-disciplines courses were examined to measure the extent and kind of revisions made by students after receiving feedback from their instructors. We conclude that revisions—especially substantive revisions—correlate with comments that address substantive matters; we provide empirical data to support that finding; and we consider the role of individual teachers and their classroom practices in encouraging substantive revision.
Over the past three-plus decades, numerous research studies have examined, separately or together, teacher commentary on early drafts of student writing and student revisions in response to such commentary, where "commentary" is defined as virtually any teacher intervention in students' writing processes, whether in the form of written marks or written or oral discursive observations and recommendations. Studies of commentary seem to outnumber studies of revision practices and studies of the relationship of commentary to revision. (For studies of commentary, see, for instance, Lees, 1979; Knoblauch & Brannon, 1981; Griffin, 1982; Horvath, 1984; Connors and Lunsford, 1993; Lindemann, 1995; Straub & Lunsford, 1995; Straub, 1997; Ransdell, 1999; Cho, Schunn, & Charney, 2006; Haswell, 2006; Hyland & Hyland, 2006; and Underwood & Tregidgo, 2006; for studies of revision, see Faigley & Witte, 1981; Faigley, 1984; Stay, 1983; Matsuhashi & Gordon, 1985; and Witte, 1985; for studies of both, see Hillocks, 1982; Sommers, 1982; Ziv, 1984; Dohrer, 1991; Beason, 1993; Prior, 1995; Berzsenyi, 2001; and Ferris, 2006.) This research has involved surveys, think-aloud protocols, interviews, samples of student writing created ad hoc for the purposes of a particular study, and student writing drawn from classrooms at both the first-year and advanced undergraduate levels and from graduate-school work.
Researchers have found that students prefer specific commentary (Sommers, 1982; Cho, Schunn, & Charney, 2006); that instructors offer insufficient praise (Daiker, 1989); that students tend to revise only what is pointed out to them as needing work (Dohrer, 1991; Ferris, 1995); that comments on substantive matters of writing, as opposed to the identification of surface errors, are preferable (Lindemann, 1995); that students find value in both directive and facilitative commentary (Ransdell, 199l); and that students prefer comments that are "clear and understandable" and "specific" (Straub, 1997). In their review of "best practices" for teacher commentary, Underwood & Tregidgo (2006) say that "It appears … that no one-to-one correlation exists between the type of feedback given and the resulting revisions or quality of compositions" and that "More research is needed in order to determine more conclusively the relationship between level of feedback [surface or substantive] and writing quality and revision practices" (p. 79). In a 2012 article in the Journal of Teaching Writing, Emily Isaacs observes that "… relatively few studies … have sought to demonstrate the effectiveness of revision activities or detail the particulars of the ways that students are able to revise their drafts" (p. 92). Indeed only one study (Beason, 1993) has undertaken a taxonomic analysis of both teacher commentary and student revision done in response to that commentary, and that work concentrated on writing-in-the-disciplines (WID) courses to the exclusion of first-year composition. At the same time, leaders in the field such as Chris Anson (2008) have called for more empirical research to help move composition studies "from belief to evidence, from felt sense to investigation and inquiry" (p. 12).
In the spirit of Anson's remark and in the context of the research and literature on comment-and-revision, the present article reports results from a detailed study of feedback and revision in papers drawn from 64 students enrolled in four first-year writing courses and seven upper-level WID courses at a small liberal arts college. It provides empirical evidence to demonstrate the close relation of comments to revisions, especially in extent and depth of revisions. In addition, it considers the role of qualitative factors such as individual teacher ethos in the dynamic of comment-and-revision. Without attempting to establish any "one-to-one correlation … between type of feedback and … resulting revisions" (p.79), as Underwood & Tregidgo (2006) suggest is a problem, our results do show a strong correlation between the extent of substantive commentary provided and the extent of substantive student revision. We believe that the data we gathered from courses at both levels of a typical Writing-Across-the-Curriculum program is useful to WAC directors, other writing program administrators, and teachers of WAC courses in identifying how both commenting and revising can be done effectively and in highlighting the relationship between effective "input" and "output" in the comment-revision transaction. In addition, without trying to be prescriptive, we contend that our analysis of the data can be directly helpful to writing teachers for promoting the substantive revision they want their students to learn and adopt.
An assessment project at Moravian College in spring 2008 examined the extent to which students in WAC courses were revising their writing. Both levels of the WAC program – first-year writing (WRIT100) and WID courses – named the ability to revise based on feedback as student learning outcomes. A careful comparison of multiple drafts of 64 student papers drawn from 10 WAC courses at M.C. – three sections of WRIT100 and seven WID courses – showed that 75% of papers were at least "moderately" revised. This finding led to a research project driven by two questions:
This was a formative study: we had evidence of extensive – or at least moderate – student revision of drafts that had been read and commented on by teachers, peers, and writing center tutors. What formed this result or caused this phenomenon to occur, at least in terms of the evidence of "input" we had in the form of comments, was what we wanted to determine. We did conclude what we have said above about the correlation between substantive comment and substantive revision, because that is what our data showed us. But when we began the project, our only hypothesis, broad as it was, was that there might be some kind of relationship.
The WAC Director collected sample papers, with a minimum of a first draft and revised final draft, from students in four sections (out of eight offered that semester) of first-year writing and seven WID courses covering all three academic divisions: two humanities courses (one each from English and Music), two natural sciences courses (one in Nursing, one in Physics), and three social sciences courses (in Economics, Political Science, and Psychology). Those seven courses constituted about half of the WID courses offered that semester. So as to represent students in each of the four years of study and to gender-balance the sample, students were chosen from class rosters made available to the WAC Director. They were not chosen for any perceived ability to revise. Meanwhile, in December 2007, Wingard had received word from the Moravian College HSIRB that the research was exempt from HSIRB review inasmuch as it studied "teaching methods or curricula." Each instructor was allowed to decide in advance which writing assignment would provide the sample papers for the study, as long as all papers submitted were written in response to the same assignment in the course. The request drew 68 sample papers: 24 from WRIT100 and 44 from WID courses. Of this number, two WRIT100 papers and two WID course papers were exempted because only one draft, early or revised, was included, leaving a total sample of 64 papers: 22 from WRIT100 and 42 from WID courses.
All 64 papers, in multiple drafts, were photocopied after names or other identifiers of students were removed. All were labeled according to course, and each paper was differentiated by number (for example, WRIT100A, Paper 1; ENGL220, Paper 4, etc.). A student – then a rising-senior English major who had been a tutor in the campus writing center for two years – worked with the WAC Director to examine the sample. First we identified and numbered individual comments in every preliminary draft of the 64 papers; we then did the same on subsequent drafts and again identified and numbered the revisions. We recorded comments and revisions in separate spreadsheets listing course, paper and comment/revision. In the process, we took marks such as checks or question marks – not just verbal comments – into consideration as comments, which we defined as reader interventions to which writers might choose to respond as they revised.
We then reviewed the literature on comments and revisions to find and adapt taxonomies that would describe comments in terms of level of attention ("surface" or "substance") and purpose (to question, to advise, to edit, etc.) and revisions in terms of level of attention and the revision "move" itself (substitution, addition, deletion, etc.). Our taxonomy of comments was drawn largely from Beason (1993). To it we added categories adapted from Ken and Fiona Hyland's chapter "Interpersonal Aspects of Response" (2006), Elaine Lees' "Evaluating Student Writing" (1979), and Lester Faigley and Stephen Witte's "Analyzing Revision" (1981). Table 1 shows our taxonomy of comments. Some examples appear below:
Each of us read a copy of every draft that had formative comments on it, and we negotiated any initial differences in categorization so as to achieve agreement. To the extent that each of us had considerable experience reading student papers and making formative comments on or about them – Wingard as a 30-year-veteran writing teacher, Geosits as a well-trained and sensitive writing center tutor – we were satisfied that we correctly and conscientiously classified every comment and every instance of revision. We had very little trouble getting on the same page as we examined the set of papers. We took our taxonomy for instances of revision directly from Faigley and Witte (1981) with no modification. We initially included a category for "Other" revisions, but their taxonomy is such that there can be no true outliers. In this analysis, as with comments, revisions were first classified as dealing with Surface Changes or Meaning Changes. For the sake of continuity we referred to these as Surface and Substance changes like the Surface and Substance categories of comments.
Surface Changes were then further classified as either Formal Changes or Meaning-Preserving Changes. Formal Changes include spelling, tense, number, modality, abbreviation, punctuation, formatting and other grammar revision moves. We included the addition of citations as a formatting move because it was driven by academic style requirements. Meaning-Preserving Changes were revision moves such as additions, deletions, substitutions, permutations, distributions, and consolidations that did not change the meaning of the phrase, sentence, paragraph or section of the paper being revised. If substance changes did change meaning, we classified them as Microstructure changes or Macrostructure changes, depending on whether they altered meaning on a local scale (Microstructure) or on a global scale (Macrostructure). Both Microstructure and Macrostructure revision moves would include additions, deletions, substitutions, permutations, distributions, and consolidations. Table 2 shows our taxonomy of revisions. Some illustrative examples appear below:
When we undertook this effort, more papers proved unusable. Those from one section of WRIT100 (four papers) and the papers gathered from the Physics course were eliminated because early drafts had no written comments. Thus our sample for this in-depth study consisted of 18 WRIT100 papers (from three sections of that course) and 39 papers from six WID courses, for a total of 57 papers. Table 3 provides the numerical results of our classification work; it lists the average number of comments and revisions made on papers within each course section. The numbers listed in the chart reflect a further distinction within the average number of comments and revisions into substance- and surface-related changes. The raw average number of comments and revision for each course can be reached by adding the surface and substance columns. The number in parenthesis next to the abbreviated course titles in the leftmost column indicates the number of papers within that course section that were used in the study.
Having read all 57 papers, in however many drafts each consisted of, having coded both comments and revisions, and having tabulated the numbers and kinds of both comments and revisions, we focused our attention next on those courses in which there were high numbers of comments and revisions addressing substantive matters in the writing. This focus was in keeping with our interest in substantive and significant revision outcomes. While substantive revision alone does not guarantee an improvement in quality, it does demonstrate student engagement in the writing process and identifies the comments that prompted such engagement. Beginning with our findings from the textual evidence in general, we will then take a closer look at particular comments and revisions in selected representative papers.
Our analysis revealed considerable variation in kind and depth of commentary by reviewers and in kind and depth of revisions by students. In the case of one WID course, for instance, the reviewer of first drafts (the course instructor) concerned herself only with matters of surface correctness. As might be expected, the revisions for that set of papers, while all at least "moderate in number," concerned matters of surface correctness only. So at least one WID course, in at least one sample of writing, attended only to editing, not real revision. On the other hand, papers from three other WID courses showed real engagement on the part of reviewers (in each case, the course instructor) in commenting on first drafts in ways that generated "moderate" to "extensive" revisions, and "global" revisions at that. From this we conclude what has already been demonstrated widely in the WAC literature: that when reviewers engage in careful reading of early drafts and provide useful formative comments, student revisions tend to be substantive and meaningful. Our statistical analysis suggests a correlation between the extent of substantive comments and the extent of substantive revision taking place across the sample of student writing (r (8) = 0.684, p < .05).
The most interesting finding is the evident correlation between the prevalence of deep (i.e., substantive rather than surface matters) comments provided by instructors and the prevalence of deep revisions students undertook in response to those comments. The average numbers of comments and revisions per paper, per course, is shown in Table 3, above. The pattern of substantive comments and the associated revision responses for the sample of courses are provided in the scatterplot below, r (8) = 0.684, p < .05. The strong positive correlation (maximum value is 1.00) and the pattern depicted in the scatterplot convey the same evidence of a consistent trend of association between the quantity of substantive comments by the instructor and the quantity of substantive revisions by the students. In other words, when instructors provide few substantive comments, few substantive revisions are generated by students and vice versa; increased levels of substantive comments trigger increased substantive revisions. Based on the current sample, this trend of association ascends in a steadily linear fashion across the range of substantive comments represented in the sample.
The statistics we have cited do indicate a correlation between kinds of comments and corresponding kinds of revisions. Scholars have long said, however, that students will revise what and how their teachers tell them to (See Dohrer; Ferris). So in order to consider the qualitative aspects of the comments and revisions in our sample, we offer some detailed discussion of what happened between reviewers and writers in one section of WRIT100 and in three WID courses. As Straub (1997) says, "Any analysis of comments must go beyond the superficial grammatical form and consider the voice and content of the comment and other formal markers that instantiate various relationships between teacher and student" (p. 99).
The most extensive and text-changing revisions we saw in our sample of student writing support the value of commenting recommendations from Underwood and Tregidgo, Straub's (1997) and Straub and Lunsford's conclusions about what kinds of comments best serve student needs. Straub and Lunsford (1995) derived seven "principles for commenting on student texts" from their study of the commenting practices of 12 expert readers. These principles are:
• Teacher comments should be well-developed and text specific.
• They should be focused on global, not local, concerns.
• They should emphasize nonauthorative modes of commentary.
• They should be carefully thought out and purposeful, with an eye to the needs and potential of the particular piece of writing.
• They should be suited to the relative maturity of the draft being read.
• They should be mindful of the rhetorical situation for the writing.
• They should be adapted to the student writer behind the text. (p. 373)
Later, in a survey of 142 first-year writing students, Straub (1997) drew conclusions about what students preferred and did not prefer in the way of teacher comments on drafts of their writing. He says "Overall, the students preferred comments that offered some direction for improvement but asserted only moderate control over the writing. They most preferred comments in the form of advice and explanations, since these comments typically are specific, offer direction for revision, and come across as help" (p. 112). Two other researchers, Underwood & Tregidgo (2006) surveyed the literature on commentary on student writing and concluded that "Feedback is … most useful to the student when the locus of control is with the student" and that "Students should be able to relate feedback to their writing products, choose which pieces of feedback are important and relevant, and understand what needs to be done in order to improve their writing" (p. 90). Specifically, they found the literature to recommend that teachers:
• Provide both content-level and surface-level feedback.
• Align directive and facilitative feedback with student goals.
• Balance control of the feedback and
• Present feedback in specific rather than general ways. (pp. 90-91)
Mindful of these recommendations, we offer some discussion of particular cases within our study which address the effectiveness of substantive commentary to provoke substantive revision, the relative usefulness of directive and facilitative commentary, and the connection between teacher ethos and praising commentary
To return again to the section of WRIT100 that produced the most extensive revisions at that level of Moravian College's WAC program, it is interesting to see how that teacher commented on drafts, not just how diligent he may have been as indicated by the raw number of his comments. He read first drafts and refrained from marking anything within the text, only making "content-level" comments in a note at the end of the draft. For example, at the end of a first draft of an argument essay about the practice of testing human cosmetic products on animals, he wrote four brief notes: "Opening ¶ meanders; far too short [overall]; many passionate arguments but almost no evidence; opposing view?" In our coding, we considered these notes as a single comment classified four ways: as problem identification, describing, questioning, and advising/suggesting. The draft was three paragraphs, 542 words. The student's second draft was 1200 words, and we charted two revisions – both long additions – to the first draft to account for that increase. When the teacher read this draft, he made 37 comments: 26 dealing with surface matters in the writing and 11 with substantive matters. The 11 substantive comments included problem identification (nine times), describing (twice), questioning (three times), and advising/suggesting (six times). Most of the comments are of the quality that Straub (1997), Straub & Lunsford (1995), and Underwood & Tregidgo (2006) recommend as most effective. To detail the interaction, we will quote from the teacher's comments and compare those to the revisions the writer made.
The student had written that the connection between "everyday products such as shampoo, toothpaste, and deodorant" and the "pain and suffering of millions of animals … is a fact that many are not aware of or may even deem necessary in order to distribute safe products to households across the nation." The teacher made the marginal comment "Who is doing the distributing?" The student's revision in her final draft inserted the words "for cosmetic companies" between "in order" and "to distribute." This may be a minor change in terms of number of words, but it is substantive in that it adds specificity to the claim. It is also a meaning-changing revision, even if just three words. In the second draft the student had also written, in the second paragraph, that "animals and humans are so drastically different, both anatomically and biologically," and the teacher – a biologist himself – wrote "clarify; this is patently false." In her revision, the student modified this claim and moved it to her fourth paragraph, where she generally enlarged upon her argument that animal tests are often "inaccurate." Here, she said "Animal testing … is not the most reliable form of tests that could be done considering the numerous counter reactions between humans and animals" and that the animals chosen for such tests "are very far off from humans when considering some of the experimental outcomes." She may be fudging her earlier claim about the "drastic…" difference between animals and humans, and her prose may not be the most felicitous imaginable, but she recognized what the teacher was recommending when he wrote "clarify" on her second draft, and she tried to accomplish that clarification in a substantive revision.
Again, at the end of the second paragraph of the student's second draft, where she had written "How the results found from testing on these creatures [rabbits, mice, and rats] could be applied to possible human reactions is baffling." The teacher commented, "You need to support this much better [student's first name]; it's the key argument for testing" [emphasis original]. The second page of the second draft of this paper was one long (414 words) paragraph, at the end of which the teacher wrote "Shorter paragraphs; this one is chock full of many different ideas." In revision, the student tried to better support her claim of the inapplicability of certain animal tests to human consumers while she distributed the strands of her argument over three paragraphs as well as including something of the "opposing view" that she had been asked about on the first draft. She eliminated the sentence expressing bafflement; broke the argument into points about inaccuracy of testing and tests conducted with foregone conclusions in order to avoid lawsuits against cosmetic makers; added a concession that "some experimentation [on animals] does have purpose and can be considered worthwhile"; then turned back to the assertion that testing of cosmetic products is not the "necessary evil" that testing for drugs to combat disease in humans might be.
At the end of the student's second draft, the student had a parenthetical statement to the effect that "need more opposing viewpoints and specific example of an effective alternative" as a note to herself. In response, the teacher wrote the single word "Indeed" then added "need to tighten up structure of what's here, too," a substantive comment that we classified as both problem identification and advising/suggesting. For her part, the writer addressed these matters through ten meaning-changing (albeit micro-level) revisions and three meaning-preserving changes involving fleshing out and rearranging some parts of her argument. The final draft was 1541 words, an increase of almost 350 over the second draft and close to triple the length of the first draft. Extensive and substantive revision indeed.
Although the categories of "directive" and "facilitative" commentary were not part of our taxonomy of comments, it would be fair to say that comments that supply actual edits and certain suggesting comments (such as the WRIT100 teacher's injunction to one student to "tighten up the structure" of a draft) are directive, while comments such as his question on the same student's first draft, "Who is doing the distributing [of "safe products to households across the nation"]?" are facilitative. In two of the WID courses whose papers we studied, some of the transactions between teacher-commenters and student-revisers speak to the relative value of directive and facilitative commentary. Straub (1996) and Ransdell (1999) tried to complicate the simple duality that comments are either directive – telling the student what to do and perhaps in the process wresting control of the text from the writer – or facilitative – invoking the teacher-as-reader's response and using moderated suggestions or questions to prompt the writer to rethink her writing at the global level. They argue that it is too simplistic to regard all directive comments as bad in that they are authoritarian and controlling and to see all facilitative comments as good because they force the writer back into her text at the deeper level of ideas. In two of our WID courses in particular, the revisions to early drafts showed the benefits of directive commentary – even of the teacher momentarily "taking over" the student text in order to make something the teacher deems desirable happen as the student revises, according to directions. In the Nursing and Psychology courses from which we collected papers, much directive commentary was focused toward conventions of the genres in which the students were asked to work, genres that are discipline-specific. We would like to discuss the dynamics at work in comment-and-revision in these two courses.
The Psychology class was an entry-level course for majors and a prerequisite for advanced courses in that department; the class standing of the seven students whose papers we collected was three juniors, a sophomore, and three first-year students. Due to the nature of the course, these students were involved in learning many of the conventions of writing in Psychology. The assignment that generated the papers we examined was to write an experimental design in which students were to cite sources in which similar experiments were discussed. Among the genre's requirements are discrete sections, with headings, in which different aspects of the proposed experiment and the literature on the topic are presented and reviewed. In reading the first draft of one student's paper, the teacher made 22 comments about substantive matters; of these, we classified ten as advising/suggesting. All ten were plainly directive, although one comment mitigated its advice by means of the word "might," as in "some of this detail might be better in 'materials' section." Most of these ten directive comments addressed matters of conventions of the genre. For instance, the teacher certainly took control of the writing at one point where she scratched through two lines of text and wrote in the margin "Never cite by title!" In another place, she wrote "insert 4 more summaries [of published studies]." In another, "Develop your literature first – then link to your hypothesis at end of introduction." At the end of the draft, the teacher wrote a summary note saying "Rework sections for greater clarity; add 4 summaries and more support for your hypothesis; remember APA format!" In the second draft, the student attended to all these matters of genre for the particular assignment except for the injunctions about including four more summaries of published studies on the topic (which was the effects of noise on memorization tasks). Inasmuch as the relevant comments were directive and straightforward and concerned conventional ways in Psychology of presenting an experimental design, the student was able to address them as he learned how to write this kind of paper. At the end of the second draft, however, the teacher wrote a long note that is more facilitative, even though it too was advising/suggesting:
… you still only have one article in your introduction. As a result, I cannot see if your study design is based on your literature, which is a key factor. … You will need to re-order your method section and add your summaries. Stop by with the revision and I can look it over and give you more specific feedback.
Whether the student followed up on the suggestion for a conference is unknown to us, but we did find evidence that he tried to follow his teacher's advice in this terminal comment. He did re-order the 'method' section of the paper, switching the order (from the second draft) of the 'participants' and 'materials' subsections, and he included a long (172 words) paragraph that summarized four previous studies on noise and memorization tasks. We identified this revision move as a micro-level meaning-changing addition, one of nine that occurred from the second draft to the final one submitted for a grade.
Similarly, in one of the extensively revised papers from a Nursing class, the teacher made 24 comments on substantive matters on the student's first draft. Of these, 13 were directive in one way or another – either advising/suggesting in direct ways or editing the student's words. The assignment here was to write a "mini-integrative review," a genre in Nursing in which a researcher reviews the literature on a topic (in this case the "significance of prompt indwelling urinary catheter removal in geriatric patients") and makes recommendations for nursing practice based on that review.
Again, it seems that directive commentary is appropriate when students who are novices in a discipline are trying to learn the conventions of a disciplinary genre. We will not rehearse all these comments, but a few examples should make the point. The teacher supplied the student with a functional, even boilerplate, sentence that would be customary in this kind of paper. After suggesting that the student move what was the second paragraph up into the first paragraph at a position immediately before the sentence, "The purpose of this paper is to discuss the importance of encouraging timely removal of unnecessary urinary catheters to avoid complications in the geriatric population," the teacher wrote, at the end of the first paragraph, "This will be accomplished through a review of selected literature, etc." Two other direct suggestions on this draft in the teacher's hand involved the insertion of headings: "Review of Selected Literature" at one point and "Practice" at another. In revising, the student added headings "Review of the Literature" and "Significance to Nursing Practice," among others. Later in the draft, the teacher directed the student's revision again by supplying alternative language but this time by explaining the change. The student had begun a paragraph with "In a phenomenologic qualitative study Mary H. Wilde (2002) explores patient's [sic] everyday experiences living with a chronic indwelling urinary catheter." The teacher underlined the first nine words of this sentence, down to the date of the Wilde study, and wrote "Using a phenomenologic design, Wilde (2002)" and added the explanatory note "makes it more concise and scientific." In the revised draft, the student complied with both directives to add words or change wording, although she resisted in one case the exact language the teacher had supplied. Her revision of the end of her opening paragraph made the paper read,
The purpose of this paper is to discuss the importance of encouraging timely removal of unnecessary urinary catheters to avoid complications in the geriatric population. This will be accomplished through a review of selected literature which will generate an understanding of the importance of timely removal of urinary catheters.
Farther down in the paper, the student introduced the Wilde study by saying "Using a phenomenologic qualitative study, Wilde (2002). …" In this paper, as with the Psychology paper, directive comments were necessary for establishing the conventions of scientific writing. With the students engaging in writing tasks necessary to their field of study, feedback on student writing can become another venue of instruction.
Finally, the effect of praising commentary on student revision is a valuable area of inquiry that this study can only briefly address through the papers drawn from a WID course in Economics. Haswell (2006) tries to untangle some of the complexities of responding to student writing and in the process emphasizes the teacher's ethos, as constructed by students reading comments on a paper, as variable yet crucial to students' willingness to do something with the advice provided in a comment. As he puts it, "the role of the responding teacher and the setting of the act of response combine to represent the teacher-responder in a way that may have complicated and powerful effects on students" (p.12). We see evidence of Haswell's assertion in the Economics class whose papers we examined. The sample showed moderate or extensive revision in seven out of nine papers. We counted 160 meaning-changing (albeit at the micro level) revisions in the nine papers, an average of 17.8 such changes per paper. In the seven papers that were at least moderately revised were 136 of these revisions, an average of 19.43 per paper. We would not wish to claim that these revisions owed entirely to the ethos projected by the teacher-reader or perceived by the students, but the professor teaching this class made 21 praising comments on nine drafts, an average of 2.33 per student; the teachers of the other five WID courses in our study made 30 praising comments altogether, so this teacher-reader stood out in this regard. His praise word of choice was "good," which he applied to a number of features of the drafts he read: working titles, sources listed on the Works Cited page, abstracts, students' research, quotations chosen to illustrate a point, word choices, and most especially drafts as a whole. The assignment was to write a condensed (i.e., not book-length) biography of a leading economist and discuss his or her contributions to the field. At the end of seven of nine first drafts, the teacher wrote some kind of praising comment, although these were almost always mitigated with a "but" clause. Typical is his note at the end of one first draft, on Irving Fisher, that reads: "Good paper. Clean up a few typos, clarify some ideas and it should be excellent." Another terminal comment on a first draft combined praise with identification of macro level concerns: "You have produced a good perspective on [Joseph Alois] Schumpeter's econ., but needs further development esp. [sic] in regard to his influences and his importance as a theorist even today." In coding the comments, Wingard remarked on this high rate of praise, and Geosits said of the teacher "He's like that," based on her experience having taken a course with him. In an interview with Wingard, this professor said that he "liked" teaching a great deal, that he found it "fun" to interact with students as they tried to make meaning through discussion and writing. The course from which sample papers were provided – "The History of Economic Thought" – was a 300-level course populated mostly by upperclassmen. Among the nine students whose papers were submitted for the assessment project were five senior and three junior Economics and Business majors, so it seems likely that the professor was familiar and comfortable with the students. This setting, as well as this professor's ethos, may account for the relatively high degree of praise in his comments on drafts.
In his study of eight students' responses to instructor comments on drafts, Dohrer (1991) found that "simpler, easier, and safer corrections dominated students' revisions, while more difficult macrostructural changes were ignored" (p. 52). Experience seems to show that student writers will revise what and how their teachers suggest in early drafts, so it should be no surprise that our findings confirm that. And although our sample size was relatively small, our study does provide empirical evidence to demonstrate the strong, positive effect of providing substantive comments to promote substantive revisions. Examining artifacts of student writing in first-year composition and in a handful of WID courses from across the disciplines, we found that across our sample the amount of deep revision by students correlates well (68 percent) with the amount of deep commentary from instructors, validating substantive commentary as an approach that fosters substantive and more effective revision on the part of students. Furthermore, our discussion of the relation of individual teachers' commenting practices to those teachers' teaching styles, their ethos, and to disciplinary conventions that are taught in certain WID courses addresses, if only implicitly, Fife and O'Neill's suggestion (2001) that studies of written comments on student papers be connected to the pedagogical context that produces them. Certainly, as Straub (1997) says, "[i]n actual practice the meaning of a comment is influenced by the teacher's persona, the ongoing relationship between teacher and student, and the larger classroom setting" (p. 100). One advantage of a small college for a study like this is the relative ease of access the WAC director has to the faculty teaching in his program. But if stakeholders in a WAC program no matter the size of the institution want their students to understand revision as involving substantive changes to a piece of writing, we need to help them see what to change and where to change it. As Dohrer (1991) says, "Good commentary … must prompt revision and must allow the student the opportunity to reconsider the text in the light of a reader's response" (p. 50). In order to make this happen, teachers need to engage with the writing in terms of its content. If this exploratory study points to a need for further research, we suggest that researchers will find our methodology useful and our taxonomies helpful. Our methodology should be easily replicable in any WAC context. The taxonomies we adapted and devised are portable and useful, so that researchers would not have to create their own scheme for identifying kinds of comments and kinds and scope of revisions. Our study should also encourage writing teachers to make their comments on student drafts substantive – to engage with student writing – as well as provide empirical substance to writing program administrators' efforts in faculty development to promote effective commentary. Replication of this study in other WAC contexts would be valuable and not terribly difficult to accomplish. We studied one semester's worth of comments and revisions; a longitudinal study using a methodology like ours and the descriptive taxonomies we adapted and devised should provide an even clearer picture of the landscape of comments on and revisions to student writing.
Anson, Chris. (2008): The intelligent design of writing programs: Reliance on belief or a future of evidence. WPA: Writing Program Administration, 32(1), 11-36.
Beason, Larry. (1993). Feedback and revision in writing across the curriculum classes. Research in the Teaching of English, 27, 395-422.
Berzsenyi, Christyne. A. (2001). Comments to comments: Teachers and students in written dialogue about critical revisions. Composition Studies, 29, 71-92.
Breidenbach, Cathleen. (2006). Practical guidelines for writers and teachers. In Alice Horning & Anne Becker (Eds.) Revision: History, theory, and practice. West Lafayette, IN: Parlor Press.
Brimner, Larry Dan. (1982). The effects of praise and blame on writing. English Journal, 71, 58-60.
Connors, Robert J., & Lunsford, Andrea A. (1993). Teachers' rhetorical comments on student papers. College Composition and Communication, 44, 200-23.
Cho, Kwangsu, Schunn, Christian. D., & Charney, Davida. (2006). Commenting on writing typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts. Written Communication, 23, 260-294.
Daiker, Donald. (1989). Learning to praise. In Chris Anson (Ed.) Writing and response: Theory, practice and research (pp. 103-13). Urbana, IL: National Council of Teachers of English.
Dohrer, Gary. (1991). Do teachers' comments on students' papers help? College Teaching, 39, 48-53.
Faigley, Lester, & Witte, Stephen. (1981) Analyzing revision. College Composition and Communication, 32, 400-14.
Faigley, Lester. (1984). Measuring the effects of revisions on text structures. In Richard Beach and Lillian Bridwell (Eds.) New directions in composition research (pp. 95-108). New York: Guilford.
Ferris, Dana. (1995). Student reactions to teacher response in multiple-draft composition classrooms. TESOL Quarterly, 29, 33-53.
Ferris, Dana. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In Ken Hyland & Fiona Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 81-104). New York: Cambridge University Press.
Fife, Jane Matthison, & O'Neill, Peggy. (2001). Moving beyond the written comment: Narrowing the gap between response practice and research. College Composition and Communication, 53, 300-21.
Griffin, C.W. (1982). Theory of responding to student writing: The state of the art. College Composition and Communication, 33, 296-301.
Haswell, Richard. (2006, November 9). The complexities of responding to student writing; or, looking for shortcuts via the road of excess. Across the Disciplines, 3. Retrieved from http://wac.colostate.edu/atd/articles/haswell2006.cfm
Hillocks, George Jr. (1982). The interaction of instruction, teacher comment, and revision in teaching the composing process. Research in the Teaching of English, 16, 261-78.
Horvath, Brooke K. (1984). The components of written response: A practical synthesis of current views. Rhetoric Review, 2, 136-56.
Hyland, Ken, & Hyland, Fiona. (2006). Interpersonal aspects of response: Constructing and interpreting teacher written feedback. In Ken Hyland & Fiona Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 206-224). New York: Cambridge University Press.
Isaacs, Emily J. (2012). Process writing in practice: When revising looks like editing. Journal of Teaching Writing, 27, 91-110.
Jerabek, Ross, & Dieterich, Daniel. (1975). Composition evaluation: The state of the art. College Composition and Communication, 26, 180-83.
Johnston, Brian. (1982). Non-judgmental responses to students' writing. English Journal, 71, 50-53.
Knoblauch, Cyril H., & Brannon, Lil. (1981). Teacher commentary on student writing: The state of the art. Freshman English News, 10, 1-4.
Lees, Elaine O. (1979). Evaluating student writing. College Composition and Communication, 30, 370-74.
Lindemann, Erika. (1995). Responding to student writing. A rhetoric for writing teachers, 3rd ed. New York: Oxford University Press, 216-45.
Lynch, Catherine, & Klemans, Patricia. (1978). Evaluating our evaluations. College English, 40, 166-80.
Matsuhashi, Ann, & Gordon, Eleanor. (1985). Revision, addition, and the power of the unseen text. In Sarah Warshauer Freedman (Ed.) The acquisition of written language: Response and revision (pp. 226-49). Norwood, N.J.: Ablex.
Prior, Paul. (1995). Tracing authoritative and internally persuasive discourses: A case study of response, revision, and disciplinary enculturation. Research in the Teaching of English, 29, 288-325.
Ransdell, D.R. (1999). Directive versus facilitative commentary. Teaching English in the Two-Year College, 26, 269-76.
Sommers, Nancy. (1982). Responding to student writing. College Composition and Communication, 33, 148-56.
Stay, Byron. (1983) When Re-writing Succeeds: An Analysis of Student Revisions. Writing Center Journal, 4, 15-29.
Straub, Richard, & Lunsford, Ronald F. (1995). Twelve readers reading: Responding to college student writing. Cresskill, N.J.: Hampton.
Straub, Richard. (1996). The concept of control in teacher response: Defining the varieties of 'directive' and 'facilitative' commentary. College Composition and Communication, 47, 223-51.
Straub, Richard. (1997). Students' reactions to teacher comments: An exploratory study. Research in the Teaching of English: 23, 91-119.
Taylor, Winnifred F., & Hoedt, Kenneth C. (1966). The effect of praise on the quality and quantity of creative writing. Journal of Educational Research, 60, 80-83.
Toedter, Lori. Personal Interview, 14 Oct. 2008.
Underwood, Jody S., & Tregidgo, Alyson P. (2006) Improving student writing through effective feedback: Best practices and recommendations. Journal of Teaching Writing: 22, 73-97.
West, James. Personal Interview, 18 Oct. 2008.
Witte, Stephen. (1985) Revising, composing theory, and research design. In Sarah Warshauer Freedman (Ed.) The acquisition of written language: Response and revision (pp. 250-84). Norwood, N.J.: Ablex.
Ziv, Nina D. (1984). The effect of teacher comments on the writing of four college freshmen. In Richard Beach and Lillian Bridwell, eds. New directions in composition research (pp. 362-80). New York: Guilford.
 Fife and O'Neill (2001) complain that studies of written comments are reductive, textual only, and incomplete in terms of the dynamic of teacher response and student reaction in the classroom context. Granting their point, we would point out that the data we have – written comments and revisions only – reflect the relative youth of Moravian College's WAC program (7 years at the time of the study) and its teachers' lack of experience with other commenting modalities such as audio or video comments. In addition, whatever happened in individual conferences between teachers or writing center tutors and students is necessarily unknown to us.
 A draft of this article was submitted for HSIRB review in Fall 2010 because we quoted student work and instructor comments and because two teachers were identified in the References list as having granted interviews to Wingard.
 Hereafter in this paper, "we" refers to the two investigators: Joel Wingard, Professor of English and WAC Director, and Angela Geosits, then an undergraduate at Moravian College. Geosits was the recipient of a SOAR grant (Student Opportunities for Academic Research) from Moravian College for the summer of 2008. That grant provides a stipend for students working with faculty on research. In fact, the WAC assessment would have ended after extent of revision was determined; the grant enabled our deeper investigation of comments and revisions. The authors also wish to thank Prof. Robert Brill of the Psychology Department at Moravian College for his input and advice on reporting statistical measures.
 The "surface/substance" distinction was adapted from Faigley and Witte's (1981) study of revision, but was included with the comments because we felt that identifying the level of attention of a comment would provide valuable information and create a compatible system of classification for both the commenting and revision portions of the study. Comments that were classified as surface attended to grammatical concerns or formatting, whether the comments were verbal or merely graphical. Comments that were classified as substantive attended to the topic, organization, content, clarity, or focus of the draft, in whole or in part.
 Further investigation revealed that the instructors in these courses used one-on-one conferences with students to comment orally on early drafts.
 The two outliers designated by a red "X" do not fit the pattern as well as the others based on the following critiera: For all the diamonds – the two values (comment and revision) that make up that plotted point for that course are either both over or both under the overall average for each of their respective variables, thus the ascending linear pattern and association. The X's are slight exceptions moving in opposite direction – one value under the average and one value over the average which is why the correlation is .684 instead of a value much closer to 1.0.
 Note that any particular comment could be classified more than one way; hence this total exceeds 11 comments.
 In an interview with Wingard, the teacher of this course – a Professor of Psychology – observed that the APA style manual "is not just about style; it's an epistemological guide too." Her stress on following APA models in her introductory-level psychology course, then, is part of introducing students to disciplinary ways of thinking, not just ways of following an arbitrary format.
 Note that this paper was one of four from the Psychology class that we considered extensively revised in our first pass through the papers. There were six papers altogether from that class.
 For her part, in reading the revised draft, the teacher here crossed out the word 'study' and inserted 'design,' so apparently she could not resist this kind of authoritative appropriation of the student's text. But we would argue that is justified to show a Nursing student one of the conventional uses of terminology in that discipline.
 Several researchers (although none very recently) have investigated the effects on students of instructors' praising comments. Taylor and Hoedt (1966), Jerabek and Dieterich (1975), Lynch and Klemans (1978), and Brimner (1982) came to similar conclusions: that praising comments have no effect on the quality of student writing but do enhance students' attitude toward writing in general. Other scholars have weighed in with opinions on the matter. Johnston (1982) says that "praise is judgmental and often … manipulative. … In many classrooms teachers' praise is not valued because students know that it is given to poorly performing students as an encouragement for those students to exceed the teacher's low expectations." (p. 50). Breidenbach (2006) argues that "at least some feedback [from teachers to students] be positive [because] … [p]raise brings solace to students' fragile and often wounded perceptions of themselves as writers. It gives them something to cling to and be proud of and build upon" (pp. 204-05). We believe that even if praising comments do nothing more than improve students' attitudes toward writing, that significant revision will follow. But further research into the effect of praising comments would be welcome.
Wingard, Joel, & Geosits, Angela. (2014, April 6). Effective comments and revisions in student writing from WAC courses. Across the Disciplines, 11(1). Retrieved October 25, 2016, from http://wac.colostate.edu/atd/articles/wingard_geosits2014.cfm