Assessment of the Writing Component within a University General Education Program

Legene Quesenberry, Tracey Johnson, Christopher Ferry, Edward Caropreso, Pamela Hufnagel, Paula Bevan, Karen Bingham, William Buchanan, Stacy Byers, Carla Diehl-Weyandt, Iseli Krauss, Wayne Miller, Randy Otte, Deborah Sarbin, Richard Smaby, Joseph M. Serafin, Jeanne Slattery, Karen Smith, Michael Vitali, Kathleen Welsch, Mary Wilson, and Paul Woodburne
Clarion University

Contact Information

View an Adobe Acrobat version of this article. (205K)

Introduction

The purpose of this study was to assess whether flagged "writing-intensive" courses within Clarion University's General Education Curriculum impacted on students' abilities to write. The major research question to be explored was, "what effect does taking writing intensive courses have on students' writing ability, when factors such as initial matriculation ability and total coursework are taken into account?" Papers written by a sample of freshmen, sophomores, juniors, and seniors were holistically scored to determine the level of writing exhibited by the student writers. Demographic information and information about the number of writing intensive courses each student had completed were correlated with the scores on the writing samples. It was found that students who had completed more writing intensive courses scored higher on the writing samples used for this research.

After providing an overview of Clarion University and the State System of Higher Education (of which Clarion University is a part), this paper provides an overview of Clarion University's General Education program. This is followed by a description of the study's methodology, demographics of the research population, review of results, and discussion of results.

Clarion University and the Pennsylvania State System of Higher Education

Clarion University of Pennsylvania is a public comprehensive university offering degree programs at the associate, bachelor's and master's levels. Like many other Pennsylvania universities, Clarion spent its first years of existence preparing teachers for the Commonwealth's schools after the Normal School Act of 1857 established regional teacher training institutions throughout Pennsylvania. The School Code of 1911 called for the state purchase of all normal schools, and in 1921, Pennsylvania's present configuration of 14 state-owned institutions was established. The normal schools evolved from state normal schools, to state teacher colleges, to state colleges. On November 12, 1982, Act 188 was signed into law establishing Pennsylvania's State System of Higher Education ("Act 188," 1982). On July 1, 1983, Act 188 was implemented. Clarion University along with 12 other former state colleges and Indiana University of Pennsylvania became the Pennsylvania State System of Higher Education ("About the System," sshechan.edu)

Today, the State System of Higher Education comprises 14 universities, four branch campuses, the McKeever Environmental Learning Center, and Dixon University Center. The System's purpose, as outlined in Act 188, is "to provide high quality education at the lowest possible cost to students." The System universities, located in rural, suburban, and small-town settings throughout Pennsylvania, educate more students per year than any other higher educational institution in the state.

A 20-member Board of Governors is responsible for planning and coordinating development and operation of the State System of Higher Education. The Board establishes broad educational, fiscal, and personnel policy, and oversees the efficient management of the System. Among other tasks, the Board appoints the chancellor and university presidents, approves new academic programs, sets policy for the State System, sets tuition, and coordinates and approves the annual System operating budget ("Board of Governors," sshechan.edu).

System-Wide Policy on General Education

In 1993, the Pennsylvania State System of Higher Education Board of Governors approved a policy on General Education, entitled "General Education at State System of Higher Education Universities" (the full text of the policy can be found under policy number 1993-01 at the web site www. sschechan.edu/BOGIndex.htm). Among other mandates, this policy states that all of the State System universities must ensure that (1) their general education programs meet State System priorities, and 2) their general education programs are properly evaluated in terms of student learning outcomes.

Clarion University's Policy on General Education

As part of its compliance with the State System policy on General Education, Clarion University engaged in a three-year process that culminated in a total redesign of its previous 1985 General Education program. The new program was adopted by the University in May of 1994, and implemented in August of 1995. In keeping with the State System's articulated spirit of general education, the Clarion University Statement of Philosophy on General Education asserts that General Education is enabling education. At its best, general education enables students to develop academic skills; acquire liberal knowledge; shape individual values; and apply all three (skills, knowledge, and values) in their academic, professional, personal, and societal lives. The goal of Clarion University's general education program is to provide students with specialized and liberal education that supports individual development and enriches professional and community life. Clarion University's General Education curriculum was designed to be a living program, amenable to review, revision, and innovation.

In order to facilitate the evolutionary nature of the program, an elected Council on General Education is charged with ongoing assessment, revision, and innovation of the University's General Education program (the Clarion University General Education Program can be found at the web site, www.clarion.edu/gened/index.htm).

The General Education Council advocates ongoing and consistent assessment that results in meaningful feedback for continuous improvement. According to the Council, assessment should:

• measure student progress toward academic markers of excellence,

• track academic progress and students' understanding of the value of general education,

• improve teaching and learning associated with general education, and

• evaluate institutional effectiveness in supporting the general education curriculum.

The Writing Component of Clarion University's General Education Program

Taken directly from Clarion University's General Education document ("General Education Requirements: Clarion University of Pennsylvania," 1994), the following describes the writing component of the general education program:

Writing is a skill that should be exercised by students across the curriculum, as appropriate ways are found in each discipline to incorporate writing as a vehicle for learning into instruction in the disciplines.

1. As a foundation for writing across the curriculum, students will be required to complete three writing-intensive courses prior to graduation: English 111 and two additional courses identified as writing-intensive courses. English 111 will be required in the program of General Education. The other two writing-intensive courses will be courses counting for credit beyond General Education as part of the total academic experience of the student, likely in the major or in support courses for the major not taken to fulfill requirements in General Education.

2. Criteria for identifying writing-intensive courses are based on the standard that the writing activity should engage students in higher order reasoning, such as synthesis and analysis, as well as the communication modes of specific disciplines. "The point, always is to keep students writing in circumstances where it matters to their intellectual and social development, where it is integral to the processes of inquiry and learning, and where it is required, not just in the seeming isolation of `English class,' but in every part of the University as a natural feature of academic life" (Cy Knoblauch, BYWORD, Fall, '86).

   RATIONALE:

   1. Writing should be used, not just to test knowledge, but to provide learners opportunities to explore ideas and develop connections between new knowledge and the understandings they bring to the pursuit of new knowledge.
   2. Writers should be encouraged to revise some portion of their work, under the guidance of the instructor, in order to better understand the constraints of the discipline within which they are working.
   3. Writers should have opportunities to use many forms, including in-class responses, journals, notebooks, reports, formal argumentative essays, research papers, and other forms pertinent to specific disciplines.

      The following guidelines are used to determine if a course qualifies as writing intensive.

      1. A substantial body of finished work. This is generally expected to total 20+ double-spaced pages in at least two, preferably more, submissions. A variety of forms is strongly encouraged – journals, reports, essays, research papers, etc. – not all of which need to be graded.
      2. Opportunity for students to receive assistance in progress. Such assistance may take several forms, from visits to the Writing Center to conferences with the instructor.
      3. Opportunity to revise some pieces. As revision is an essential characteristic of good writing, students should be able to revise some portion of their work.
      4. Response to student writing. Such response may take many forms – from extended comments from the instructor to peer evaluation in student groups. It is expected, however, that the instructor will respond in detail to some extended work of the students.

3. Departments will propose such courses to the Council on General Education for review and recommendations.
4. Courses so identified will be flagged with a WI designation.
5. All students entering the university will be tested prior to their initial registration for classes in order to assess their writing skills. Students with skills not appropriate to university-level work will be required to complete a course credited outside General Education designed to prepare them for the expectations of university-level writing.
6. Other than those students who require remedial course work, a further delineation is made between students who are in need of preparatory skills such as those offered in English 110 and those more advanced students who are placed in English 111.
7. Appropriate incentives will be offered to faculty to encourage the development of writing-intensive courses across the curriculum.

   The creation of writing-intensive courses should enhance present initiatives in the university focused on writing in the disciplines and across the curriculum.

Since 1995, over 100 existing courses within the university course inventory have been redesigned and approved as "writing intensive." These courses represent many disciplines, including accounting, anthropology, art, biology, chemistry, computer information science, education, English, geography, history, marketing, mathematics, modern languages, physics, philosophy, and psychology.

Purpose of the Study

The purpose of the current research was to determine how well Clarion University's "writing intensive" courses were meeting the student learning outcomes expectations articulated in the university's philosophy of general education. Specifically, the present research was designed to assess whether flagged "writing-intensive" courses within Clarion University's General Education Program impacted on students' abilities to write.

Methodology

Term papers completed by 120 Clarion University students in courses which enrolled a mixture of freshmen, sophomores, juniors, and seniors were analyzed using the Pennsylvania System of School Assessment (PSSA) "Pennsylvania Holistic Scoring Guide" (Pennsylvania Department of Education, 1998-1999).

In the fall 1998 semester, sample papers from CIS 110 (Introduction to Computer Information Systems) were utilized by members of the General Education Council to validate and calibrate the scoring instrument. The sample papers were later used as training materials and "anchor papers" for faculty who would be involved in the actual research. In preparation for the field test scoring sessions, the Council members modified the original scoring guide somewhat so that the definitions of writing characteristics would be clear for the current research. A copy of the final scoring instrument that was used for the current research is included as Appendix A.

Also during the fall, 1998 semester, a general "call for participation" was sent to all Clarion University faculty via e-mail and in hard copy. Faculty were invited to participate in the assessment of Clarion's General Education writing requirements by volunteering for four days to read and score student papers collected from courses in three to four different disciplines. The call for participation indicated that the General Education Council was particularly interested in faculty who had a commitment to teaching writing, including, but not limited to those who taught writing-intensive "flagged" courses. New faculty were especially encouraged to participate, and the Council indicated that the scoring experience would be formative for the faculty who were involved. Faculty who volunteered were asked to sign up for one of three assessment time periods during the spring, 1999 semester. In addition to the three facilitators, the Chair of the Council on General Education, and the Chair of the Assessment subcommittee, 16 faculty signed up to participate in the study. A copy of the "call for participation" is included as Appendix B.

During the Spring, 1999 semester, 120 term papers written by students in four classes which enrolled a mix of freshmen, sophomores, juniors, and seniors were scored according to the Pennsylvania Department of Education Holistic Scoring Sheet (modified as described above). The term papers represented four classes in three different disciplines. Thirty papers were from CIS 110, section 8 (Introduction to Computer Information Systems); 31 papers were from CIS 110, section 9 (Introduction to Computer Information Systems); 35 papers were from HIST 120, section 5 (United States History to 1877); and 24 papers were from ENG 200, section 4 (Composition and Literature). Faculty scorers were provided with copies of the papers and the accompanying assignment (see Appendix C for the assignments). Only one paper from each student was used. Any students who were represented by a paper in more than one class were assigned to one class. Their second paper in a different class was not used.

In order to divide the scorers into equal groups, four groups of faculty met for four days each to score the papers. The first group comprised four faculty representing the departments of Academic Support Services (one faculty member), Education (one faculty member), Biology-Science Education (one faculty member), and Chemistry (one faculty member). The facilitator for the first group was a faculty member in the department of English. The second group comprised four faculty representing the departments of Economics (one faculty member), English (two faculty members), and Library Science (one faculty member). The facilitator for the second group was a faculty member in the department of Education. The third group comprised four faculty members representing the departments of Psychology (one faculty member), English (one faculty member), Communication (one faculty member), and Accounting (one faculty member). The fourth group comprised four faculty members representing the departments of Communication (one faculty member), Accounting (one faculty member), English (one faculty member), and Psychology (one faculty member). The facilitator for the third and fourth groups was a faculty member from the department of English. Three of the four groups met for four consecutive days; one of the groups met on four Fridays throughout a 15-week semester.

Eight raters (two from each of the four groups) read each of the 120 papers. The redundancy was important in order to be able to determine "inter- and intra-rater reliability" and in order to ensure that each paper would have enough scores to provide a means for statistically analyzing the scores.

Demographics of Student Writers

Majors of Students

The 120 student papers included in the study were written by students with 28 different declared (or undeclared majors). Table 1 provides a breakdown of the majors represented by the student papers in this study.

Table 1. Student Majors

Transfer Status of Students

As Table 2 demonstrates, 100 (83.3%) of the total students in the study were not transfer students; 20 (16.7%) of the students were transfer students. This information is important since if a student transfers into Clarion University he/she may have received more or less preparation for writing at the previous institution.

Table 2. Transfer Status of Students

Gender of Students

As Table 3 demonstrates, 73 (60.8%) of the students in this study were female; 47 (39.2%) were male.

Table 3. Gender of Students

Class Standing of Students

As Table 4 demonstrates, 43 (35.8%) of the students were Freshman, 43 (35.8%) of the students were Sophomores, 25 (20.8%) of the students were Juniors, and 9 (7.5%) of the students were Seniors.

Table 4. Class Standing of Students

Academic Data

Table 5 provides the minimum, maximum, means, and standard deviations of several academic variables including number of credits taken by the end of the semester in which the paper was written, number of "writing-intensive courses" that a student passed with an "A," "B," or "C" grade by the end of the semester in which the paper was written, Cumulative QPA, Major QPA, Total SAT score (math and verbal combined), SAT verbal score, SAT Math score, global score on the writing assessment, total score on the writing assessment, and number of "writing-intensive" courses taken without regard to the grade received.

Table 5. Academic Data

Results

Inter-Rater and Intra-Rater Reliability of Scoring

Of the original 120 papers, four were judged to be unscorable (e.g., they were illegible, incoherent, or blank) by all raters, and were eliminated from the analysis. Each of the remaining 116 papers received a holistic "Global" rating and qualitative assessments for five characteristics of writing that are used in most typical holistic scoring rubrics for writing (Focus, Content, Organization, Style, and Conventions) from up to eight evaluators. The Global scores ranged from a low of 1 to a high of 6. The qualitative assessments ranged from low quality to high quality and were based on specific criteria for each characteristic. For example, the qualitative assessments for Focus ranged from "unfocused" to "sharp, distinct focus." The qualitative analyses were not assigned a quantitative value in order to ensure that the global scoring was holistic rather than based on adding the scores for assessments of writing characteristics. However, for all of the readers in this study, large-scale assessment was a new experience. The researchers were concerned about the reliability and validity of the scores for a group of previously untrained assessment readers. Therefore, for reinforcement purposes, the researchers chose to collect both global and characteristic subscores. To the researchers' knowledge, this "double collection" of scores is a unique method of data collection. For the analysis of intra-rater reliability, each characteristic assessment was assigned a score, referred to in this analysis as a "subscore" ranging from 1 (lowest quality) to 6 (highest quality). For purposes of determining intra-rater reliability, the subscores were added, creating a new variable called Total score, which ranged from a theoretical low of 5 to a theoretical high of 30.

Correlations among each of the five variables comprising the Total score (characteristics scores) and the holistic (Global) score were not calculated. However, intra-rater reliability was evaluated by correlating the Global and Total scores assigned to each paper by each evaluator (r = .550, n = 934, p = .000). Although the r-value was moderate rather than high (the r-squared accounted for a little more than 25 percent of the variance), the two sets of scores were positively and significantly correlated. Since a correlation of .80 or higher would have indicated more acceptable levels of intra-rater reliability, an analysis of the correlations among specific characteristics subscores and the Global scores might be indicated for future studies.

Inter-rater reliability was evaluated by computing intraclass correlations on samples of papers that were all scored by at least four raters in common. Intraclass correlations provide a means for assessing consistency and agreement among multiple scores of a set of targets (McGraw & Wong, 1996). Correlations from samples of papers from each course (CIS, English, and History) were computed separately because it was not possible to select a sample with good representation from all subjects that shared four common raters. If a paper was missing a score on a variable, it was omitted from the analysis for that variable only. The results are shown in Table 6. Most measures of consistency were significant, which means that papers that received high scores tended to receive relatively high scores from all four raters. Although most measures of absolute agreement were also significant, they tended to be somewhat lower than the measures of consistency. This means that despite a general consensus on which papers were better, raters varied in the actual scores assigned. Variance in the consistency and agreement ratings for style and convention were interesting to the researchers. It is possible that different approaches to training and rater calibration by the three trainers might account for this. It is also possible that the intervening days between scoring sessions for the group that met on four successive Fridays might have interfered with rater calibration.

Table 6. Intraclass Correlations Based on Samples of Papers

To present a clearer picture of the level of agreement among different raters on absolute ratings assigned to papers, the mean and standard deviation of the Global scores assigned to each paper by its raters were also calculated. The overall mean and standard deviation were computed using the individual papers means and standard deviations as data. As seen in Table 7, the mean of the standard deviations is .78. This indicates that in most of the cases, the global scores assigned by different raters for the same paper were within one point of its mean Global rating.

Table 7. Descriptive Statistics on Individual Mean Global Scores

Finally, as one further test, the Global and Total scores assigned to each paper by its evaluators were averaged to produce the scores used in this analysis. The correlation between the mean Global and Total scores assigned to each paper was .971 (n = 116, p = .000). Since these two scores were produced in quite different ways, but they correlated so strongly with each other, averaging the scores assigned by different raters apparently produced more reliable measures. It was therefore decided to continue with the analysis, using mean Global and Total scores as each student's essay scores.

Correlations

Total SAT (Verbal plus Quantitative) was found to be the best predictor of Global score (r = 0.524). This suggests that students who entered Clarion University with higher levels of developed ability (as measured by SAT scores) wrote better essays than students who entered with lower levels of developed ability.

Other potentially causal variables that correlated significantly with Global score were SAT Verbal (r = .504), SAT Math (r = .431), and the number of writing intensive courses passed with a grade of "A," "B," or "C" (r = .199).

The patterns revealed by an analysis of the Total scores on the essays were essentially the same. Significant correlations were found between the essay Total score and SAT Total (.543), with SAT Verbal (.521), with SAT Math (r = .447), with number of writing intensive courses passed with a grade of "A," "B," or "C" (r = .196).

These correlations, with number of cases included in each and their significance levels can be found in Table 8.

Table 8. Correlations Among Writing Scores and Academic Data

** Correlation is significant at the .01 level (2-tailed).
 *Correlation is significant at the .05 level (2-tailed).

Relationship Between Completion of Writing Intensive Courses and Students' Writing Abilities

Since the purpose of this study was to assess whether flagged "writing-intensive" courses within Clarion University's General Education Curriculum impacted on students' abilities to write, the following analyses were performed to determine the impact of writing intensive courses, controlling for entering ability (as measured by Total SAT), which was identified above as the best predictor of essay scores.

The correlation between Global score and the number of writing intensive courses taken was .175 (n = 116, p = .060). Controlling for Total SAT, the partial correlation between Global score and the number of writing intensive courses taken was .2227 (df = 95, p = .028).

The correlation between Global score and the number of writing intensive courses completed with a grade of "A," "B," or "C" was .199 (n = 116, p = .032). Controlling for Total SAT, the partial correlation between Global score and the number of writing intensive courses completed with a grade of "A," "B," or "C" was .2347 (df = 95, p = .021).

These partial correlations indicate that if one were able to compare students with similar SAT scores, those who had completed more writing intensive courses and those who had completed more writing intensive courses with grades of "A," "B," or "C" would have had higher Global scores on the essay. The correlations, with number of cases included on each and their significance levels can be found in Table 9.

The patterns revealed by an analysis of the Total scores on the essays were essentially the same. The correlation between Total score and the number of writing intensive courses taken was .161 (n = 116, p = .074). Controlling for Total SAT, the partial correlation between Total score and the number of writing intensive courses taken was .2306 (df = 95, p = .023).

The correlation between Total score and the number of writing intensive courses completed with a grade of "A," "B," or "C" was .196 (n = 116, p = .035). Controlling for Total SAT, the partial correlation between Total score and the number of writing intensive courses completed with a grade of "A," "B," or "C" was .2456 (df= 95, p = .015).

These partial correlations indicate that if one were able to compare students with similar SAT scores, those who had completed more writing intensive courses and those who had completed more writing intensive courses with grades of "A," "B," or "C" would have had higher Total scores on the essay. The correlations, with number of cases included on each and their significance levels can be found in Table 9.

Table 9. Partial Correlational Coefficients Between Number of Writing Intensive Courses and Global/Total Scores Controlling for SAT Total

All tabled significance levels are two-tailed.

Although the correlations outlined in Table 9 are significant, there is a possibility that students who had completed more writing courses were more advanced students, and therefore that their higher essay scores might have been partly accounted for by their having had more college experience overall. To test this, the effect of the number of credits earned was partialled out of the correlations between essay scores and the number of writing courses.

As Table 10 shows, none of these partial correlations reached significance, although the number of writing courses completed with grades of "A," "B," or "C" would reach significance in a one-tailed test, which is probably justifiable, since a positive correlation would be predicted.

Table 10. Partial Correlation Coefficients Between Writing Intensive Courses and Global/Total Scores Controlling for Number of Credits Earned

All tabled significance levels are two-tailed.

It could also be argued that students who remained in college long enough to earn more credits were more academically talented upon matriculation. To test that possibility, Total SAT was partialled out of the correlation between number of credits earned and essay scores. As seen in Table 11, these partial correlations are significant, meaning that controlling for the effect of SAT, students who earned more credits earned higher essay scores.

Table 11. Partial Correlation Coefficients Between Credits Earned and Essay Scores Controlling for SAT Total

As shown in Table 12, combining these factors in a multiple regression analysis reveals that although SAT Total and number of credits explain significant amounts of variance in essay Global score, once they are controlled, the number of writing intensive courses with grades of A, B, or C does not add much explanatory power. In fact, the partial correlation between the number of writing intensive courses with grades of A, B, or C and essay Global score, with SAT Total and credits partialled out is 0.0202 (df = 94, p = .845).

The researchers hope to determine in a later study whether part of the explanation for this finding is the positive impact our writing across the curriculum program has on students' abilities to write. It may be that because students receive many opportunities to write in a variety of disciplines and in a variety of forms (journal entries, essay exams, memoranda), their writing naturally improves.

Table 12. Regression Analysis of SAT Total, Number of Credits, and Number of Writing Intensive Courses with Grades of A, B, or C on

Essay Global Score

R² = .353

Together, these correlational data show that students who completed more writing intensive course received higher scores on their essays. However, because these results are based on correlational data and because the correlation becomes negligible when number of credits are controlled, future research should attempt to control for the possibility of a self-selection factor (e.g., good writers are more likely to remain in college and take more writing-intensive courses).

Discussion

This first foray into writing assessment is considered a pilot study for Clarion University. As the institution attempts to assess its General Education program, the institution also seeks to find manageable, cost-effective, formative means to assess student progress. As the Council on General Education and those involved with writing intensive courses continue to discuss and modify programs, so too will the assessment approximate a model suitable for the students at Clarion University.

As a pilot study, the researchers chose not to control for many variables in this process. As a result, not all findings reached significance. However, two findings did emerge that proved to be important in assessing the writing components of the General Education program: (1) the scoring rubric worked well for faculty readers; (2) when comparing students with similar SAT scores, students who took more writing intensive courses scored better on their essays.

In its initial form, the modified PSSA Pennsylvania Holistic Scoring Guide rubric provided a common basis for the discussion about writing assessment. The faculty readers eventually came to consensus on the qualities of good writing. Initial scorings were often non-adjacent, at which point readers negotiated their scores to adjacency. Over time, readers achieved adjacency and congruency in scoring with no negotiations.

The data from this study indicate that those students who had engaged in more writing in their courses consistently scored better on their assessed essays. This indicates the need for continued support of writing intensive courses in the General Education program.

Recommendations for Further Research

Refined Measurements

Despite extensive training and effort, there was still some disagreement among raters in the evaluation of individual papers. This reduced the reliability of the data, which in turn introduced error variance, obscuring potentially significant relationships among variables. Steps should be taken to improve inter-rater reliability in future studies. It is therefore recommended that there be one training session for all raters, or, if this is not possible, the same facilitator should be employed for each training session.

The student papers used in this study were written for different courses. Inspection of the assignments used in each class (Appendix C) revealed that although all assignments required focused papers and evidence to support statements, the instructions themselves varied with respect to the amount of information provided on how to write a good paper. The assignments also varied with respect to the amount of creativity required for completion. These variations in assignments may have introduced extraneous variance into the students' scores. For future studies, one solution to this problem would be to use the same assignment in the same course for all of the students. Another possible solution might be to ask students to write in response to a prompt during a set period of time. The advantages to using embedded assessment (e.g., using writing produced for a class assignment) include increased motivation of the student to write well and the ability of the student to work through a longer period of writing and revision. The embedded approach may more accurately reflect an approach to writing that is aligned with the academic writing processes articulated in the General Education requirements for writing intensive courses (e.g., 20+ pages of text with opportunity to revise in response to self-assessment and instructor input). However, the prompt/response approach would help to reduce the effect of variables such as the instructor, the relationship between the instructor and individual students, the amount of instructor input, and the effect of the course and/or course content.

English 111: Writing II and English 110: Writing I

The present analysis did not examine the effect of English 111 on the writing of students. It also did not examine the impact of this course within the general education curriculum. Since English 111 is cited in the Clarion University General Education Program as the first of three required writing courses, it would be reasonable to examine the impact of this course, either by itself, or in combination with other variables such as other writing intensive courses or demographic variables. Additionally, although not specified in the Clarion University General Education Program, analysis of the relationship between English 110 and writing might be helpful in order to determine future directions for this sequence of courses in terms of student's abilities to write.

Assessment Tests and Scoring Rubrics

The General Education Program mandates that all students are tested prior to their initial registration for classes in order to assess their writing skills. This study used SAT scores as the best indicator of prior ability. However, scores on a test that specifically measures writing ability upon entrance to Clarion University might be a better covariate. One way to achieve this might be to create a writing assessment tool and/or a scoring rubric that could be used for initial testing of students prior to registration, and then later as a measure of student's writing as they progress through Clarion University's general education writing components.

Formative Assessment: Changing Attitudes and Pedagogy

Another component of research involves the changing attitudes of faculty toward the use of writing in their classes. In this pilot study, anecdotal, qualitative evidence indicates a possible change in attitudes. Faculty who once identified "good writing" as writing that was grammatically correct began to see the importance of expression of ideas in writing. Further research should explore the changing attitudes of faculty readers toward the use of writing in their own classrooms.

Other Types of Analyses of the Current Data

The analyses presented in this paper treated all scores as interval level data, enabling the subscores for writing characteristics to be summed to create the essay Total scores and enabling scores assigned by different raters to be averaged. The mean scores were then used for further analysis. This procedure may have reduced the score variance. One alternative would be to use the modal rating assigned to each paper instead of the mean.

As noted above, the analyses reported in this paper combined the data from papers written for different courses. Further analyses of the current data set could focus attention on the papers from one course, keeping in mind that the sample size would be reduced.

Finally, the current analysis did not correlate demographic and academic data with specific characteristic assessments (Focus, Content, Organization, Style, Content). This type of analysis might provide interesting and helpful data that could facilitate better definition or confirm the focus of current criteria for writing intensive courses and writing general education courses at Clarion University.

Conclusion

This study assessed the scoring rubric selected by the researchers for use at Clarion University with Clarion University students. The results of this study indicate that completion of more writing intensive courses has a positive impact on students' abilities to write. Because the study was based on correlational data, and many of the significant correlations were relatively modest, consideration of changes in programs or policies should incorporate an analysis of costs relative to benefits and the results of subsequent research on the topic. Specifically, further studies should attempt to more clearly separate the influence of writing intensive courses from the impact of the overall university experience and the impact of students' entering abilities. Further studies might also attempt to:

• assess the impact of English 110 (Writing I) and English 111 (Writing II) on students' abilities to write, especially as the effects interact with students' completion of writing intensive courses, and
• assess the impact of English 110, English 111 and writing intensive courses on characteristics of student writing as these are defined in the scoring rubric used by researchers.

References

Clarion University of Pennsylvania. (May 2, 1994). General Education Requirements: Clarion University of Pennsylvania. Retrieved October 5, 2000 from the World Wide Web: http://www.clarion.edu/gened/guidelines.html#writing

McGraw, K.O. & Wong, S.P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods 1: 30-46.

Pennsylvania Department of Education. Division of Evaluation and Reports. (1998). Writing Assessment Handbook. Harrisburg, PA: Pennsylvania Department of Education.

Pennsylvania State System of Higher Education. (October 5, 2000). About the system. Retrieved October 5, 2000 from the World Wide Web: http://www.sshechan.edu/sssystem.htm

Pennsylvania State System of Higher Education. (October 5, 2000). Act 188. Retrieved October 5, 2000 from the World Wide Web: http://www.sshechan.edu/ssact188.htm

Pennsylvania State System of Higher Education. (October 5, 2000). Board of Governors. Retrieved October 5, 2000 from the World Wide Web: http://www.sshechan.edu/ssbogmnu

Appendix A