WAC Clearinghouse home page
About ATD   Submissions   Advanced Search

WAC and Assessment: Activities, Programs, and Insights at the Intersection

Profiling Programs: Formative Uses of Departmental Consultations in the Assessment of Communication Across the Curriculum

Abstract: Implementation of communication-across-the-curriculum initiatives has outpaced their systematic assessment, leaving many stakeholders wondering whether faculty and students are benefiting from the emphasis on writing, speaking, and other communicative media in discipline-based courses and curricula. Increasing interest in assessment, however, has generated questions about which methods can best gauge the influence of CAC programs and activities on students' performance, faculty involvement, and curricular change. This essay describes a departmentally-based methodology for the formative assessment of CAC programs within academic disciplines. This methodology—the departmental profile—involves creating a status report of communication activities based on identified communication outcomes. Drawing on one departmental profile to illustrate this process, we explore ways in which the method can map a department's progress toward CAC implementation and thereby reinvigorate its attention to CAC as a sustained element of its teaching mission.

In recent years, communication-across-the-curriculum (CAC) programs have diversified in structure and administration as a natural outgrowth of increased sensitivity to local contexts, including institutional missions, curricular emphases, student populations, and the needs and dispositions of the faculty. Approaches to CAC have also expanded because of a growing emphasis on assessment, which requires systematic attention to the relationship between broad curricular goals and individual teachers' instruction, a process not typical in the informal, grass-roots orientation that characterized the genesis of CAC.

One recent approach driven by an interest in assessment locates CAC at the level of the department or undergraduate program. Each department generates communication outcomes specific to its discipline and to students' needs, then decides how it will assess those outcomes and what it will do internally to help to achieve them. This approach gives authority and control to those closest to the courses where instruction takes place, provides greater incentive for change based on the desire to prepare majors effectively, and avoids the imposition of generalized requirements that may have little meaning to those who must enact them. It also emphasizes the relationships among writing, knowing, and doing within the discipline, which helps faculty to recognize the importance of their own roles in students' writing development (Carter, 2007). However, without continued intervention and support from CAC experts, over time departments may not act fully on their outcomes, "close the loop" from outcomes to assessment and curricular implementation, or continue to experiment with new instructional and faculty-development methods.

In this essay, we describe a follow-up methodology for the formative assessment of academic programs within this decentralized, departmentally-based model. This methodology—creating a profile based on internal, consultative study of a program—employs the same approach used to coach the development of departmental writing and speaking outcomes, but results in a report representing the department's current status: how writing and speaking are used, where, to what ends, and in what relationship to broader curricular, pedagogical, and career goals. Although designed eventually to map an institution's progress toward full implementation of CAC, these profiles play an interestingly heuristic role for specific departments and programs, moving them beyond the status quo and re-energizing their interests in CAC as a sustained focus of their teaching mission.

We first discuss the decentralized model of CAC in more detail, acknowledging that without continuous support, some departments experience entropy in implementation and assessment. We then describe a general protocol for the creation of departmental profiles designed, in part, to counteract this possible loss of energy. Finally, we provide examples of the state of CAC in one department—Food, Bioprocessing, and Nutrition Science—to illustrate the underlying approach and its processes.

A Systematic, Localized Model of Communication Across the Curriculum

Although CAC programs have taken a number of forms, from large-scale communication-intensive programs managed at the institutional level to grass-roots efforts sustained in a purely ad hoc way by the enthusiasm of individual teachers, a promising new approach organizes the program at the departmental level (see Anson, 2006; Carter, 2002). An outcomes-based system of improvement, locally managed by each department, allows faculty to take ownership of their efforts based on their own goals for students' education and career preparation. In this approach, an undergraduate program first generates a set of communication outcomes that reflect its unique disciplinary orientations and discourse practices. Creating these outcomes usually involves consultation with an interested group of faculty within the department, from whom the CAC consultant draws information about what faculty and potential employees expect a successful graduate to know and be able to do, what values are embedded in the work of the field, what kinds of communication activities are typical, and so on. Through successive meetings focusing on drafts of the outcomes, the small group eventually prepares a document for presentation to the department, which yields further refinements. The entire process is not achieved quickly or without deliberation; at our institution, creating outcomes statements for every undergraduate program in nine different colleges took over five years.

As an example, the Department of Microbiology at North Carolina State University included in the third of its three clusters of outcomes the following items addressing students' communication abilities:

Upon graduation, microbiology majors should be able to:

3. Understand, manage, and apply information about microbiology from both scholarly and popular sources and to communicate their understanding clearly and coherently for different audiences.

As in most good communication outcomes, those in the Department of Microbiology are embedded within the broader learning outcomes for the major. Consequently, the communication activities supporting them will wed attention to the subject matter with attention to the genres, audiences, language, presentational modes, and other aspects of communication in the field.

Having created its own outcomes, each department then can choose to work on assessment, implementation, or both simultaneously. For example, one department may already know that it is not fully achieving its outcomes because of significant gaps in its curriculum. Given its particular disciplinary orientation, the nature of its classes and students, and the disposition or preparation of its faculty, it may find that the best method for implementation is a kind of miniature communication-intensive program in which majors must take a sequence of required courses emphasizing communication in the disciplines. Another department might choose a "saturation" model in which every instructor must incorporate some writing and/or speaking depending on the type and nature of the course (for more on the departmental approach, see Anson, 2006; Anson, Carter, Dannels, and Rust, 2003; Carter, 2002; Carter, Anson, and Miller, 2003).

Just as implementation can take its own unique forms within each department, so assessment methods can also be shaped to the department's culture and students' career trajectories. In one department, perhaps the best assessment method is a survey of employers (with the permission of their alumni hires) to gauge preparation in communication skills. This method might work effectively in a small, career-oriented department that carefully tracks each student's job placement. In another department where graduates scatter into different kinds of professions or go into advanced degree programs, perhaps a portfolio review is the most effective method of assessing students' abilities. The department decides how best to determine the effectiveness of its efforts to produce competent writers and communicators. It does so not in a single, labor-intensive process common in periodic accreditation reviews, but in a series of more modest, ongoing inquiries.

Although ideally the motivation for engaging in this work should be intrinsic—it would be an odd department that didn't care whether majors learned anything or succeeded as a result of its teaching and mentoring—a localized program must be maintained and supported. Institutional incentives and requirements can help. On our campus, for example, there are no requirements for what departments do, but they must report the results to the administration each year. Every six or seven years, on a rotating basis, departments are required to submit a more substantive report of their cumulative efforts, with descriptions of what else they need to work on and what resources might help them. (Many departments also participate in discipline-based accreditation reviews such as those required by ABET, the Accreditation Board for Engineering and Technology, and the materials generated for the University's process can be used in those reviews as well.)

But even driven by such expectations, it is easy for some departments to neglect the work of implementation and assessment. Courses passed from instructor to instructor can lose their strong focus on certain outcomes. Department leaders and members of curriculum committees can change, and the history of the effort can be lost. Routines can settle in, yielding complacency and a sense that everything is working well. Resources that nourish a plan can dry up. And further modifications to courses and requirements can erode the relationship between outcomes and instruction. While continued emphasis on communication at the institutional level can help, entropy is still a real threat. Even in those departments where the communication assessment cycle has become second nature to their curricular oversight, there may be a need for outside experts to provide new insights, methods, and strategies to reinvigorate the department's work and help them to see themselves through new eyes. In support of our departmentally-based CAC program, we have chosen to provide this external consultation through what we call "profiles," formative reports of the role that writing and speaking play within a department or undergraduate program.

Revitalization and Renewal: Departmental Profiles

The concept of the profile is consistent with many routine reviews and assessments of programs, departments, majors, initiatives, colleges, and entire institutions. Typically, external reviewers visit a campus and, with or without team members from the institution itself, collect data through interviews, observations, tours of facilities, examinations of documents and Web sites, and the like. In our development of "internal" profiles, we have relied on our experience conducting such external reviews as well as consulting with faculty and departments on our own campus.

Although CAC leaders have always played a consultative role in working with individual faculty and departments, systematic efforts to provide localized formative evaluation within a college or university are relatively new. In one of the first published accounts of this process, Moran and Herrington (1997) described a "distributed" WAC program at UMass-Amherst, with "responsibility for program quality located in individual departments" (p. 123). Under the aegis of the University Writing Program, which monitors the Junior-Year Writing Component, reviews of academic departments are conducted every three years by one of the ten members of the program, which yield reports for discussion and action. The process also involves a system in which departments are rated as "exemplary," "functioning well," or "experiencing difficulty" in terms of their support of writing. Reviews of two departments—mathematics and anthropology—are included as examples.

More recently, the writing-across-the-curriculum program at the University of Denver has implemented an "action research" project involving the creation of profiles in up to five departments per year in order to document "student writing experiences and performances within specific majors … with a corollary element of portraying current assignment practices and beliefs related to writing" (University of Denver Writing Program, 2009, n.p.). Each profile yields a 5-10 page report that is used internally in the sponsoring department for its own considerations and improvements. Small grants are provided to the departments to help with their work, and students, who are part of the research team, receive a modest stipend (see http://www.du.edu/writing/documents/CFP Writing in Majors Project Year 2.pdf).

Unlike the reviews at UMass, our profiles are only optionally part of each department's required review process. Like the initiative at the University of Denver, we view the profiles as entirely for the benefit and use of the departments themselves. Consistent with the initial departmentally-driven process of outcomes development, our campus-wide consultations depend on a "ground up" philosophy and process. First, participating is entirely voluntary for departments, and they are free to decline the opportunity. However, there is little reason for them to do so because the information provided is purely formative and can be used, optionally, in the department's annual and periodic reports (thus saving them some time and effort). Second, as consultants, we do not enter departments armed with a fixed protocol for gathering data or presenting results about the department's writing and speaking instruction; rather, we adapt our methods to each department's culture and needs. Guided by an ethnographic orientation and committed to learning about the departmental culture from an insider perspective, we use a five-stage process for completing the profile:

  1. contacting key administrators or committees to outline the initiative and gain entry;
  2. with their help, choosing course and curricular foci for the profile;
  3. collecting data on the status of writing and speaking in the department's course(s);
  4. transcribing and analyzing the data; and
  5. producing a formative report for the department that may help to set the agenda for future collaboration.

The first step in the profile process is to gain entry into the department. This involves a meeting with key administrators, typically a head or associate head of the department or a faculty chair of a curriculum committee. This meeting allows us to describe the profile process, gauge the department's willingness to participate, and gain initial information about the best way to proceed. The second step in the process is to begin articulating and defining the methodology we will use to gather data for the profile project. Some of this can take place in the initial entry meeting and may require a second meeting with other faculty. For example, an initial meeting with a department head could lead to a meeting with the undergraduate curriculum committee in order to identify courses, curricular boundaries, and key players who will be involved in the profile process. In defining the boundaries, we look to department leaders to identify the kinds of courses that are central to achieving their communication outcomes, point to specific faculty teaching those courses, and highlight any other curricular activities that could provide insight into the process. In some departments, this results in a list of required or core courses for the major, while in others the boundaries are more defined (e.g., entry and capstone courses or lab courses). In this step, we also gather any curricular materials the department has already constructed that provide insight into the communication activities (course descriptions, planning documents, reports, results of previous assessments, etc.).

The third step in the process—conducting the profile—involves a variety of methods depending on the preferences of the department's administrators or committees, who usually represent the interests of the faculty. Sources of data can include faculty surveys, course syllabi, in-depth interviews, or focus groups. In our experience as of this writing, department administrators have favored one-on-one interviews with faculty, perhaps because they do not require administrative coordination and because we have suggested the importance of hearing faculty talk about their courses in order to gain an in-depth understanding beyond what a survey might provide. Allowing the department to express its preferences for data collection is important to establish a productive and unobtrusive relationship. If during the process we believe we are not collecting the most appropriate information from the best sources, we can always broach some change in method to the administrators.

For several reasons, profiles do not typically involve the collection and analysis of direct measures. Our role as campus-wide consultants does include helping departments, as needed, to design appropriate ways to assess student communication abilities or working with individual faculty to improve their instruction, but it would be impossible to do so as part of the profile process and achieve the goal of consulting with most or all departments in a few years. It's also important to avoid creating perceptions among the faculty that we are auditing their teaching for purposes of judging their success at supporting communication instruction. A number of specific faculty have asked us to help them in this way over the years, but such consultations are not part of a departmental or curricular effort, and the faculty member controls all the information the consultation yields. Finally, our entire departmentally-focused CAC model is designed to engage administrators and faculty within the department in assessing their outcomes and implementing curricular and pedagogical change (see Carter, 2002). In this respect, we tacitly assign to the department a role described by Walvoord (1997) as "client-customer," in which the faculty are free to use the advice or service in whatever way they wish. We can provide expert advice about the assessment process, but, like writing center staff who consult with students about their writing, we don't do the work for them. For these reasons, we have focused mainly on collecting indirect data, beginning with interviews and/or focus groups and allowing those to yield other sources of information such as syllabi, assignments, departmental planning documents, or surveys.

Regardless of the method we use to collect these data, we typically approach the profile with two general questions, each with two sub-questions. As the profile process begins, we remain open to adding other questions depending on our initial meetings with the department administrators:

  1. To what extent are communication activities in your department consistent with and working toward the department's stated outcomes?
    1. What communication competencies do faculty identify as important for their students to acquire?
    2. What kinds of communication assignments and assessments exist in the department?
  2. What challenges does your department face in implementing communication activities and/or instruction in order to achieve your outcomes?
    1. What teaching and learning difficulties emerge in the implementation of communication assignments?
    2. What departmental challenges exist in the implementation of communication assignments?

For interviews, we use an ethnographic interviewing framework (Spradley, 1979). The framework relies on friendly conversations (with gestures of politeness, question asking, interest, turn taking, etc.) in which the researcher slowly introduces new research-based elements to help the participant respond as an informant. This kind of interviewing focuses on getting the interviewee to provide insider information about writing and speaking—for example, to translate native language used to name communication genres in the department (Spradley, 1979) and to provide explanations of the department's norms and values surrounding communication activities and assignments. Interviews typically last about an hour. We also try to gather any materials the faculty is willing to share that focus on communication activities in their courses.

After the interviews are completed and transcribed, we analyze them alongside other materials we have gathered from the department. We use typological analysis—an inductive framework based on three steps: reducing data, creating thematic categories, and drawing conclusions (Goetz & LeCompte, 1984; Miles & Huberman, 1994). After sorting data according to the guiding questions, we use inductive analysis and a process of constant-comparison (Glaser & Strauss, 1967) to find thematic categories for each question. This process also allows for emergent categories outside of the prescribed questions, which is important given somewhat organic nature of the ethnographic interview.

When we have generated thematic categories that respond to each question, we move to the final step of the profile, which is to write a formative report that summarizes our results in a relevant and accessible way. The report can include various syntheses of the information we've collected. For example, for insight into the first question (the extent to which the department activities are addressing stated outcomes), the report might include a grid of the department's curriculum. Using the outcomes statement and the data we have collected as a guide, we list every course within the boundaries of the profile project. We place the courses along the horizontal axis of the grid and specific outcomes along the vertical axis. In this way, it is possible to consider every course in terms of its contribution or potential contribution to realizing each of the major outcomes. Next, we create a set of values—typically 1-3—to indicate the "strength" with which a particular outcome may be realized. For example, "3" might mean that the outcome is "strongly and fully incorporated into the course through both in-class and out-of-class writing and speaking assignments and activities." "1" might mean that the outcome is reinforced somewhat, through short, low-stakes writing and speaking assignments and activities (see Figure 1). Further consultations with faculty and administrators help us to refine the grid and make sure we have accurately determined the strength of attention to each outcome across the courses on the grid.

Figure 1: Example of Curricular Grid in Anthropology

Example of Curricular Grid in Anthropology

Another report format might involve a narrative description of the status of writing and speaking in the department's courses and curricula, using excerpts from faculty interviews to highlight particular types of communication activities, challenges students face, and broader departmental and administrative concerns. This narrative might be organized by the department's outcomes (for example, under each outcome there is a list of courses that address the outcome and excerpts from interviews highlighting the way in which the outcome is addressed), or it might be organized by particular courses identified as important to the profile, along with a supporting narrative. Consultants choose the structure and form of the report after gaining an understanding of the department and its norms for presenting data. Typically, the report will include recommendations, based on the descriptive data, for how the department can more successfully align its stated outcomes with current activities in the context of its current administrative and pedagogical challenges.

Figure 2: Symmetry of Assignment, Course, and Program Goals

Figure 2: Symmetry of Assignment, Course, and Program Goals

As part of this fifth and final step of the profile, the consultants present the report to the departmental administrators and faculty. Using the report as a starting point, the department can determine the extent to which certain outcomes are or are not being fully supported. Further faculty development, departmental discussions and consultation, and individual work by instructors can begin to strengthen weak areas within coursework. Implementation might include consultations with instructors to explore ways to create symmetry between their course goals, assignment goals, and the goals of the program or major (see Figure 2). In this process, consultants work with faculty to explore ways to tie all significant writing or speaking assignments in a course to its goals and outcomes (and in turn, the general outcomes for the major). This "scaffolding" allows for an otherwise disparate assortment of courses to serve students' preparation in a more systematic, articulated way. For example, in one department where we recently consulted, and where faculty had not discussed each other's courses in several years, three different instructors were teaching students how to create a resume in three different courses in the major. Unlike practices that need continuous reinforcement, such as certain kinds of discipline-based analysis, the skills required to create a good resume are somewhat more closely tied to the genre and, in the context of a specific discipline, usually can be learned through practice and support in a single course. This process allowed the department to substitute other outcomes-supporting assignments for two of the three resume-building exercises.

In this final step in the profile process, the report should be clearly understood as a type of formative assessment, void of judgment or evaluation about how "good" the department is or how it measures up against other departments (although successes are readily acknowledged, especially in the department's own processes teaching to and assessing communication outcomes). Because the department has opted to engage in the profile process, it is free to keep the results as a purely internal document, not publicized to any administrative committee or intended for any assessment purposes; or it may wish to use the report as part of its ongoing assessment or accreditation activities and make it available to various groups (or even post it online). The department's control over its report is essential to the integrity of the profile process.

A Profile of the Department of Food, Bioprocessing, and Nutrition Science

The Department of Food, Bioprocessing and Nutrition Science (hereafter FBNS) at North Carolina State University is one of almost twenty departments in the College of Agriculture and Life Sciences. It offers four undergraduate majors—food science, food science technology, bioprocessing science, and nutrition science—as well as five graduate degrees: an M.S. and an M.F.S. in Food Science, an M.S. in Nutrition, and two Ph.D.s, one in Food Science and one in Nutrition. There are approximately 117 undergraduate and 86 graduate majors in the department.[1] FBNS's basic communication outcomes are as follows.

By the end of the major, students will be able to[2]:

  1. Engage in clear and careful scientific inquiry.
  2. Apply critical thinking to solving problems and generating designs related to food science and technology.
  3. Understand, manage, and communicate source materials related to food science and technology.
  4. Work effectively in teams.
  5. Give effective oral presentations.
  6. Develop and utilize the personal and professional attributes that mark a successful Food Science graduate.

The profile process in FBNS involved first meeting with two key administrators in the department, both of whom had participated in several faculty activities offered by the Campus Writing and Speaking Program, which we administer. During this meeting, they agreed that we should talk about the profiles project with the undergraduate curriculum committee. Additionally, the administrators provided us with a curricular grid that they had already created in order to map their courses against their department's outcomes (see Figure 3[3]). Initially, they used the grid to discuss courses that might be important but suggested we discuss the process with the full curriculum committee. Although the curricular grid identified ten courses that incorporated writing and speaking, faculty on the undergraduate curriculum committee suggested that we look at six that served as required courses for most of their majors. The six courses included two at the 200-level and four at the 400-level, one of which was the capstone course. The committee then suggested that we interview the primary faculty associated with these courses (and who are consistently assigned to teach them). This yielded six interviews, which we transcribed and analyzed, using our guiding questions.

Figure 3: Food, Bioprocessing and Nutrition Science Curricular Grid

Lab Notebooks Reports
Abstracts
Journal Format Reports Resume writing Lab
Activities
Presentations Discussions Special Projects
2. Apply critical thinking to solving problems and generating designs                
2a. Identify, define and analyze a problem   231, 402, 421 402   231, 402, 421, 475 231 405 231, 406, 475
2b. Determine information needed, find it, assess its validity and use it   231, 402, 403, 421 402, 406   402, 421, 475     231, 406, 475
2c. Integrate and apply basic science and mathematics 475 231, 402, 403, 421 402   402, 421, 475 231 201 231, 475
2d. Demonstrate, based on information gathered, a range of viable solutions   231, 402, 403 402   402 231 405, 475 231, 475
2e. Evaluate solutions, choose the most viable, make a convincing case   231, 402, 403 402   402 231 405, 475 231, 475
3. Understand, manage and communicate source materials                
3a. Summarize source materials   421 402, 406   421 231, 290, 475 475 231, 406, 475
3b. Synthesize, analyze and come to conclusions from multiple source materials   421 402, 406     231, 290, 475 201, 475 231, 406, 475
3c. Make an argument for a judgment or a proposed action based on source materials   421 402, 406   421 231, 290, 475 201, 475 231, 406, 475
3d. Effectively translate food science to lay persons           290, 475    
4. Work in teams                
4a. Demonstrate group dynamics and strategies for working efficiently and productively with conflict   421     231, 402, 403, 421 231, 290, 475 405, 421, 475 231, 406, 475
5. Give oral presentations                
5a. Show effectiveness in presentations   421       231, 290, 475 475 406, 475
5b. Demonstrate skills in presentation technologies   421       231, 290, 475 475 406, 475
5c. Address audience clearly and answer questions           231, 290, 475 405, 421, 475 406, 475
6. Develop personal and professional attributes of a successful graduate                
6a. Show self-motivation, organization, and ability to prioritize goals 475 421     421, 475 475   406, 475
6b. Show managerial skills, ability to utilize diverse people         421     231, 475
6c. Show familiarity with emerging information and technologies     406     290, 475 475 406, 475

Results from the data analysis produced several thematic categories that addressed the broader topics of 1) valued communication competencies; 2) types of writing and speaking assignments; and 3) challenges (pedagogical and departmental) in implementing the writing and speaking initiative. From these results, we made several observations about our two general questions (relationship of the curriculum to the stated outcomes, and challenges in implementing the outcomes) and several recommendations addressing current challenges and future progress.

Communication Competencies. First, data revealed four general categories of valued communication competencies. A competent communicator, according to the faculty we interviewed, is professional (shows polish, uses email effectively, uses professional visuals, covers points relevant to professional arena, has clear audience awareness, is succinct for busy managers and faculty); uses information and research accurately (gets the facts straight, uses careful language about claims, gathers reliable information); employs reason well (works logically, thinks out problems thoroughly, follows a reasonable train of thought, provides evidence for claims, develops appropriate conclusions), and manages interpersonal tensions in teams (deals with conflict, effectively manages team delegation of tasks). For example, in talking about his expectations, one faculty member suggested that students' writing should not be about "marketing … how great this is going to be … without the substance, the evidence to show that or support it or back it up." Another faculty member suggested that "we are graduating scientists here … . They [should] speak succinctly and clearly about the issue at hand," and added that "certainly the ability to work with a team is critical because much of what is done is working as a team." Another faculty member suggested that "it would be helpful for them to have some training in how to handle e-mail. You know, what needs to be responded to and what's the appropriate way to respond to something, and what's the appropriate expectation in terms of time."

Types of Writing and Speaking Assignments. The second emergent set of categories revolved around the types of writing and speaking assignments being used in the department. Data showed two general types of assignments—industry simulation assignments (most of which were team based), and academic/graduate school preparation assignments. Of the former, faculty highlighted phase reports on design projects (written), management memoranda (written, 1-page), weekly management updates (oral, 2 to 5 minutes), mid-semester status reports (oral), executive summaries of final projects (written), and final team presentations. For the high-stakes industry simulation assignments, such as final presentations and phase reports, faculty provided detailed assessment rubrics, but for other less formal assignments, they produced fewer assessment tools. One faculty member described a short "update" assignment:

When the presentation is done in five minutes or less we ask questions. It is me, another faculty, and three TAs, and then any consultants that are involved in the project, typically faculty, but also industry people have an open invitation … to show up, and they do, and they are free to ask questions, and they do, because they are invested.

Another described short writing assignments such as "creating a memorandum for their team meetings that articulates the agenda, member responsibilities, and time allocations for each topic."

Of the academic assignments tied to graduate preparation, faculty identified abstracts, lab reports, scientific conference presentations, research papers, literature reviews, research presentations, and poster presentations. Again, faculty typically had assessment instruments for the larger projects—research papers, literature reviews, and research presentations—but had fewer assessment instruments for those that were not semester-long final projects. One faculty member mentioned that "there's a lot of work with data notebooks and lab notebooks," and another explained that "they do mini-proposals like for a conference or symposium … then they have to make a poster like they would for that conference."

Challenges in Implementing the Writing and Speaking Initiative. The third group of categories focused on the challenges students face (as identified by faculty) engaging in writing and speaking tasks. Data analysis revealed genre challenges (figuring out how to use email appropriately, learning how to communicate in a team, learning to speak extemporaneously in short presentations, using visuals well in poster presentations); skill challenges (thinking critically, being clear, being succinct, presenting accurate information, using evidence); and process challenges (transferring knowledge from one course to another, overcoming procrastination, reading directions, dealing with anxiety, setting aside time for preparation). One faculty member suggested that students have difficulty when asked "to think critically about what they are doing—I feel like so much of their writing is empty sentences," and another suggested a challenge in helping students to realize the importance of "the transfer of knowledge … making them realize that sophomore year builds on freshman year, junior year builds on sophomore year." Another faculty member claimed that students "don't know how to have a two-minute conversation with their boss and don't realize they can't waste time."

The final set of categories focused on departmental challenges with writing and speaking implementation. Faculty highlighted three departmental challenges: multidisciplinarity (being in a varied department and needing to be inclusive of expectations of all sub-disciplines, working with students with different backgrounds and skills sets, understanding the different writing and speaking tasks for all subprograms); learning to teach communication (developing personal ability to provide good instruction, handling discomfort with assessment and student response, knowing how to write assignment directions); and systemic challenges (balancing teaching and research, worrying about lack of institution-wide requirements and rewards for the communication initiative, perceiving a need to centralize communication instruction in a single course). For example, one faculty member lamented the difficulty in grading papers:

Where I can pick through an engineering exam or homework, 20 pages of derivation, I can fly through the grading of it, because I read it like a fine book. But when I look at writing I have to consciously think about it … we all struggle with that in the sciences.

Another suggested that "there needs to be something higher up that supports what we do—some kind of central course that covers these issues."

Outcomes Analysis. After analyzing the data and comparing them to the curricular grid and the departmental outcomes, we made several observations that address the extent to which this department is meeting its stated outcomes. First, the data revealed an emphasis on professionalism in the department's expectations for communication competence and in the writing and speaking assignments they use. This emphasis is consistent with outcomes 3-5 in the outcomes statement (teams, oral communication, professional development). Second, there was less stated emphasis in the interviews on outcomes related to critical thinking and source materials (outcomes 2 and 3). Third, there was some lack of symmetry between the expected level of competencies, skills, and assignments and the different course levels (for example, faculty teaching 200-level courses did not necessarily expect more novice communication competencies or use less complex communication assignments). From the interviews, we did discern a logical progression in the skills of teamwork: 200-level courses focused more on individual communication competencies and assignments whereas the 400-level courses did much more with team-based assignments. Fourth, formal communication assignments did involve some complex assessment, but assignments that were less formal or did not involve semester-long work had less fully articulated assessment criteria and support for students to internalize them. Finally, the faculty seemed to have a more sophisticated understanding of communication genres than the language of the departmental outcomes revealed, specifically with regard to oral communication. According to the outcomes, students should be able to "give effective oral presentations," yet faculty understood clear distinctions among types of oral presentations (extemporaneous updates, formal presentations, team-based oral activities) and the skills necessary to do these effectively. Additionally, faculty seemed to have a broader view of the media and genres of writing and speaking tasks, especially concerning the "less formal" use of email (written) and one-on-one updates (oral). Most faculty discussed the importance of communicating appropriately via email and in short updates with managers, but those genres were missing from the stated outcomes.

Recommendations. It became clear during our consultation with FBNS that this is not a department that has neglected its focus either on its educational goals more generally or its communication outcomes more specifically. Regular curricular meetings, a dedicated faculty, leaders who are fully invested in preparing students for the complex demands of their field, and a positive climate all contribute to a well-managed department that values the role of writing, speaking, teamwork, and digital technologies in students' academic work and career preparation. Based on our analysis of current activities and challenges, we provided four general recommendations to engage the department in discussion about the future development of its attention to communication. The first three are mainly classroom-based strategies that can be implemented with additional faculty-development opportunities, individual consultations, or departmental meetings with an action agenda and some follow-up from faculty; if acted on, the fourth will require that the department engage in some further curricular discussions based on its overall goals for its majors.

First, we suggested that the department explore more fully whether outcomes 2 and 3 (critical thinking, using source materials) are being incorporated in courses because we saw evidence that faculty were working towards these outcomes but in less formal communication activities. Second, we suggested that the department explore ways to increase student accountability for less formal communication outcomes and assignments. We were unclear if this accountability existed but was not articulated given faculty perceptions of what "counts" in grading, but sharing this observation is a first step in promoting discussions of the issue. Third, we suggested that the department work toward strategically and explicitly scaffolding assignments, activities, and competencies developmentally, in ways that mirror its attention to the movement from individual to team-based communication assignments. Finally, we suggested that the department revisit its outcomes to explore the extent to which they reflect the recent move toward multidisciplinarity (and the challenges that come with it) as well as the increased emphasis on emerging communication technologies in the discipline. This recommendation is especially important in light of the increasing subdisciplines in the department and their intersections, the growing connections with other majors and minors, and the more complex discussion of the breadth of genres.

Conclusion

In recommendations for site-based school development, Thompson (1997) points out that "the concept of continuous improvement is quickly replacing old ideas of improvement-as-an-event" (22)—exemplified, we would suggest, in the one-time accreditation reviews or periodic program evaluations so common in our colleges and universities. Without a commitment to continuous improvement, he argues, "new practices can quickly become fixed and stale" (23). His solutions, grouped into the concept of "monitoring and maintenance," include collecting data in the form of surveys, interviews, and focus groups in order to affirm that there is alignment between practices and processes, and providing a synthesis of that information back to those responsible for creating change (see also Selfe, 1997).

Although designed for the K-12 schools context, Thompson's model of site-based educational improvement finds many affinities with our profiles process. To succeed, CAC programs must ideally provide departments with whatever it takes to create a self-sustaining interest in communication, along with the structures, knowledge, and incentives to do so. In such an ideal world, the CAC program would eventually lose its usefulness and, having "passed itself on" within the disciplines, leave behind a legacy of its own consultative processes and materials. The rapidly changing circumstances of many departments and the increasing knowledge, strategies, and new perspectives at CAC programs' disposal to help those departments suggest a future of continued need. But like the departments they are designed to inform and reform, CAC programs must themselves be willing to assess the function and usefulness of their methods. For us, the profile process represents a significant addition to the "faculty workshop" model of development—still the most commonly used form of implementation (Thaiss, 2009). This communally-based but focused process is one that we believe has much potential to enhance the integration of communication activities into all departments and disciplines in ways that enrich learning and produce more successful, insightful, and productive students.

References

Anson, Chris M. (2006). Assessing writing in cross-curricular programs: Determining the locus of activity. Assessing Writing, 11, 100-112.

Anson, Chris M., Carter, Michael, Dannels, Deanna, & Rust, Jon. (2003). Mutual support: CAC programs and institutional improvement in undergraduate education. Journal of Language and Learning Across the Disciplines, 6(3), 26-38.

Carter, Michael. (2002). A process for establishing outcomes-based assessment plans for writing and speaking in the disciplines. Language and Learning Across the Disciplines, 6, 4-29.

Carter, Michael. (2007). Ways of knowing, doing, and writing in the disciplines. College Composition and Communication, 58(3), 385-418.

Carter, Michael, Anson, Chris M., & Miller, Carolyn. (2003). Assessing technical writing in institutional contexts: Using outcomes-based assessment for programmatic thinking. Technical Communication Quarterly, 12(1), 101-114.

Glaser, Barney, & Strauss, Anselm. (1967). The discovery of grounded theory. Chicago: Aldine.

Goetz, Judith, & LeCompte, Margaret. (1984). Ethnography and qualitative design in educational research. Orlando, FL: Academic Press.

Miles, Matthew, & Huberman, Michael. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

Moran, Charles, & Herrington, Anne. (1997). Program review, program renewal. In Kathleen Blake Yancey & Brian Huot (Eds.), Assessing writing across the curriculum: Diverse approaches and practices (pp. 123-140). Greenwich, CT: Ablex.

Selfe, Cynthia (1997). Contextual evaluation in WAC programs. In Kathleen Blake Yancey & Brian Huot (Eds.), Assessing writing across the curriculum: Diverse approaches and practices (pp. 15-67). Greenwich, CT: Ablex.

Spradley, James. (1979). The ethnographic interview. New York: Holt, Rinehart and Winston.

Thaiss, Chris. (2009). Challenging standard views of teaching, curriculum, and learning in higher education: Results and implications of the international WAC/WID mapping project. Paper presented at the Conference on College Composition and Communication, San Francisco, CA, March 13, 2009.

Thompson, Steven R. (1997). Site-based development. In Sarah. D. Caldwell (Ed). Professional development in learning-centered schools. Oxford, OH: National Staff Development Council.

University of Denver Writing Program. (2009). Writing in Majors Profiles Project, Call for Proposals. Retrieved from http://www.du.edu/writing/documents/CFP Writing in Majors Project Year 2.pdf

Walvoord, Barbara. (1997). From conduit to customer: The role of WAC faculty in WAC assessment. In Kathleen Blake Yancey & Brian Huot (Eds.), Assessing writing across the curriculum: Diverse approaches and practices (pp. 15-36). Greenwich, CT: Ablex.

Notes

[1] Numbers were retrieved from the North Carolina State University census data at: http://www2.acs.ncsu.edu/UPA/enrollmentdata/index.htm

[2] At the time the outcomes were created, the department was named "Department of Food Science." Since then it has expanded its curricular offerings and specialities and renamed itself the "Department of Food Science, Bioprocessing, and Nutrition."

[3] The grid shown in Figure 3 is missing outcome #1 because it was less focused on communication competencies. Additionally, it is missing "exams" and "homework problems" in the assignment type column because the faculty did not define those as communication assignments.

Contact Information


Complete APA Citation

Anson, Chris A., & Dannels, Deanna. (2009, December 3). Profiling programs: Formative uses of departmental consultations in the assessment of communication across the curriculum. [Special issue on Writing Across the Curriculum and Assessment] Across the Disciplines, 6. Retrieved August 27, 2016, from http://wac.colostate.edu/atd/assessment/anson_dannels.cfm