WAC Clearinghouse home page
About ATD   Submissions   Advanced Search

Using Citation Analysis Heuristics to Prepare TAs Across the Disciplines as Teachers and Writers

Abstract: National discussions about source-based, academic writing in higher education have been and are increasingly tied to concerns about citation proficiency, plagiarism, and academic integrity. In response to these discussions, scholars have argued for better pedagogical strategies to teach students how to work with sources in effective and ethical ways. Since WAC graduate teaching assistants (TAs) work closely with student writers, they too need support in crafting these pedagogical tools. Based on quantitative and qualitative research, this article argues that the use of citation analysis heuristics in WAC TA professional development programs fosters metacognitive awareness and an improved understanding of source-based writing. This study of ten first-year graduate students at a U.S. research institution found that TAs who code their own academic writing developed an awareness and knowledge that influenced their development as both teachers and writers.

Introduction: WAC TAs and the Transformative Potential of Citation Coding Heuristics

I'll put it like [this]… I didn't have a clue about [citation practices] before this [coding experience]. How the hell could I teach it to other people? The short [answer] is that I can't teach it. I can grade it, but can't teach it. TAs can't teach what they can't do and don't think about. Coding my writing showed me what I need to learn to help undergrads write,especially if they will be writing in a discipline other than English Studies.
- Participating Graduate Teaching Assistant (Author G)


Inquiry about and with sources is an integral part of academic writing. While disciplinary attitudes about the best ways to work with sources vary, there is general agreement about the necessity of effective source use—finding, selecting, comprehending, and using them effectively and ethically. Educators across different contexts share concerns about how students work with sources, creating a kind of hypervigilance (e.g. Silverman, 2002) and even hysteria about plagiarism and inappropriate source use. An entire industry dedicated to plagiarism detection (e.g. Turnitin) has developed in response to this cultural climate, although the efficacy and ethics of this industry has been met with increasing skepticism even as it increased interest in understanding student source use and research-based writing practices more fully.

In response to this growing interest in student source use, from 2007-2009, a group of researchers collected data in an effort to better understand how undergraduate students use sources in research-based writing (Howard, Serviss, & Rodrigue, 2010; Jamieson & Howard, 2010; Jamieson, 2013; Jamieson & Howard, 2013). The researchers worked together to code how students work with sources, classifying the citations as incidents of copying, paraphrasing, or summarizing, or patchwriting[1]. Initial findings highlighted two important trends: students were not summarizing sources and were predominantly working with sources at the sentence-level (Howard, Serviss, & Rodrigue, 2010). More data—and more applicable definitions of these citation strategies—was needed to contextualize these initial observations and thus the Citation Project (CP), a large-scale study[2] on student citation practices, was born. The CP evolved into a multi-institution study, ultimately collecting and analyzing data from 16 campuses across the United States and refining coding definitions and methods. The cumulative findings of the CP are numerous but those that have reverberated most are the coding results that reveal the infrequency of summary and the prevalence of quoting, particularly from the first pages of a source, in undergraduate source-based writing. The descriptive data generated by the CP have prompted many pedagogical and curricular questions about information literacy, writing instruction, and writer development. The Phase I findings—particularly the low rates of summary in the student papers and the high rates of citations from the first three pages of a source—generate questions for participating pedagogical communities about how we ought to be teaching student writers to work with sources in that local disciplinary or departmental and institutional context. Further, the Phase I findings of the CP have sparked national conversations about student source-based writing, student source use, and resulted in new ways of thinking about "plagiarism" in this era of hypervigilance (E.g. Berrett, 2012).

Based on the CP's compelling findings about student source use and my own curiosity about how CP research could function as a pedagogical tool, I decided to incorporate CP research into teacher-preparation courses for graduate teaching assistants (TAs) at a large, public American research institution. My initial goal was to use CP research to teach TAs about the slipperiness of student source use in first-year writing courses. Later, with the help of my graduate students, I came to see CP research as a valuable pedagogical tool in other ways as well. In this article, I present findings from a pilot study that explores the use of CP research methods in a TA practicum, arguing that citation analysis heuristics are valuable for WAC TA professional development programs in the following three ways, inviting TAs to (1) think critically about different kinds of citations; (2) make difficult judgments about students' citations and citation decisions; and (3) develop a metacognitive awareness about their own citation practices and source-based writing in their respective disciplines. I argue that these heuristics—comprised of workshops and reflection activities—have the potential to professionalize TAs as both disciplinary writer-researchers and teachers of writing.

This article contributes to the growing interest in preparing TAs as both writers and teachers, as recently observed by Brooks-Gillies, Garcia, Kim, Manthy, & Smith (2015b). Historically, discussions about TAs and writing education have positioned TAs as an audience meant to learn what writing studies has come to understand about undergraduate writing development and pedagogical strategies to propel that development. This article, as well as some of those published in ATD's special issue on graduate student writing (Brooks-Gillies, Garcia, Kim, Manthey, & Smith, 2015a), suggests the value of research with as well as about TAs as writers/writing pedagogues. Further, this article contributes to our understanding of how WAC TA professional development programs can nurture TAs' development as writers and teachers and thereby enhance undergraduate and graduate student education.

Research Study and Methods

The ten participants in this study were English and Education MA and PhD students specializing in secondary education (in second-language acquisition, English, and history), professional and technical writing, rhetoric and composition, and literature. They were all enrolled in a three credit graduate English course, Introduction to Composition Pedagogy. The course had two primary purposes: to introduce students to the key concepts and questions of writing studies and to orient students to teach in the composition program housed in the same department. Students read seminal writing studies articles in topical clusters from sources such as Cross Talk, A Rhetoric for Teachers, Writing Teacher Sourcebook, and Guide to Composition Pedagogies, wrote weekly short responses to the readings, and completed three major assignments: an ethnographic study of a writing classroom, a collaborative research project, and a draft of a teaching portfolio that showcased artifacts from the course. The course was consistently at capacity (with a cap of 15) and, although embedded within an English department, the course included regular conversations about writing instruction in different disciplinary contexts.

While the TAs shared practical reasons for taking the graduate course where this research project began (including requirements for their employment at the university and state-requirements for teaching credentials), they brought divergent attitudes about writing, knowledge, and experiences to the course that informed their participation in first the graduate pedagogy course and then the pilot study. Secondary education TAs, for example, took the graduate course about writing pedagogy as part of their requirements for the master's degree in education, a field that typically uses APA citation style and its approaches to source-based writing. Other TAs, such as those specializing in professional and technical writing, were preparing to mentor others about writing in the workplace and were likewise asked to use APA citation styles in their graduate writing. Given the TAs' role in teaching source-based writing, I began to see the CP research terms and methods as potentially useful pedagogical tools in a graduate course officially established to prepare "TAs to teach within the first-year composition program." I therefore developed a pilot curricula drawing on CP research, and the curricula became the pilot study presented in this article.

This pilot study of ten TAs from across the disciplines sought to answer three main questions that positioned TAs as developing teachers and writers:

The study was designed to harness the reflective habits already encouraged in TA preparation programs (Park, 2004; Burk, 2001). Participants used CP terms and methods to code one of their own academic papers, generate individual reflective writing and contribute to cohort discussions, as well as participate in individual interviews with me to capture how their understanding and behaviors surrounding citation began and evolved during their first year of graduate education.

Inspired by the features of design-based research methodologies (Brown, 1992; Collins, Joseph, & Bielaczyc, 2004; Design-Based Research Collective, 2003), I designed a series of activities to serve as learning opportunities for TAs who were observed at work in the naturalistic setting of their own graduate education and intervention opportunities for me as a teacher-mentor. As a founder and current principal researcher of the Citation Project, well-practiced in the CP coding methods, I created a series of workshops that introduced participating TAs to the research tradition of citation analysis (White, 2004; Pecorari, 2003; Roig, 2001); to the coding terms of the CP, particularly (summary, paraphrase, patchwriting, and copying); to CP methods for preparing student writing to be coded (retrieval and archival of cited sources, systematic identification of citations, systematic numbering of sources and incidents of citations, etc.); and to CP coding mechanisms (e.g., coding sheet). The initial workshops focused on collective coding of the same piece of student writing. The workshops calibrated the group's understanding of CP coding terms and enabled TAs to troubleshoot as they learned the coding process. Much time was spent during these workshops helping the TAs understand the terms—particularly paraphrasing, summarizing, and patchwriting—and discussing the interpretations required when applying these terms. After these initial discussions about terminology, we practiced applying the codes to student writing together. Following CP coding practices—and drawing on CP coder training materials currently used by the PIs of the CP (Sandra Jamieson, Rebecca Moore Howard, & Tricia Serviss), we coded a four-page excerpt of a student text, classifying each citation incident as predominantly one of the four citation moves: summary, paraphrase, patchwriting, or copying. Once TA participants became familiar with the coding method through trial and error, the self-study portion of the pilot study began.

TA participants chose one piece of academic writing they were composing or had already composed for a graduate level course. After preparing these texts for coding via CP procedures, students coded six-page excerpts from that academic essay, mimicking the CP citation analysis strategy of sampling citation work from the area most likely to be populated with citations in a student paper.[3] Participants reported their results at the end of the first coding workshop and in a group discussion that followed. The TAs were then asked to identify particularly difficult coding moments in their own texts and share their coding decisions with the group. At this stage, they were asked to make analytic observations but not interpretative claims. The TAs then composed reflective writing[4] to capture their reactions about coding their own writing and to enable them to situate their results in conversation with those of their peers.

Two experienced external coders (CP founders[5] who have coded more than a combined 200 papers) coded the same pages using CP methods. Through a series of individual, semi-structured interviews (see Appendix) after this coding, participants were asked to discuss observations they had as they coded their citations. In the course of these conversations I introduced the coding results of the external coders alongside the research participant coding results, asking the TAs to compare their coding results to the external coders' results, discussing convergences and divergences, which are shown in Table 1.

Table 1: Predominant Source Use as Coded by Author-Coder and External Coder Fall

Predominant Source Use within Citation Frequency Percent
Copying (no quotes), TAs
Copying (no quotes), External
0
0
0.0
0.0
Copying (quotes), TAs
Copying (quotes), External
52
52
60.5
60.5
Patchwriting, TAs
Patchwriting, External
0
14
0.0
16.2
Paraphrasing, TAs
Paraphrasing, External
27
13
31.4
15.1
Summarizing, TAs
Summarizing, External
7
7
8.1
8.2
TOTAL 86 100.0

Note: N=86

This two-step process, the participants' self-coding of selected writing followed by an individual data-processing interview with me, was repeated across two terms with different pieces of source-based writing that participating TAs had selected. These two steps were designed to encourage and develop the TAs' metacognitive awareness. The external coding results were not presented as the "correct" citation codes; rather, they represented a typical reading of the citations within the academy, using CP citation definitions as a typified schema. This comparison prompted the TAs to attend to the difficulties and ambiguities raised by such work, which was challenging even for relatively experienced academic writers such as themselves. Given the goals of this study—learning more about how TAs conceptualized and practiced citations and fostering their development as teachers and writers through critical reflection—these two steps mimicked both the feedback loop typically valued in writing instruction (write, reflect, and revise) and introduced TAs to the widely held belief that writing teachers need to study writing to better understand and teach it. Through these steps, TAs were explicitly challenged to simultaneously be the student writer and the writing teacher, a task intended to give them new understandings of the recursive processes of writing development and the complicated moves novice writers must master to join an unfamiliar discourse community.

Findings and Implications of Round 1 Coding

Tables 1 and 2 show the coding results of the fall and winter papers, including the authors' coding results, as a group, alongside the results of the external coder of the same paper. The participants coded their own writing in the structured environment of a facilitated, cohort specific workshop; I guided them through the processes of the CP methods but offered no interpretive support as TAs made decisions about how to code their own citation incidents in pages 4-10 of the manuscript they had selected for coding.

After the fall workshop I synthesized the anonymized TAs' coding results (Table 1 shows the TA codes) and shared the results with participants in a cohort-wide discussion. Research participants expressed satisfaction with their own source use once they saw the codes of the TAs combined, explaining that the citation strategies they had found were "expected" and "acceptable" academic practices.

Author D[6] (a first-year TA studying secondary English education) described the collective coding results as "indicative of conscientious citation work" or what might be understood as sanctioned source work. Author G (a first-year TA studying literature) believed that the coding results could be explained by the cohort's shared context, explaining that "we were the diligent high school kids who took the anti-plagiarism talks to heart," attributing the group's "highly successful citation moves" with their cumulative preparation as high school and undergraduate student writers. There were recurring comments about how reasonably "clean" the citations seemed: Author B viewed his as "clean" because there were no reported instances of copying without quotation marks. Likewise, the TAs were pleased with the fact that they had identified no patchwriting in any of their 86 citations in the fall term.

While these TAs expressed confidence in their citation savvy, others were reluctant to make definitive pronouncements and waited eagerly to compare their results with those of the external coders. Author F, for example, described the absence of patchwriting and copying as "convenient" and wondered about the accuracy of these findings, commenting, "Who wants patchwriting in their essay? Nobody wants that and ….so no one found patchwriting. I'm guessing it isn't that clean…?" Author C (a first-year TA studying rhetoric and composition) expressed "relief" with the coding results yet also challenged the cohort's general sense of relief and satisfaction asking:

Are we confident that we are using sources in rhetorically effective ways in our seminar papers? Or are we just celebrating being legit?… I'm thinking about the difference between being competent and confident...I'm thrilled that these codes lead to us as competent compared to first-year undergraduate writers, but what is the criteria for competence for us as grad students and writing teachers? The bar is low, folks.

During the cohort-wide discussions of the coding results, Author C was one of the few who expressed hesitation about the results that the rest of the cohort found generally "positive" and "encouraging." Author C returned the conversation to earlier discussions of citation styles and disciplinary expectations, describing the codes as "affirming our MLA instincts" while also asking how different citation styles and disciplinary conventions—like avoiding quoting in APA—might influence the "success" of the TAs. In this way, differences in disciplinary perspectives began to emerge among the participants.

The results of external coders (as shown in Table 1) in the fall term were aggregated after the workshop and submitted for large group discussion (recounted above). I shared the fall external coding results with the research participants and had them write reflective responses to these comparative coding results prior to individual interviews. In their written reflections, the TAs recognized a disconnect between their findings and the findings of the external coders (shown in Table 1), which prompted moments of crisis and reconsideration.

Author D, who had noted the group's "conscientiousness" after the first coding workshop, wrote about the "strange differences in coding based upon interpretation of paraphrasing… It all comes down to [how we define] paraphrasing and patchwriting." Author D described the "shocking difference" between TA codes and external codes, particularly the coders' disagreement about the frequency of patchwriting and paraphrasing (see Table 1). Collectively, the TA coders found that 27 of the coded source incidents were paraphrases whereas the external coders found that only 13 of these incidents were paraphrasing and deemed the remaining 14 incidents patchwriting. All the coders (authors and external) agreed on the frequency of copying with quotation marks (52 incidents) and the occurrence of summary (7 incidents). The TA coders were remarkably similar to the external coders in their coding of copying and summary, but had more divergent understandings of paraphrasing and patchwriting as terms used within CP coding. The TAs were asked in interviews about what the differences in coding results might mean. Some TAs, such as Author B, concluded that the TA codes were "inaccurate" or "biased." Notably, Author A, argued that the divergent results "weren't about accuracy but development," explaining that "these terms are dependent upon context and audience…and we don't have [the] experience or familiarity with these moves in our disciplines so we are…adrift but it is temporary. Developmental maybe." This kind of critical reflection about the complexity and challenges in learning disciplinary writing is the goal of the coding heuristic.

Notably, this divergence around use of sources occurred at the sentence level (a significant tension first noted in Howard, Serviss, & Rodrigue, 2010), suggesting that TAs, like undergraduate writers, struggle with sources at both the macro and micro level as they develop as writers. TAs' difficulty working beyond the sentence level suggests their struggle to master disciplinary and source-based writing. Comparing TA and external codes fostered critical reflection and author development for the TAs. One of the most valuable results of the coding heuristic, then, is how it increases TAs' awareness about the terms used for coding, the ambiguities of these terms, and the challenges of learning the citation practices of their own specific disciplinary community.

Although this study suggests that the comparative coding results evoked analytical thinking about coding categories for all participants, it also shows interesting trends in what TAs in different disciplines learned about themselves as writers and teachers from these heuristics. Without prompting, participants noted one of these trends within disciplinary cohorts, which prompted them to consider the relationships between, as Author F put it, "the citation system with its rules and… our habits as writers." The four TAs in English education and rhetoric and composition (Authors A-D) talked about "learning to read student writing differently" to track developmental trends within "student writing." These TAs treated their own coded writing as "student writing," adopting a writer-based approach to analysis of the coding results. They were surprised by the comparative coding results and processed the coding results by "looking for strategies to teach students how to avoid patchwriting."

When asked in interviews about their reactions as writers to the coding results, Author D, for example, described feeling "as if I'm a panic attack waiting to happen when I think about what this patchwriting would mean for my career if I stupidly published this seminar paper" and expressed a "sense of worry that I don't know what is happening in my writing even though I'm a good reader…I've been thinking about my education career as all about grades but now I need to think more about how to actually do the things I will be grading students on."

Author A's coding experience led her to reflect on the citation decisions she made in recent research-based writing:

it isn't that I worry about plagiarism…or about comprehending the source I'm quoting…but about having little to say myself. The sem paper I worked on is about the texts I read and the ideas of the sources instead of my ideas. That isn't all that shocking given that [I was an] undergraduate last week practically… what is shocking is that… I didn't see that and turned in a paper that describes and restates what a few texts say… I quote because I have little to say.

Author A's reflection illustrates the productive nature of these heuristics. They help TAs understand writing as developmental; they promote learning about writerly identities and practices as well as the teaching of source-based writing.

Reactions from the six TAs studying literature (Authors E-J) were significantly different than the four TAs in education and rhetoric and composition. The literature TAs experienced the most "copying, cited and marked as quotation" incidents. In written reflections one of these TAs, Author G, describes "how ideal these coding results are given what literary scholars do in practice…we work with small pieces of text that we first copy out for readers and then comment on…These codes are [those of] literary scholarship." Author G found the coding results revealing in terms of what they revealed about disciplinarity, exposing and perhaps even emphasizing the values supporting citation practices in literature while also recognizing those as different from the citation epistemologies of other disciplines. For Author I, finding copying as the prevalent citation incident in his writing prompted a productive confusion about the expectations of the literary criticism discourse community. Author I explains:

I'm not sure why they would want an essay that is 60% quotations and only 40% analysis... Nothing we would ever read for class would be positioned like this… But how [did I get an] A on this seminar paper? She [the professor] didn't say anything about all of this [points to the highlighted quotes in the text] so why not?

In short, the use of citation coding analysis invites graduate student writers to study their own writing, consider a wider disciplinary audience beyond the singular "professor," and increase their metacognitive awareness. These realizations rippled through TA lives both as professionalizing writers and writing teachers.

Findings and Implications of Round 2 Coding

In the winter term TA participants selected and coded a second piece of academic writing written in their second term of graduate school. As they had done in the fall, the TAs then responded to the same reflective writing prompts, compared their own coding results with those of the external coders (results are shown in Table 2), and discussed their reactions in an individual interview. Winter coding results between the TA coders and the external coders were less divergent, specifically in how incidents of patchwriting and paraphrasing were categorized: decreasing from a 16% differences in the fall coding in these categories to ~ 8% differences in these categories in the winter coding. TA coders thus demonstrated an improvement in their understanding of the CP coding terms, particularly when recognizing both patchwriting and paraphrasing.

Table 2: Predominant Source Use as Coded by Author-Coder and External Coder Winter

Predominant Source Use within Citation Frequency Percent
Copying (no quotes), TAs
Copying (no quotes), External
0
5
0.0
3.9
Copying (quotes), TAs
Copying (quotes), External
52
52
40.6
40.6
Patchwriting, TAs
Patchwriting, External
11
22
8.6
17.2
Paraphrasing, TAs
Paraphrasing, External
44
34
34.4
26.6
Summarizing, TAs
Summarizing, External
21
15
16.4
11.7
TOTAL 128 100.0

Note: N=128

The two major differences between fall and winter terms (shown in Table 3) were an increase in the number of citations in TA papers in the winter term papers (n=86 in fall, n=128 in winter) and more recognition of patchwriting and paraphrasing in the winter. The difference between TA and external coding results of patchwriting (16.2% in the fall) and paraphrasing (16.3% in the fall) narrowed in the winter term to much smaller margins. In the winter term the TAs' and external coders' results agreed more closely with only 8.6% difference in their patchwriting codes and 7.8% in their paraphrasing codes. The only increase in the disparity between the coders was about summary: in the fall all coders had agreed on summary codes, but in the winter there was a discrepancy of 4.7%. Therefore, although sentence-level work with sources became more recognizably acceptable to academic readers, in CP terms, more global work with sources—such as summary—became more opaque.

Table 3: Predominant Source Use within Citations Across Fall and Winter Terms

Predominant Source Use within Citations Fall Winter
Patchwriting, TAs
Patchwriting, External
0%
16.2%
8.6%
17.2%
Paraphrasing, TAs
Paraphrasing, External
31.4%
15.1%
34.4%
26.6%
Summarizing, TAs
Summarizing, External
8.1%
8.2%
16.4%
11.7%
Copying (quotes), TAs
Copying (quotes), External
60.5%
60.5%
40.6%
40.6%

Note: Fall N=86, Winter N=128

The TAs' explanations for these results vary. By the end of the winter term the TAs had more experience with graduate-level writing in their courses and had received several different kinds of feedback (such as formative and summative assessments of their writing from professors and via newly established peer-mentorship relationships) from disciplinary audiences that may have helped strengthen their writing. The TAs also had more practice applying CP citation codes to writing, which may have made their coding more consistent with that of the external coders. Further, the acceleration of TA development may be attributed to TAs actively struggling to clarify citation concepts (like paraphrasing and patchwriting) and practices.

TA Critical Reflections: Self-Study and Transformation

The TAs' reflection responses and interviews suggest that the experience of coding their own papers hastened and deepened their disciplinary and writing development as TAs. Author G, for example, explained how the heuristics influenced her in both personal and professional ways. As a result of the coding experiences, Author G said she (1) felt "more knowledge about audience expectations;" (2) had more frequent engagement with disciplinary-specific mentors about audience expectations as a result of citation analysis activities; (3) shared coding results with mentors and peer-mentors to assign specific meaning to those results; (4) used coding strategies to plot revision strategies for seminar papers; and (5) used citation analysis (in short hand form) to read disciplinary publications as models. During our final interview, she described these experiences in greater detail, describing how "self-study" also influenced her preparations for teaching an undergraduate writing course the next fall:

I'll put it like [this]… I didn't have a clue about [citation practices] before this [coding experiences]. How the hell could I teach it to other people? The short [answer] is that I can't teach it. I can grade it, but can't teach it. TAs can't teach what they can't do and don't think about. Coding my writing showed me what I need to learn to help undergrads write, especially if they will be writing in a discipline other than English Studies.

In this way, Author G identified the coding workshops—and comparisons with external coder results—as a pedagogical intervention in her TA preparation as well as in her development as a graduate student writer joining a disciplinary community. The experiences related by Author G were typical of the entire cohort: the TA participants saw these citation analysis heuristics as formative and influential in both their disciplinary writing development and pedagogical maturation. It was in the reflection processes that followed the coding workshops (written reflections and interviews specifically) that TAs identified the coding heuristics as pedagogical and professionalization tools.

These coding experiences were useful to TAs bound for post-secondary classrooms, but also had surprisingly high impact on pre-service secondary teachers as well. Author A, an education student preparing for secondary education, described his coding experiences as useful for preparing for his upcoming student-teaching placement at a local high school:

[this experience was] transformative because first… I didn't get what metacognitive regulation meant at all until this. I read Ed theory that tells us that metacognitive regulation is what experts do; people who follow directions step by step aren't doing anything metacognitively…The big lesson here for me is that I can't teach writing by directing students through steps. They have to understand what the steps do for them, especially when they do research projects, and then I have to engage them metacognitively if they [get] any chance of using writing across the curriculum. Listening to my literature friends talk about the coding results was…wild and crazy because we read the results so totally different. I was like—WOW!—how does a teenager navigate this, especially if we don't give them a tour?

Author A attributed his new understanding of citation epistemologies and practices to the coding workshops. Notably, he described his realizations as a writer as "secondary" professionalization when compared to the pedagogical lessons the coding workshops offered him. Author A and I discussed how these two progressions—pedagogical formation and writerly development—were linked, especially for teachers engaged with teaching disciplinary writing outside of writing programs and English departments. For TAs, these coding workshops offered opportunities to increase their metacognitive awareness and metacognitive self-regulation as writers simultaneously. The heuristics also offered them new perspectives on how students develop such metacognitive skills. Author A described it as follows: "the time I spent studying my sem[inar] paper turned into time spent studying how writing generally works" and that realization "will become praxis for me" in the future. By gesturing towards praxis, Author A suggested that citation analysis practices such as the coding heuristics used in this study allowed TAs to investigate theories and practices of source-based writing simultaneously and therefore substantially.

Conclusions

This pilot study of ten TAs suggests that citation analysis coding workshops are a promising tool in professionalizing TAs both as writers and as teachers. Further, the initial research presented here suggests that the heuristics—coding workshops and reflection activities—used to prompt TAs to study their own writing while considering their own disciplinary citation practices—were useful tools for graduate students who will become professors and scholars. Additional studies (with larger cohorts from other disciplinary contexts, with a longitudinal orientation, and at different kinds of institutions) investigating the intersections of pedagogical and writerly TA development are needed to further assess the value of the citation analysis heuristics for metacognitive as well as practical purposes.

This pilot study offers four working theories for consideration and further exploration:

The most promising result of this pilot study is the potential of such coding heuristics, developed through sustained graduate education programming, to prepare TAs who become knowledgeable about student source use, including disciplinary differences. Such teaching interventions will benefit graduate students as they learn to write and conduct research independently, certainly, but they can also spark transformative movements within departments and even institutions as TAs share what they've learned about writing with the undergraduate students they teach, their fellow graduate students, and their faculty mentors. In short, the citation analysis tools derived from the CP provide a useful heuristic that can be used to engage TAs as agents of change in the WAC movement.

Appendix - Basic Open-Ended Interview Questions Posed to TAs (Pre- and Post-Coding)

Background information (Pre-Coding):

  1. What is a citation? What do we mean when we talk about "citations"? What about when we talk about "source use"? What about "research-based writing"?
  2. What were you taught about these phenomena, across your education?
  3. How would you define plagiarism?
  4. What citation style have you most frequently been asked to use? Do you know why you've been asked to use that style?
  5. What are the features of MLA citation style? Values?
  6. What is your greatest strength as a writer? As a researcher?

Post-Coding information (Post-Coding):

  1. Describe the coding process as you experienced it. Which moments were most significant for you? Why?
  2. Describe your coding results. What did you find out about your writing? What do you think the results might mean?
  3. What did you learn about the citation style you were using?
  4. What did you learn about the different source use moves we've talked about (summarizing, paraphrasing, patchwriting, quoting)?
  5. Did your coding experience reveal anything in particular to you about yourself as a writer? About how citations work?
  6. What encouraged you, based up on the coding you did? What worried you?
  7. What is a citation? What do we mean when we talk about "citations"? What about when we talk about "source use"? What about "research-based writing"?

References

Berrett, Dan. (2012). Freshman composition is not teaching key skills in analysis, researchers argue. The Chronicle of Higher Education. Retrieved March 25, 2015 from http://chronicle.com/article/Freshman-Composition-Is-Not/131278/

Brooks-Gillies, Marilee, Garcia, Elena G., Kim, Soo Hyon, Manthey, Katie, & Smith, Trixie. (Eds.). (2015a). Graduate writing across the disciplines. [Special issue]. Across the Disciplines, 12(3). Retrieved from https://wac.colostate.edu/atd/graduate_wac/index.cfm

Brooks-Gillies, Marilee, Garcia, Elena G., Kim, Soo Hyon, Manthey, Katie, & Smith, Trixie. (2015b). Graduate writing across the disciplines, introduction. [Special issue on graduate writing across the disciplines]. Across the Disciplines, 12(3). Retrieved from https://wac.colostate.edu/atd/graduate_wac/intro.cfm

Brown, Ann L. (1992). Design experiments: Theoretical and methodological challenges creating complex intervention in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178.

Burk, J. E. (2001). Preparing the professoriate: instructional design training for TAs. Journal of Graduate Teaching Assistant Development, 8(1), 21-26.

Collins, Allan, Joseph, Diana, & Bielaczyc, Katerine. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15-42.

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.

Howard, Rebecca Moore, Serviss, Tricia, & Rodrigue, Tanya. (2010). Writing from sources, writing from sentences. Writing and Pedagogy, 2(2), 177-192. Retrieved from https://journals.equinoxpub.com/index.php/WAP/article/view/6723

Jamieson, Sandra. (2013). "Reading and engaging sources: What student's use of sources reveals about advanced reading skills." Across the Disciplines, 10(4). Retrieved February 23, 2016, from https://wac.colostate.edu/atd/reading/jamieson.cfm

Jamieson, Sandra & Howard, Rebecca Moore. "Phase I Data." The Citation Project. Retrieved from http://site.citationproject.net/results/phase-i-data/

Jamieson, Sandra & Howard, Rebecca Moore. (2013). Sentence-mining: Uncovering the amount of reading and reading comprehension in college writers' researched writing. In Randall McClure & James P. Purdy (Eds.), The new digital scholar: Exploring and enriching the research and writing practices of nextGen students (111-133). Medford, NJ: Information Today.

Jamieson, Sandra & Howard, Rebecca Moore. Terms and definitions. The Citation Project. Retrieved from http://site.citationproject.net/about/terms-and-definitions/

Jamieson, Sandra & Howard, Rebecca Moore. What is the Citation Project? The Citation Project. Retrieved from http://site.citationproject.net/

Park, Chris. (2004). The graduate teaching assistant (TA): Lessons from North American experience. Teaching in Higher Education, 9(3), 349-361.

Pecorari, Diane. (2003). Good and original: Plagiarism and patchwriting in academic second-language writing." Journal of Second Language Writing, 12, 317-345.

Roig, Miguel. (2001). Plagiarism and paraphrasing criteria of college and university professors. Ethics and Behavior, 11(3), 308-323.

Silverman, Gillian. (2002). "It's a bird, it's a plane, it's plagiarism buster!." Newsweek. Retrieved from http://www.newsweek.com/its-bird-its-plane-its-plagiarism-buster-147537

Turnitin. (2015).What we offer: Plagiarism prevention, online grading, and peer review in one easy-to-use service. Retrieved from http://www.turnitin.com/en_us/what-we-offer

White, Howard D. (2004). Citation analysis and discourse analysis revisited. Applied Linguistics, 25(1), 89-116.

Notes

[1] The Citation Project (CP) research team developed definitions for these terms across the different stages of the research, ultimately settling on the following definitions:

[2] The Citation Project became a national study of 800 pages of research writing produced by 174 undergraduate students enrolled in required first-year writing courses at 16 U.S. colleges and universities. A team of coders analyzed citations within this corpus of student work, labeling each citation moment as either a moment of summary, paraphrase, quotation, or patchwriting. The project expanded the kinds of coding conducted on the student writing, eventually describing the type, length, and reading level of sources used in these same moments of citation. Raw data and results of the coding of this data came be accessed through the Citation Project official website.

[3] Undergraduate papers in the CP corpus are between 6-8 pages, while graduate were typically 15-20 pages; the coding sample of undergraduate papers is four pages so the coding sample of graduate students is six pages.

[4] Reflective prompts asked students questions such as: What surprised you about your own writing as you coded your citations? Why did it surprise you? Is there any trend in your writing sample you plan to address going forward? Why? How do your citations align with the kinds of citations you read in the scholarship of your field?

[5] Sandra Jamieson and I served as external coders of TA papers.

[6] Participating TAs worked with me to determine the best way to reference them. The group decided to designate each participant with a letter. Authors A-D represent students specializing in education and rhetoric and composition; Authors E-J come from the wide disciplinary community of literary studies.

Contact Information

Complete APA Citation

Serviss, Tricia. (2016, September 30). Using citation analysis heuristics to prepare TAs across the disciplines as teachers and writers [Special issue on TAs and the teaching of writing across the curriculum]. Across the Disciplines, 13(3). Retrieved from https://wac.colostate.edu/atd/wacta/serviss2016.cfm