Abstract: The body of literature on a Writing-to-Learn (WTL) approach in math courses offers up a variety of assignment types from which to choose. However, few of these articles provide empirical evidence on the ways these writing assignments contribute to students' learning. This mixed-methods study, conducted at the University of New Mexico, a Hispanic-Serving Institution, examines the effect of WTL assignments on students' success in two "killer courses": a Survey of Math class for non-STEM majors and a Calculus I class for STEM majors. While the quantitative results did not prove statistically significant, the qualitative results suggest that high-impact assignments are those that ask students to focus on procedural knowledge, or analyzing the process, rather than simply solving for the right answer.
At the start of the 2013 academic year, the University of New Mexico (UNM), a "very high research" Hispanic-Serving Institution (HSI), published internally a list of undergraduate "killer courses" offered in the fall 2012 semester. These killer courses, which often serve as the "gateway" to the major in that subject, are known, as the moniker suggests, to "'kill' a student's GPA, motivation, academic progress, scholarship eligibility and interest in remaining in college" (Barefoot, 2013). The anecdotal causes for these effects include students' lack of academic preparation in a subject (especially mathematics), a lack of placement procedures for a class, (large) class size, and a lack of early feedback to students (Barefoot, 2013). At UNM, the "killer course" designation was given to those aggregated sections of a particular course with a total fail rate of 20% or higher, where failing is a grade of C- or below. Seventy-nine courses at UNM made the list in fall 2012, with STEM (science, technology, engineering, and math) courses making up 33% of these. Among the killer STEM courses, 42% were math classes (or 11% of all killer courses). Students' difficulty in the STEM courses at UNM reflects a larger trend as seen in the nationwide attrition of STEM students (see Chen & Soldner, 2014). Of particular concern is the high dropout rate or switching out of STEM majors by women and minority students, as illustrated in the 2010 U.S. Commission on Civil Rights briefing report "Encouraging Minority Students to Pursue Science, Technology, Engineering and Math Careers".
Recently Cristyn had co-developed a new first-year Stretch/Studio composition curriculum that focused on providing students with additional in-class support on their writing assignments. The new curriculum resulted in the elimination of "remedial" writing courses at UNM and increased student pass rates that exceed those of the traditional composition courses offered. Following this experience, Cristyn became interested in the possible intersectionality between "killer courses," as a framework for identifying courses where students could use more support, and writing, as a tool for increasing students' success. "Writing-intensive courses" are recognized by the Association of American Colleges and Universities as one of 10 high-impact practices with a positive effect on student engagement (Kuh, 2008). Therefore, Cristyn was interested in investigating the effects of a "writing-to-learn" (WTL) approach in killer courses across the curriculum. Of particular importance was the dramatic, positive effect that such an "educationally purposeful" activity could have, as reported by Kuh (2008), on the GPA and retention rate of minority students in their first and second year, respectively (pp. 18-19), as UNM's beginning freshmen class profile most recently reported in fall 2015 was 52% Hispanic (Office of Institutional Analytics, 2015). As mathematics is one of the leading "killer course" subjects at UNM, it seemed like a good place to start.
UNM does not have a formal Writing Across the Curriculum program beyond the individual, isolated efforts of faculty and graduate students on campus. Therefore, independently, Cristyn teamed up with Karen, a full-time lecturer with more than 20 years of experience teaching math courses that span the curriculum. For this collaborative, IRB-approved research project, our main objective became that of assessing the effectiveness of Math writing assignments on students' learning. In fall 2014, we began a pilot study with MATH 129: Survey of Math, a class for non-majors. Karen was already incorporating two writing assignments of her own into the course, and the curriculum was one which we thought we might have the greatest influence on since Karen is the program coordinator for the course. Also, while not officially designated a "killer course" in the traditional sense, as the only way to "fail" MATH 129 is to withdraw from it, the course is one where Karen often sees students repeatedly re-enroll as they attempt to earn a C or better to satisfy the university's core curriculum requirements.
Following our fall 2014 pilot study during which we tested new and revised writing assignments, student surveys, and focus group questions, we continued our research in spring 2015 with MATH 129 and included an additional course in our investigation that Karen would also be teaching that semester: MATH 162: Calculus I. MATH 162 is a designated "killer course" for STEM majors with an aggregated fail rate in fall 2012 of 41.6% across sections. With the course instructor a controlled variable, we wanted to evaluate the possible effect of writing assignments on students' learning in MATH 129, an un-official killer math course for non-STEM majors, and MATH 162, a designated "killer" and required course for STEM majors. We expected to find that the non-math majors would respond more positively than the STEM majors to the writing assignments and that the assignments would contribute more to non-math majors' learning, as we thought the non-math majors, who are largely enrolled in humanities courses, would be more familiar with writing assignments as a form of learning and assessment. On the other hand, STEM majors, we predicted, would perhaps be more familiar with solving numerical problems as a form of developing and assessing their knowledge about math concepts and, therefore, less amenable to writing assignments related to math as a tool for learning and assessment.
As we prepared to collaborate on the design of writing assignments for Karen's two math classes and assess their impact on students' learning, we reviewed a number of existing articles that describe the kinds of WTL assignments that might be used in a college math course. We began with "Writing to Learn Mathematics," in which Russek (1998) describes a range of writing assignments for use across the math curriculum. For example, in the course Theory & Methods of Mathematics, as described by Russek, students learning about mathematics education respond to prompts such as "What is Mathematics?" or "What Makes an Effective Math Teacher?" (pp. 36-37). In an introductory algebra class, students solve for a weekly "math trick," explaining in a couple of paragraphs why it works. Or in Algebra Part I, students write a "mathography" in which they describe their feelings about and experiences with math. Other courses Russek describes require students to solve mini-research problems and submit an end-of-semester portfolio. For a "service" or "core" course similar to MATH 129, Crisman (2008) describes asking students to write a three-page essay on a "historical mathematician." Crannell et al. (2004) provide a range of problem-based written assignments for use in Survey of Math to Calculus courses. For these assignments, students receive problems written in the form of letters from a well-defined (yet fictional) character and then offer responses that include the solution and an explanation of the solution "in precise mathematical prose" (p. 3).
Â An example of a math assignment for STEM majors comes from Goodman (2005), who asks his calculus students to write to a friend or family member a one-page, weekly letter in which the student summarizes two or three main concepts or issues discussed in class that week and comments on any aspects the student is struggling to understand. Parker and Mattison (2010) describe in detail asking math majors and minors to write a math course textbook on proofs for their peers. And most recently in "Transfer and the Transformation of Writing Pedagogies in a Mathematics Course," Bryant, Lape, and Schaefer (2014) promote the genre of proof writing as a way to transform mathematics instruction through writing.
As demonstrated by the examples above, the body of literature on WTL in math offers up quite a range of assignment types to choose from, including those that require students to write about math in historical terms, others that ask students to define mathematical concepts, and still others that ask students to articulate their process as they solve mathematical equations. The authors often offered anecdotal evidence regarding the positive ways that students responded to the assignments. However, very few articles offered empirical evidence on which and how these writing assignments contributed to students' learning. Nor did they specifically address increasing students' success within a "killer course" framework. Therefore, with this study, we seek to add to the above body of literature with an empirical examination of whether and which kinds of writing assignments impact students' learning in a math class for majors and a math class for non-majors and to what effect within a killer course context. Using what Haswell (2005) calls "RAD" research—that which is repeatable, aggregable, and data-drive—the main research questions driving this study are as follows:
Participants: In spring 2015, students were recruited during the third and fourth week of the semester from a section of MATH 129: Survey of Math and a section of MATH 162: Calculus I, both taught by Karen. Of the 40 students originally enrolled in MATH 129, three students dropped the course and 37 completed the class, with 23 of these students (62%) consenting to be a part of this study. Of the 22 students originally enrolled in MATH 162, three students dropped while 19 completed the course, with 13 students (68%) consenting to participate.
Design of Writing Assignments: After reviewing example math writing assignments found in the literature, we collaborated on the creation of new assignments on topics specific to MATH 129 and MATH 162 and also modified assignments Karen was already using in MATH 129. As a guide for the creation or modification of these assignments, we incorporated the following five principles of assignment design as outlined on the WAC Clearinghouse website:
Please see the Appendix for the writing prompts used in both MATH 129 and MATH 162 for this study.
Data Collection: The following data were collected from each group of consented students enrolled in Math 129 and Math 162: 1) a pre-survey in which students reported demographic information including gender, race/ethnicity, native language(s), other languages spoken, college major, status as first-generation college student (e.g., true or false), other college math classes taken, as well as their attitudes about math, writing, and themselves as writers; 2) post surveys following students' submission of each written assignment in which students reported how the writing assignment contributed to their learning; 3) an end-of-semester student focus group; 4) students' written assignment, tests, and course grades. Additionally, Cristyn interviewed Karen about her perceptions of the impact of writing assignments on students' learning of course content.
Students in each math course under examination in this study were randomly divided into groups A or B. For topic #1 covered in each course, Group A completed a problem-solving assignment and an additional writing assignment on that same topic (i.e., the experimental treatment). At the same time, Group B completed the same problem-solving assignment as well as additional problems (e.g., homework in their course textbook) related to the assignment topic, without completing the writing assignment (the control). (The additional problems were given to Students B so that "time on topic" might not influence students' attitudes about or success with the topic.) Following the completion of instruction on topic #1 and the submission of students' written assignments or additional problems (but before receiving a grade on these assignments), Groups A and B were then given the post-survey that asked about their attitudes and level of confidence with course content and, if applicable, whether the writing assignment had contributed to their learning.
For course topic #2, Group B completed a problem-solving assignment and an additional writing assignment on that same topic while Group A completed the same problem-solving assignment as well as additional similar problems, without completing the writing assignment. In this way, both Groups A and B had the opportunity to receive the treatment of completing an additional writing assignment on course content, although on separate assignments. As we hypothesized that the writing assignment would help students with their learning, we did not want one group to be penalized by not having the opportunity to improve their learning through completing a written assignment.
In the last week of the semester, consented students in each course voluntarily participated in a focus group in which they responded to questions about the strengths and challenges of the course as well as how the writing assignments might have impacted their learning (or not). Karen was also interviewed about her reflections on the contribution of the writing assignments to students' learning. Students' written assignment grades, test grades, and final course grades were also collected. Tables 1 and 2 below illustrate the research design for the data collection methods used in this study for each class.
Students' Attitudes About Writings' Impact on Their Learning: Over the course of the semester, upon the submission of each writing assignment, students in both MATH 129 and 162 were asked how well a particular writing assignment had contributed to their learning of the course content (Research Question #1). As illustrated in Table 3, MATH 129 students' survey responses were somewhat mixed as the contribution of the assignment to their learning depended on the writing assignment.
Students responded less positively to the Articles Assignment and were less sure about its contribution to their learning. For this assignment, students were asked to find two articles in the news that incorporate mathematics (preferably beyond "simple arithmetic" or statistics), summarize the articles, and explain how the math illustrated exemplified the use of mathematics in the everyday lives of a general population. Estes (1989) defines this category of college-level math writing as "ordinary narrative," referring to that difficult-to-define genre "the academic essay." As Estes explains, the overall purpose of an assignment like this one might be to learn something of the history or wider application of mathematics while also learning to express one's self in a precise way. The purpose of the Articles Assignment for MATH 129 was to raise students' awareness of the transferability of math beyond the classroom and its applicability to students' everyday lives. It was expected that the MATH 129 students, non-math majors, would prefer this type of narrative assignment as it would be a form of assessment most familiar to them and didn't actually require an understanding of math concepts. However, a majority of students responded negatively to it in terms of how it contributed to their learning, expressing confusion about the assignment's purpose or relevance to the class. The following comments came from the focus group interview:
MATH 129 - Student (S) 4: …the articles, I didn't really feel like those were necessary at all because they didn't really help me at all.
MATH 129 - S1: Well, I remember her comment on the articles was, the assignment for the articles was, "Read an article related to math and write about it." So I can't remember exactly what my two were on, but it was kind of the same thing, you know, how are terrorists funded, something like that, so all I thought she was looking for was to point out that there was a bit about math or there was something math related and what we got back was, on mine, there wasn't enough statistics, so I didn't really get that.
The single advantage that students reported for the Articles Assignment was that, for some, if they didn't understand the math in later assignments and on tests, which would bring down their grade, the narrative assignment was one way they might improve their course grade.
On the other hand, as indicated in Table 3, MATH 129 students responded more positively to the Reaction Paper on Euclid's Proof and the Geometry and Consumer Math memo assignments. These assignments fall under what Estes (1989) categorizes as "technical writing". As the author explains, this type of writing can help students clarify their thinking on a mathematical concept by requiring them to explain "some mathematical or statistical procedure or result" (p. 11). In the end-of-semester focus group, MATH 129 students explained why they found these "technical writing" assignments to be more useful to their learning. Regarding the Reaction Paper, students reported that being required to explain the proof made it easier to process and understand this course concept, as the following student comment illustrates:
MATH 129 - S5: I had no idea what exactly the method was at first and then I kind of figured it out as I was writing about it. And that kind of helped me because that's kind of how I learn. Because I'm an English major, so I'm all into writing and stuff like that, so [writing about Euclid's proof] helped me figure it out more easily than just having all of the, than just have it lectured on and stuff because I had to research it and put it in my own words.
In a similar manner, students found that the Geometry and Consumer Math memo assignments helped them clarify their thinking on those topics. For example, for the Geometry Memo Assignment, students were required to describe the process by which they calculated the amount of paint (two coats) and water needed to cover and fill a new pool and how they came to their answer. Students wrote the memo for submission as evidence to a small claims court following the cracking of paint and leaking of water from the pool. Students reported that they found it helpful to have to write out the systematic process by which they came to their answers for the geometry problem being solved, as reflected in the following student comment:
MATH 129 - S3: With the memo, you talked about the process of what you did, so you had to go through each step. So instead of seeing the step with only numbers, you were also doing that with words. That was also the way [the teacher] kind of taught the class. Like sometimes she would put a sentence of the step you were doing. You know, she would like write the sentence out for the step. That's kind of what the memo was for me.
The technical writing assignments were also greatly valued by the STEM majors enrolled in MATH 162 for the contribution these assignments made to their learning. Of the 11 out of 13 consented students who completed one of the two writing assignments (Group A or B) and who responded to the survey item, 100% reported that the writing assignment had positively contributed to their learning, as seen in Table 4.
This sentiment was repeated in the MATH 162 end-of-semester focus group when 100% of focus group participants reported that the writing assignments on linearization or related rates contributed significantly to their learning. The following is a representative student comment from this group:
MATH 162 - S1: I think the concepts were very difficult to grasp at first, so I think the papers did help me kind of organize my thoughts in saying, well this is why we take the derivative and subtract from the original and then multiply it by it, so it's one of those things where writing it down helped me figure out this is why we do it, not just this is the equation of how to get it done.
This comment is significant in that the student recognizes it is not enough to simply memorize a type of problem or algorithm in order to reach the correct answer. Rather, it is also important to understand how and why one arrives at that number for the number to have more meaning and in order to interpret its significance.
Students' View of Writing's Impact on the Transfer of Knowledge: The technical writing assignments not only helped students to understand the various concepts they were learning, but, as MATH 162 students reported, the act of writing the assignments increased the probability that this new knowledge would carry over to like problems and help students with their unit and final exams. In other words, by having to write out the process and their understanding of the new concepts, students felt more confident about completing problems that looked similar to ones they had already seen but that might contain different variables. This connection, the students explained, would not have been possible with just practicing and completing multiple additional problems:
MATH 162 - S4: By doing the writing assignments, it makes you get a really good visual map of how to do the problem, not just do it one way, but do it all the different ways. Like, so you could put it in different forms, and so if [the instructor] put it in a different form [on the exam] and you're just practicing one type of problem, then you wouldn't know how to do the form, but the paper forces you to really grasp the concepts and then you'd be able to do it in different forms.
MATH 162 - S6: Yeah, and if you just memorize the process for one specific type, you're not going to have any idea of what to do when confronted with the same problem just set up differently.
These students recognize the transferability of their knowledge from one problem to another, as the result of writing out the process for solving a particular kind of problem.
Written Assignments Versus Test Grades: In terms of research question #2, "Does the scaffolding of smaller writing assignments increase students' success on the completion of larger summative assignments (e.g., unit exams, final exams, etc.) in the course?", the results were limited, despite students' positive reports of the writing assignments' impact on their learning. For both Groups A and B in MATH 129 and MATH 162, there was little to no correlation (and no statistical significance) between students' grades on their written assignments and their score on related test items on the unit exam.
For MATH 129, using Pearson's r to examine the degree of relationship between Group A's scores on the Geometry Memo and this same group's scores on the Geometry test item, we did not find evidence of a significant association between the memo and the test item scores (p-value = 0.2377). These results are illustrated in Figure 1.
Similarly, when using Pearson's r to examine the correlation between Group B's scores on the MATH 129 Consumer Math Memo and their score on the Consumer Math test item, we did not see evidence of a significant association between performance on the memo and performance on the test item (p-value = 0.8849) as illustrated in Figure 2.
Similarly, when examining the test scores of each group, we did not find a significant relationship between students having completed the memo writing assignment and their performance on the corresponding test item. As seen in the box plot in Figure 3 and its accompanying five-number summary in Table 5, the median test score on the Geometry test item for Group A, who wrote the Geometry Memo, is 8.5, while the median score for Group B, who did not complete the memo, is lower at 6.75. Despite the difference in median scores, the Wilcoxon rank sum test with continuity correction determined that there is no statistically significant difference between the scores of the two groups (p-value = 0.08947).
As with the Geometry test item, and contrary to our expectations, Group A also performed better on the Consumer Math test item, earning a median score of 8.5, after not having completed the Consumer Math Memo, while Group B, who did complete the Memo, earned a median score of 6.25. (See Figure 4 and Table 6 below). However, the Wilcoxon rank sum test with continuity correction determined that there is no statistically significant difference between the scores of the two groups (p-value = 0.1282).
While counter to what we were predicting, it is perhaps not surprising that Group A performed better on both test items when considering that, although both groups were randomly assigned, Group A did better in the MATH 129 class overall, with 92% passing, compared to Group B, of whom 50% passed the course. (Student pass/fail rates for MATH 129, along with MATH 162, are reported in Table 9 below.)
For MATH 162, Pearson's r was also used to examine the correlation between the two written assignments in the course (linearization for Group A and related rates for Group B) and the corresponding test items. Because there was no variability in the scores for the Linearization Memo completed by Group A (i.e., everyone scored 10/10), it is impossible to compute Pearson's r or assess much from the scatterplot found in Figure 5 below.
The box plot in Figure 6 and the accompanying five-number summary in Table 7 illustrate that the median test score for Group A (who completed the Linearization Memo) was higher (7.5) than that for Group B (6). However, the Wilcoxon rank sum test with continuity correction again illustrates no statistically significant difference between the scores of the two groups (p-value = 0.08149).
On the topic of Related Rates, despite the variation in how students scored on the written assignment, there was no significant relationship upon using Pearson's r between Group B's scores on the written assignment and how these same students scored on the Related Rates test item (see Figure 7.)
As reported above with MATH 129, a group's completion of a written assignment in MATH 162 does not appear to have had a significant effect on the group's performance on the test item. As illustrated in Figure 8 and Table 8 below, the median test score for Group A (who did not complete the Related Rates Memo) is higher (9.5) than that for Group B (8.5), who did complete the Related Rates Memo. However, again, after performing a Wilcoxon rank sum test with continuity correction, there is no statistically significant difference between the scores of the two groups (p-value = 0.5694).
And as with MATH 129, Group A had a higher pass rate (86%) than Group B (83%) in the MATH 162 class overall.
Due to a lack of statistical significance, perhaps as a result of small sample sizes, it is impossible to determine in either MATH 129 or MATH 162 a statistical correlation between students' performance on a writing assignment and their scores on a corresponding test item. Therefore, the null hypothesis for research question #2 should be accepted for this study: the scaffolding of smaller writing assignments did not increase students' success on the completion of larger summative assignments in the course. Future research should include a larger number of participants across course sections in an attempt to increase the significance of these findings. Additionally, more test items related to the writing assignment topic should be included, and these test items should also seek to assess the type of knowledge developed in the writing assignments while addressing the learning outcomes of the course. In this study, while we collaborated on the creation and revision of writing assignments, we did not focus on the revision or modification of test items. Perhaps if we had revised the way that students' procedural knowledge was being assessed, we might have seen a clearer correlation between students' performance on the written assignment and the exam.
Student Pass/Fail Rates: Research question #3 asks, "Does a focus on writing as it relates to course content improve students' success (i.e., pass rate) in the course?" The pass rates (as defined by a grade of C or better) for the consented students in this study for MATH 129 and MATH 162 were 74% and 85%, respectively. The total pass rates for the whole class for MATH 129 was 76% and for MATH 162 was 84%. These pass rates along with the comparable fail rates for each class are illustrated in Table 9.
When we compare the pass/fail rates of Karen's Spring 2015 course section for MATH 129 included in this study with those rates for the previous four semesters Karen taught the class, there is a large variation in rates, as seen in Table 10. The highest pass rates were in Fall 2014, with the course section in which we piloted the writing assignments. Because there is so much variation in the pass/fail rates from one semester to another, it is difficult to compare one (or two) semesters of WTL intervention with a number of semesters without the intervention. Regardless of whether this study produced pass/fail rates of statistical significance, pass/fail rates for course sections including and not including the intervention would need to be recorded over multiple semesters.
The same variation in pass/fail rates is seen among the course sections of MATH 162 Karen has taught since Spring 2013, as seen in Table 11. (Karen did not teach MATH 162 in the 2012 academic year.)
While the pass rates for MATH 162 are significantly higher for the course section in Spring 2015 where we introduced the WTL intervention, we cannot make any claims about the writing intervention as the cause for these higher pass rates. While the instructor remained constant across these course sections, there are a number of intervening variables that make the MATH 162 course section under study different from the others. For example, the students enrolled in the second section of Spring 2015 were all freshmen in their second semester (as opposed to students at various stages of their studies), the class size was significantly smaller than for other course sections (19 students were enrolled versus the 60 students originally enrolled in section 1 of the same semester), and Karen taught both the lecture and the lab segments for the course section under study (whereas the lab is typically taught by a teaching assistant).
Because we are particularly interested in the impact WTL assignments may have on women and minorities enrolled in STEM classes, the pass rates for consented students who identified as women or minority are given in Table 12. It is important to note that the results for gender and race/ethnicity given in Table 12 are not mutually exclusive (i.e., female pass rates include those for white and minority females). Also, we are aware that generalizations should not be made about the effect of a WTL approach on gender and race/ethnicity from such a small sample size of students.
When compared to the total pass rates for spring 2015 for consented students reported in Table 9 for MATH 129 and MATH 162, female and minority students passed at a higher rate compared to the total consented student population in each course. (Comparisons cannot be made to the total enrolled population in the two course sections as demographic information was only recorded for those students who consented.) Future research should continue to track the comparative pass rates and progress of these underrepresented student populations in STEM courses.
Despite the lack of correlation between students' performance on the writing assignments and corresponding test items and despite our inability to make claims about the writing intervention's effect on pass rates, the qualitative data reported from students is still helpful for understanding students' perceptions of what aids them in their learning in the class. In the next section, the qualitative results of the instructor's perception of writing's impact on students' learning are given.
Instructor Interview Data: Following students' submission and Karen's grading of each written assignment, Cristyn interviewed Karen on the following: "Tell me about your impressions of students' writing as it relates to course content. For example, how did the writing assignments impact students' learning (or not)?" Karen's comments paralleled those of her students in two significant ways, as described below.
Karen's View of Writing's Impact on Students' Learning and Her Teaching: Just as students had reported that the writing assignments helped them to clarify their thinking, Karen also reported that the writing assignments helped her to understand what was clear or still unclear in students' thinking about course concepts. And as the students reported, Karen found the writing assignments rather than the problems much more useful in understanding students' thinking on a topic. For example, returning to the geometry project and calculating the surface area of the pool to be painted, Karen could see from the students' calculations submitted in table format that they understood how to calculate the area of the individual sides of the pool. However, what the write up of the process made much clearer to her was that students did not understand which calculations they would need to combine in order to determine how much paint or water they would need to protect and fill the pool. As Karen explained,
Really I can tell more from the paper than even the projects. Because the projects are kind of mechanical. They're broken down into pieces. In little pieces, [students] know this stuff. They know the area of this wall is the width times the height. When you put it all together and they have to describe a whole process, that's where you start to see, you get the normal distribution, right? Some of them, it is very clear that they know what they're doing, right? Some, in parts of the paper, it was like, what are you talking about? And they were confused, okay? Nobody was terribly confused. They got pieces of it, but there were issues in the middle on the papers.
It was recognizing "the issues in the middle on the papers," the part of the calculation process students were not understanding, as made visible by the geometry memo, that was of most use to Karen and her formative assessment of students. As Karen discussed further on in the interview, what can be most difficult with teaching an introductory class such as MATH 129 is trying to understand what it is that students are just "not getting." As Karen is so well versed in the topic, she cannot "remember ever not understanding this." Therefore, the writing assignments help her to know which aspects of math, related to geometry in this case, she needs to spend more time on in order to better support students in their learning. Similarly, Karen is able to use students' confusion or lack of understanding as feedback for revising in-class activities and the scaffolding of writing assignments and test preparation.
Karen's View of Writing's Impact on the Transfer of Knowledge: A second parallel theme between the students' and instructor's comments on the usefulness of the writing assignments was that regarding transferability. As students in the MATH 162 focus group clearly understood, there is a difference between knowing how to get the right answer and understanding the significance of that answer. As Karen reported, the writing assignments helped her see when students understood the how and were arriving at the correct answer but perhaps did not understand the why and what significance the application had or how it could be applied to other contexts. One example the instructor gave was when MATH 129 students worked to calculate the interest on a loan for the unit on consumer math. As was clear from their writing of the memo, students could calculate the interest when told how to do so, but they weren't clear on the rationale behind each step, which limited their ability to apply the math to other contexts:
On the add on method, they didn't understand why you did things in a certain order, but even more so, total interest paid…. Test-wise, I don't think [they'll have a problem]. I expect they're going to do very well on the test. I think they're good at saying, "Okay, this is what I do when I have this. I memorize the math and I do it." Okay. As far as anything in the real world, you know, when they have to buy a house, yeah, huge implications. They can't figure out, you know, even a basic idea of how much my payments are going to be or how much do I want to put down. Or even buying a car or anything like that. So that's where I think it will impact them the most.
As the MATH 162 students realized in their writing of their assignments, and as Karen acknowledged in her reading of MATH 129 students' assignments, it is important to focus not only on the what (i.e., the correct numerical answer) but also on the how (how did I arrive at that answer) and the why (why does this method work and why is it useful) if one is to successfully transfer knowledge of a particular math concept to other contexts.
The data reported in this article, particularly that collected through the student focus groups and instructor interviews, helps us to reconsider one basic assumption about well-designed WTL assignments for "killer courses" and reinforce other aspects we know to be true of these assignments as they contribute to students' positive perceptions of their learning. The assumption to reconsider is that (non-) math majors would prefer narrative assignments to other kinds of written assignments due to students' familiarity with (and perhaps the perceived ease of) these assignments. Rather, as the students in this study report, "technical writing" assignments that ask students to reflect on the problem-solving process may have more to offer students in terms of their learning of new information. The traditional "academic" or "research" paper has already lost favor with many in the field of rhetoric and composition for its lack of authenticity and usefulness (Fister, 2011), something students may have understood intuitively all along. There are perhaps, then, even fewer reasons to promote these kinds of writing assignments across the curriculum, as they are even less likely to be viewed by students as useful for understanding new course content, particularly within killer course subjects, and less useful for instructors trying to assess students' understanding of these subjects.
Therefore, if we are to listen to our students, and the learning of mathematical concepts is the primary goal, a WTL approach in killer math courses will place an emphasis on using writing to clarify students' thinking about mathematical concepts, rather than simply (or predominantly) give them the practice of writing clearly and concisely. Of course as students come to better understand the mathematical concepts about which they are writing, their ability to express their understanding should improve as well. However, as for MATH 129 and her own narrative assignments, Karen has decided to no longer use the Articles Assignment with students, in favor of spending more time on assignments that help students build their procedural knowledge.
This procedural knowledge, or what the WAC Clearinghouse terms "analyzing the process," is what makes for a high-impact approach to learning and teaching. As this study bears out, "a more useful approach to process analysis—from the learner's point of view—is to trace in writing the steps required to complete the process or to capture the thinking that leads from one step to the next" ("Analyzing the process," n.d., emphasis in the original). This approach also enables students to receive more individualized help as "teachers can look over the process analyses to see if students have misapplied fundamental principles or if they are making simple mistakes" ("Analyzing," n.d.). The student and instructor comments reported in this study above reinforce this stated best practice.
Furthermore, as students' thinking about course concepts is still under development during the completion of a writing assignment, these assignments should be viewed as just one step in the students' process of coming to understand new content, particularly within a killer course. These assignments should be used as low- and mid-stakes assignments for which students simply receive acknowledgement for having completed them, for example, or be worth a limited percentage of a total course grade. The heavy weighting of WTL assignments is counter-intuitive, as students should not be heavily graded on what it is that they're still trying to understand. For this same reason, WTL assignments should be used largely for formative assessment, so that students and instructors can receive on-going feedback on what it is students do and do not understand. Students can use this feedback to focus their studying and requests for help while instructors can use these formative assignments when making decisions about supplementary coverage of course topics. In this way, we may increase students' positive attitudes about their learning and in turn improve their success within a killer course context.
Finally, if WTL activities are to have a greater impact on improving pass rates and perhaps decreasing attrition rates of students in killer STEM courses such as MATH 162, our summative assessments should reflect the writing prompts that we ask students to respond to as a part of their formative assessment. This is to say that our exams should reflect our writing-to-learn assignments in terms of what it is we want our students to value about mathematical concepts: not just the what but the how and the why.
Analyzing the process. (n.d.) The WAC Clearinghouse. Retrieved from https://wac.colostate.edu/intro/pop5i.cfm
Barefoot, Betsy O. (2013). Killer courses: Understanding and overcoming roadblocks to first-year academic success and retention [PowerPoint slides]. Gardner Institute for Excellence in Undergraduate Education. Retrieved from Retrieved from http://sc.edu/fye/events/presentation/sit/2013/files/CT-27%20Killer%20Courses-%20Roadblocks%20to%20First-Year%20Academic%20Success%20and%20Retention.pdf
Bryant, Sarah, Lape, Noreen, & Schaefer, Jennifer. (2014). Transfer and the transformation of writing pedagogies in a mathematics course. The WAC Journal, 25, 92-105. Retrieved from https://wac.colostate.edu/journal/vol25/bryant.pdf
Chen, Xianglei, & Soldner, Matthew. (2014). STEM attrition: College students’ paths into and out of STEM fields. Washington, D.C.: National Center for Education Statistics and U.S. Department of Education. Retrieved http://nces.ed.gov/pubs2014/2014001rev.pdf
Crannell, Annalisa, LaRose, Gavin, Ratliff, Thomas, & Rykken, Elyn. (2004). Writing projects for mathematics courses: Crushed clowns, cars, and coffee to go. Washington, D.C.: MA.
Crisman, Karl-Dieter. (2008). Write about mathematicians in non-major courses. MAA Focus, 28(8), 20-21. Retrieved from http://www.maa.org/sites/default/files/pdf/pubs/nov08web.pdf
Estes, Paul L. (1989). Writing across the mathematics curriculum. The WAC Journal, 1(1),10-16. Retrieved from https://wac.colostate.edu/journal/vol1/estes.pdf
Fister, Babara. (2011). Why the 'research paper' isn't working. Inside Higher Ed.Â Retrieved from https://www.insidehighered.com/blogs/library_babel_fish/why_the_research_paper_isn_t_working
Five principles. (n.d.) The WAC Clearinghouse. Retrieved from https://wac.colostate.edu/intro/pop10a.cfm
Goodman, Russell E. (2005). Using letter-writing to enhance a calculus course. PRIMUS, 15(4), 298-302. Retrieved from http://www.tandfonline.com/doi/pdf/10.1080/10511970508984124
Haswell, Richard H. (2005). NCTE/CCCC's recent war on scholarship. Written Communication, 22(2), 198-223. doi: 10.1177/0741088305275367
Kuh, George D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Association of American Colleges and Universities. Retrieved from https://keycenter.unca.edu/sites/default/files/aacu_high_impact_2008_final.pdf
Office of Institutional Analytics. (2015). The University of New Mexico Fall 2015 Official Enrollment Report Albuquerque Campus. Retrieved from http://oia.unm.edu/documents/enrollment-reports/fall-15-oer.pdf
Parker, Adam & Mattison, Michael (2010). By the numbers. The WAC Journal, 21, 37-51. Retrieved from https://wac.colostate.edu/journal/vol21/parker_mattison.pdf
Russek, Bernadette. (1998). Writing to learn mathematics. Writing Across the Curriculum, 9, 36-45. Retrieved from https://wac.colostate.edu/journal/vol9/russek.pdf
U.S. Commission on Civil Rights. (2010). Encouraging minority students to pursue science, technology, engineering and math careers. Washington, D.C.: U.S. Government Printing Office. Retrieved from http://www.usccr.gov/pubs/EncouragingMinorityStudentsinScienceCareers.pdf
 While the report's purpose of investigating the high rate of attrition among minority students in STEM fields was a valid one, the three main recommendations described in the report are far from satisfactory: 1) Do not admit STEM students with a "large deficit in academic credentials" without fully informing them of the impact of the deficit on their ability to succeed; 2) provide academic support programs for students with "modest deficits"; 3) recruit qualified K-12 math and science teachers.
 Stretch courses "stretch" a traditional one-semester first-year composition course over two semesters. Studio courses include an additional hour of small group writing lab. I offer acknowledgements to my colleague Dr. Bethany Davila, with whom I worked side by side on developing this new curriculum.
 Math Department policy dictates that MATH 129 students cannot earn less than a D- in the course, which allows students to still earn course credit toward graduation (although their GPA will be negatively affected). Students, however, do need to earn a C or better in the class to meet core curriculum requirements for graduation.
 As classrooms are unpredictable, dynamic spaces, the method for data collection was altered slightly as illustrated here for each class due to the instructor's thoughtful modifications to the course schedule in response to students' needs. Therefore, the tables do not look exactly the same for MATH 129 and MATH 162. However, overall, the same experimental design was used in both classes.
 Not to be confused or conflated with the robust field of technical communication.
 Perhaps there is a parallel to be made here to teaching to the test versus teaching to develop life skills.
Elder, Cristyn L. & Champine, Karen. (2016, December 26). Designing high-impact "writing-to-learn" math assignments for killer courses. [Special issue on WAC and high-impact practices]. Across the Disciplines, 13(4). Retrieved from https://wac.colostate.edu/atd/hip/elder_champine2016.cfm