Good writing research is characterized by evidence that is trustworthy, applicable to multiple practical settings, consistent and transparent about its position—regardless of whether a qualitative or a quantitative approach is used. Qualitative and quantitative writing research both require standards for good evidence, even though the articulation of criteria in the two approaches is different. Below, we provide a description of high quality writing research practices. While individual descriptors might not apply equally to all approaches, editors and authors can refer to these guidelines in assessing the quality of chapters.
|Techniques for Quality in Quantitative Writing Research
||Quality Criteria in Quantitative Writing Research
||Quality Criteria in Qualitative Writing Research
||Techniques for Quality in Qualitative Writing Research
The extent to which observed effects can be attributed to the independent variable
- Determine your sample size relative to your research objective.
- Describe details of the educational context and intervention.
- Minimize participant attrition and provide information on non-responses and subject withdrawals.
- Standardize experimental conditions.
- Use control groups whenever possible and ethically sound.
- Discuss findings relative to existing literature.
|Truth Value of Evidence
The extent to which study findings are trustworthy and believable to others
- Use multiple data sources (data triangulation), methods (methodological triangulation), researchers (investigator triangulation) and theories (theory triangulation).
- Choose an appropriate time period for collection of information to sufficiently address research questions.
- Ask for feedback from participants on the data or interpretation of the data where relevant (member checking).
- Discuss findings relative to existing literature.
The extent to which results can be generalized from the research sample to the population
- Use random or stratified sampling (population generalizability).
- Enable replication of the study in other contexts (ecological generalizability).
- Verify predicted relationships between dependent and independent variables (construct validation).
- Minimize the use of invasive techniques.
|Applicability of Evidence
The extent to which findings can be transferred or applied in different settings
- Make the findings meaningful to others by describing them and their context in detail (thick description).
- Explain the sampling strategy (e.g. typical case sampling or maximum-variation sampling or justify convenience sampling).
- Prioritize primary data.
The extent to which results are consistent if the study would be replicated
- Estimate the internal consistency across repeated measures (classical test theory).
- Estimate sources of variance affecting the measurement (generalizability theory).
- Estimate item, test, and person parameters (item response theory).
|Consistency of Evidence
The extent to which findings are consistent in relation to the contexts in which they were generated
- Collect and analyze data until no new themes emerge (saturation).
- Use pilot data to inform further data collection (iterative data collection).
- Continuously re-examine the data using insights that emerge during analysis as appropriate to the method, but do not manipulate data (iterative data analysis; flexible/ emergent research design)
The extent to which personal biases are removed and value-free information is gathered
- Use blinded assessors or coders during data-gathering. Anonymize respondent identities.
- Let the facts speak for themselves.
- Maintain and safeguard the original data for accountability to journals and the public.
|Ethical Treatment of Evidence
The extent to which findings are based on the study's participants and settings and not researchers' biases
- Search the data and/or literature for evidence that disconfirms the findings.
- Discuss the research process and/or findings with peers/ experts (peer debriefing).
- Keep a diary to reflect on the process and the
- researcher's role and influence (reflexivity).
- Document the steps and decisions taken in the research.
Please also consult the series statement of ethical practices, its language policy, and the WAC Clearinghouse peer review process.
Suggestions for further reading:
- Levitt, H. M. (2019). Reporting qualitative research in psychology. APA.
- Cooper, H. (2019). Reporting quantitative research in psychology. APA.
This overview of quality criteria for the International Exchanges on the Study of Writing book series uses a similar layout and is informed by the article, "AM Last Page: Quality Criteria in Qualitative and Quantitative Research" by Janneke M. Frambach, Cees P. M. van der Vleuten, and Steven J.Durning, which was published in Academic Medicine (volume 88, issue 4, page 552) in April 2013. With the first author’s permission, we have adapted the table for research in writing studies. We also acknowledge revision comments for this document offered by Rebecca Babcock (https://www.utpb.edu/directory/faculty-staff/babcock_r) and Ruth Villalón (https://orcid.org/0000-0003-1600-8026).