Criteria for Quality in Quantitative and Qualitative Writing Research

Internal validity

The extent to which observed effects can be attributed to the independent variable

• Determine your sample size relative to your research objective.
• Describe details of the educational context and intervention.
• Minimize participant attrition and provide information on non-responses and subject withdrawals.
• Standardize experimental conditions.
• Use control groups whenever possible and ethically sound.
• Discuss findings relative to existing literature.

Credibility

The extent to which study findings are trustworthy and believable to others

• Use multiple data sources (data triangulation), methods (methodological triangulation), researchers (investigator triangulation) and theories (theory triangulation).
• Choose an appropriate time period for collection of information to sufficiently address research questions.
• Ask for feedback from participants on the data or interpretation of the data where relevant (member checking).
• Discuss findings relative to existing literature.

External validity

The extent to which results can be generalized from the research sample to the population

• Use random or stratified sampling (population generalizability).
• Enable replication of the study in other contexts (ecological generalizability).
• Verify predicted relationships between dependent and independent variables (construct validation).
• Minimize the use of invasive techniques.

Transferability

The extent to which findings can be transferred or applied in different settings

• Make the findings meaningful to others by describing them and their context in detail (thick description).
• Explain the sampling strategy (e.g. typical case sampling or maximum-variation sampling or justify convenience sampling).
• Prioritize primary data.

Reliability

The extent to which results are consistent if the study would be replicated

• Estimate the internal consistency across repeated measures (classical test theory).
• Estimate sources of variance affecting the measurement (generalizability theory).
• Estimate item, test, and person parameters (item response theory).

Dependability

The extent to which findings are consistent in relation to the contexts in which they were generated

• Collect and analyze data until no new themes emerge (saturation).
• Use pilot data to inform further data collection (iterative data collection).
• Continuously re-examine the data using insights that emerge during analysis as appropriate to the method, but do not manipulate data (iterative data analysis; flexible/ emergent research design)

Objectivity

The extent to which personal biases are removed and value-free information is gathered

• Use blinded assessors or coders during data-gathering. Anonymize respondent identities.
• Let the facts speak for themselves.
• Maintain and safeguard the original data for accountability to journals and the public.

Confirmability

The extent to which findings are based on the study's participants and settings and not researchers' biases

• Search the data and/or literature for evidence that disconfirms the findings.
• Discuss the research process and/or findings with peers/ experts (peer debriefing).
• Keep a diary to reflect on the process and the
• researcher's role and influence (reflexivity).
• Document the steps and decisions taken in the research.