![ORCID Icon]({"type":"image","src":"/templates/jsp/_style2/_tandf/images/orcid.svg"} "ORCID Icon"){kind=small}
https://orcid.org/0000-0003-4070-7513
ABSTRACT
ABSTRACT
The underdevelopment of a pedagogical culture for research methods education and the lack of a body of knowledge with the potential to influence practice have been highlighted by previous studies. This systematic review explores the pedagogic approaches and strategies evident in recent literature (2014–2020) on teaching social science research methods in higher education. It synthesises 55 papers offering a detailed rationale for the approach and strategies employed in doctoral/post-doctoral education. While dispersed across journals, there is a plethora of case studies and reflective accounts about teaching approach, strategy, tactics and tasks in research methods education. Most studies reviewed report on teaching qualitative methods and represent authors’ own teaching practices. Consistent with previous studies, experiential, active learning and student-centred approaches are predominantly discussed, often overlapping or combined with other approaches. This paper illustrates a growing pedagogic culture, represented by an increased volume of papers and theoretical discussion of practices, rationale and reflection on how research methods are taught and learnt. It is concluded that clear intention to engage in dialogue and contribute to evidence-based practice and knowledge in research methods education is evident, and that the ‘how to’ element is richly articulated and justified.
1. Introduction
The teaching of social research methods has been problematised by authors who have noted the inherent challenges (Andrzejewski and Baggett, 2020; Daniel, 2018; Nind and Lewthwaite, 2018), the policy concern with capacity-building (BIS, 2014; Kottmann, 2011), and the lack of supporting pedagogical culture (Earley, 2014; Wagner et al., 2011). One response has been to take stock of the pedagogical literature – to synthesise what is and is not being said about research methods teaching. Indeed, it was the systematic reviews of Wagner et al. (2011) and Earley (2014) that put the underdevelopment of pedagogical culture for research methods education on the agenda for education studies. This referenced a failure of the community to (i) debate, investigate and evaluate how research methods are taught and learnt, and (ii) when writing about their (or others’) pedagogy to connect to other literature and build a body of knowledge. In 2014, following the concern of Earley (2014, p. 2) that methods teachers were left to ‘“rely on a network of peers, scattered research literature, and much trial-and error” for developing their practice’, Kilburn et al. (2014, p. 192) drew a more optimistic conclusion from thematic qualitative exploration of 24 papers published 2007–2014. This evidenced that ‘considerable attention is being paid to the ways in which teaching and learning is structured, delivered and facilitated’, with methods teachers ‘developing conceptually or theoretically useful frames of reference when reflecting on their teaching practice’ (Kilburn et al., 2014, p. 204).
In this paper, we show and reflect on the state of pedagogical practice and inquiry evident from a systematic review of the literature 2014–2020. The context of this period in the UK is government/research council initiatives stimulating activity and interest, including the Q-Step initiative focused on the quantitative methods skills development pipeline, the National Centre for Research Methods provision of training, the Advanced Training Initiative, and Scotland’s Applied Quantitative Methods Network. Worldwide strengthening research capacity is a concern, particularly health and social research in low and middle-income countries where this is a prerequisite for meeting development goals (Essence, 2014). Efforts may be at individual, organisational, or national levels, often embedded in other programmes and focussing on research environment, making evaluation of any pedagogic component challenging. Previous syntheses of the field (e.g. Kilburn et al., 2014) have indicated a focus on student-centred, active and experiential learning approaches in research methods education. Driving the field forward, Nind and Lewthwaite (2019) prompted methods teachers and pedagogic researchers to focus not just on teaching approaches, but also on strategies, tactics and tasks. This involves connecting the approach (in which teaching practices cohere around principles or theory), the strategy for implementing that approach, the tactics in translating it into action, and at the procedural or classroom level, the tasks that teachers and students undertake. We apply these concepts when synthesising the literature in our systematic review, which is aimed at understanding the pedagogic approaches and strategies employed in the teaching of social science research methods discussed in the literature 2014–2020. This includes the rationale for the selection of approaches and development of strategies, including what teachers and students value about them. We consider evidence of their effectiveness where included and any light shed on the accompanying pedagogic tactics and tasks.
2. Methods
Identification and Selection of Studies
A systematic literature search was conducted in April−June 2020 and updated December 2020 to identify final papers published that year. The search strategy involved searching the databases Scopus; PsycINFO; Social Science Premium Collection/Education Collection/Sociological Abstracts; ERIC; Social Sciences Citation Index; Academic Search Index; and Science Direct. Previous reviews indicated that these would be fruitful options, though Wagner et al. (2011) and Earley (2014) did not publish their search strategies in sufficient detail for replication.
Search terms used for searching the bibliographic databases included the following sets in combination:
Terms to indicate that the paper is about research methods: (‘research method*’ OR methodology OR qualitative OR quantitative OR ‘mixed methods’)
AND
Terms to indicate that the paper will be about education and training and thereby potentially about pedagogical approaches or strategies: (teach* OR learn* OR train*)
AND
Terms or filters to limit the papers to being about social science.
The parameters of the searches were set to search titles only and to select only papers published in the English language, in journals, during the period January 2014-December 2020.
Studies were identified that met all of the following inclusion criteria:
Scope
Focuses on teaching (training) in social science research methods.
Provides description and/or rationale for the pedagogical approaches and/or strategies.
Concerns adults learning research methods as part of their degree, professional training or development as researchers.
Paper Type
Journal article report or discussion of individual empirical studies or close to practice research or synthesis/review of these.
Written in English
Timespan
Published 2014–2020
Studies were excluded if they did not meet any one of these criteria.
We screened papers in three stages proceeding through a series of graduated filters: First, we identified all studies with potential to meet the inclusion criteria, creating a database of all the studies retrieved from the electronic databases. Second, we scrutinised titles, and where necessary the abstracts applying the inclusion and exclusion criteria to exclude papers that were clearly not about teaching research methods. Third, we recorded all studies apparently meeting the inclusion criteria in a second database and applied the inclusion screening again to full papers.
The primary researcher (Nind) conducted the bulk of the search with a sample of the databases also searched by a second researcher (Katramadou) to provide a point of comparison, highlight potential skewing of the search and double-check application of the search strategy and inclusion/exclusion criteria. The main issue to resolve was whether to include papers from health sciences/nursing and we assessed these together on a case-by-case basis to see if they pertained to social research.
Data Extraction and Analysis
We keyworded the papers iteratively, each taking a tranche of papers in turn and highlighting papers that required discussion or issues to resolve. A deliberate overlap of 10% of papers allowed checking for consensus. Mapping the teaching approaches and the strategies was complex in that the papers varied in how explicit these were and in the terminology used to describe teaching practice. We were both involved in this key element and double-checked each other. The keywords to identify teaching approaches were drawn from previous reviews of the literature to aid comparison across time and studies; these were student-centred, active learning, experiential, problem-based, visual, verbal, standpoint-led, collaborative, varied, other; for those keyworded as other the authors’ nomenclature was applied.
Keywording served two core functions: (i) to build a descriptive map of the published literature, and (ii) to identify studies for in-depth review and synthesis. The first involved providing a comprehensive picture of the kinds of research conducted, in which countries and disciplines, whether in face-to-face, online or blended mode, in classroom/lab, supervision or apprenticeship context, and teaching what kind of research method to what level of learners, using what approaches. The second involved ascertaining papers with a detailed rationale for the pedagogical approaches and strategies, highlighting tactics and tasks, and noting whether learner perspectives and efficacy data were included. Assessing the quality of the studies was not part of this stage. Ultimately, in selecting papers for in-depth synthesis we included only those about teaching methods at doctoral/post-doctoral/researcher level, based purely on what was most relevant to interests of the research funder (ESRC National Centre for Research Methods).
Quality Assessment
We began a process of subjecting papers to quality assessment in order to exclude low-quality papers from the synthesis. As this was not a ‘what works’ systematic review, and as the majority of the papers were reporting on the authors’ own pedagogic practice, the quality criteria were adapted from those used by Wyse et al. (2020) in the rapid evidence review of close-to-practice research. These related to explicitness and appropriateness of research design, clear theoretical framing, focus on issues important to teachers, and contribution to knowledge. However, it soon became apparent that some level of theorising was inherent to the inclusion criterion of having a rationale for the pedagogical approaches/strategies. Moreover, as these were almost exclusively papers written by methods teachers, they all focussed on issues important to teachers. The discriminating factor therefore would have needed to be explicit and appropriate research design, but as many of the papers were reflective monologues this criterion could not be consistently applied. Ultimately, we opted to include all the papers, confident that none was too weak to be there.
3. Findings
We identified 314 papers and after the removal duplicates we retrieved 209 for mapping (see Figure 1). Most of the teaching discussed was in the USA (113 papers), with 31 papers pertaining to the UK, 15 Canada, 7 Australia, 4 New Zealand, a further 13 across European countries, 11 across Asia and 4 across Africa. In nine papers the countries were varied or unknown.
Published online:
05 July 2022Figure 1. Flow chart showing PRISMA process for identification of papers (Moher et al., 2009).
![]({"type":"image","src":"/na101/home/literatum/publisher/tandf/journals/content/rbje20/0/rbje20.ahead-of-print/00071005.2022.2092066/20220705/images/medium/rbje_a_2092066_f0001_oc.jpg"})
Figure 1. Flow chart showing PRISMA process for identification of papers (Moher et al., 2009).
Characteristics of the Included Studies
There were three times as many papers on the teaching of qualitative methods (96) as the teaching of quantitative methods (33), with 22 papers on the teaching of mixed methods and the remaining papers addressing general or unspecified methods. In terms of learners, 94 papers addressed teaching undergraduates, 60 primarily postgraduates, 55 doctoral and postdoctoral students, with the remainder unspecified, mixed or other groups such as community-based researchers. Twenty-seven papers addressed teaching methods online; these included teaching qualitative, quantitative and mixed methods and at all levels. In terms of discipline, education (29), psychology (27), sociology (19), politics (18) and social work (13) dominated as the fields of study, with a further 23 papers across social sciences and 23 papers across health sciences, with the remainder addressing varied or unspecified groups.
There were 55 papers providing a detailed rationale for the approach and clear exposition on strategies (see ); the majority of these (36) relate to teaching qualitative methods. The other most striking pattern was that most of the papers report on the authors’ own teaching practices (49) rather than pedagogic research beyond their own contexts. Experiential and active learning approaches were abundantly represented, often combined with each other or with student-centred learning, reflective learning, problem-based learning or collaborative learning. We turn now to a synthesis of the literature in relation to the research question: What pedagogic approaches and strategies are employed in the teaching of social science research methods and how does the literature illuminate the accompanying pedagogic tactics and tasks?
TABLE 1. Papers included in the synthesis
Synthesis
Active learning: As expected, many papers discuss active learning whether or not they use that term. One of the attractions of an active learning approach is the alignment between this and other valued approaches, notably experiential learning (Bourque and Bourdon, 2017; Corti and Van den Eynden, 2015; Hazzan and Nutov, 2014; Hsiung, 2016; King, 2018; Mallette and Saldaña, 2019; Pfadenhauer et al., 2018), collaborative (Boström, 2019; Bourque and Bourdon, 2017; Hesse-Biber, 2015; Mallette and Saldaña, 2019), non-linear (Hsiung, 2016), problem-based (Bowers, 2017; Dyrhauge, 2014), student-centred (Dyrhauge, 2014) and reflective learning approaches (Corti and Van den Eynden, 2015; Orange, 2016). Moreover, active learning is valued as a good fit for teaching core concepts, for example, by using case studies, practical activities, exercises and group discussions (Corti and Van den Eynden, 2015) and fun simulations/gaming (Mallette and Saldaña, 2019).
Essentially, the hands-on work of active learning is seen as consolidating and expanding knowledge and its application in practice, such as in learning about interviewing (Hsiung, 2016) or when learning to apply mixed methods (Hesse-Biber, 2015). Through active learning students are seen to become active agents in identifying methodological problems and revising and justifying methods (Hsiung, 2016). They are supported to build on their experiences and knowledge (Ivankova and Plano Clark, 2018), for example by applying analytical thinking, sharing decision-making (Bowers, 2017), or via peer scaffolding (Saeed and Al Qunayeer, 2021 [published online in 2020]).
Papers focus on the suitability of active learning for the subject matter, whether this be qualitative research methods (Bourque and Bourdon, 2017; Hazzan and Nutov, 2014; Mallette and Saldaña, 2019), mixed methods (Hesse-Biber, 2015; Ivankova and Plano Clark, 2018) or quantitative methods (Bowers, 2017). For some authors, the theoretical influences are important, with connections made to critical pedagogy (Hsiung, 2016), constructivism (Bourque and Bourdon, 2017; Dyrhauge, 2014; Hazzan and Nutov, 2014; Beres and Woloshyn, 2017; King, 2018), cognitivism (Bourque and Bourdon, 2017), and sociocultural theories of learning (Bourque and Bourdon, 2017; Saeed and Al Qunayeer, 2021).
Favouring an active learning approach is also based on experience, in that prior teaching of research methods had taught methods teachers the importance of what they might call learning by doing or hands-on practice (Bourque and Bourdon, 2017; Danquah, 2017). Here we see the mix of doing and reflecting (Orange, 2016; Tan and Hew, 2016), particularly for learning complex subject matter as in qualitative research (Hazzan and Nutov, 2014). There is consensus that students learn through conducting research (Hazzan and Nutov, 2014; Pfadenhauer et al., 2018) and doing analysis processes (Mallette and Saldaña, 2019), thereby taking risks, applying and reflecting on learning (Hesse-Biber, 2015).
Active approaches to building research competence is valued whether teaching in person, or creating teaching toolkits (see Corti and Van den Eynden, 2015; Hsiung, 2016; Ivankova and Plano Clark, 2018), or in blended learning where students can participate and be active at their own rhythm and pace (Bourque and Bourdon, 2017; Tan and Hew, 2016).
Learning to use data analysis software seems to demand that students work in active ways, often on their own projects and data, and teachers value this for enabling learners to appreciate that it is the researcher who is in control of the analysis (Bourque and Bourdon, 2017) and of their learning about posing and addressing data-informed questions (Bowers, 2017). Active learning is chosen to prepare students for their roles in conducting research (Ivankova and Plano Clark, 2018; Pfadenhauer et al., 2018) and for finding the story in the data (Mallette and Saldaña, 2019). It is chosen to enhance the fun and accessibility of learning when the concepts are hard (Danquah, 2017; Mallette and Saldaña, 2019) and when creativity, reflection (Boström, 2019), autonomy and self-determination (Tan and Hew, 2016) are needed.
The papers tell something about what learners value in active learning, but this learner data is largely filtered through the teacher lens. Reference is made to valuing gaining experience and confidence (Bourque and Bourdon, 2017) and to learning by doing (in workshops) being helpful, supportive, engaging and interactive (Danquah, 2017). The valuing of elements of good preparation combined with fun (Ivankova and Plano Clark, 2018) parallel the teachers’ perspectives.
In operationalising active learning, the strategy of engaging students in ‘hands on’ work to form the building blocks of learning is used when teaching data management (Corti and Van den Eynden, 2015) and data analysis (Bourque and Bourdon, 2017). Teachers have strategies for interspersing being ‘hands on’ with ‘hands off’ reading, thinking and exchanging ideas (Bourque and Bourdon, 2017), or for making space for active learning by using flipped classroom curriculum design (Earley, 2016).
Moving to the more practical tactics and tasks of active learning pedagogy, teachers use research journal clubs (Bowers, 2017), playful tasks (King, 2018), interim reporting (Hazzan and Nutov, 2014), learning logs with reflexive exercises (Hesse-Biber, 2015) and other ways for students to share their research projects (Dyrhauge, 2014). The tactic is often to provide carefully staged practical tasks, exercises, discussions, role plays and group work (Boström, 2019; Bowers, 2017; Corti and Van den Eynden, 2015; Danquah, 2017; Pfadenhauer et al., 2018), games and simulations (Boström, 2019; Mallette and Saldaña, 2019; Tan and Hew, 2016) interspersed with lectures. Exercises might be combined in linear series progressing through the stages of a research project (Boström, 2019; Bourque and Bourdon, 2017) or made relevant though their connection to the learners’ own organizations or plans (Bowers, 2017).
To summarise, the pedagogic literature in this area includes very little efficacy data but plentiful sound reasoning and passionate commitment to active learning.
Experiential Learning Approaches
We have noted above the overlap between active and experiential approaches in the literature. We also found experiential approaches integrated with arts-informed (Bogumil et al., 2017; Chatfield et al., 2014; Lapum and Hume, 2015), problem-based (CohenMiller et al., 2020), project-based (Yang et al., 2020) and service learning (Bryant et al., 2017).
Experiential learning approaches feature in contrast to more textbook or procedural pedagogies, with teachers favouring creative engagement with the craft of research (Chen, 2016) over transmission of knowledge. The rationale is about the need for a more holistic approach, drawing in the learners’ personal reflections and emotions (Bartels and Wagenaar, 2018; Bogumil et al., 2017; Call-Cummings et al., 2019; Chatfield et al., 2014) as well as their judgment and reasoning (Lapum and Hume, 2015), such that ‘experience is translated through reflection into concepts, which becomes guides for active experimentation and the generations of new experiences’ (Bartels and Wagenaar, 2018, p. 195).
Where they discuss theory, the papers on experiential learning draw on Kolb’s (1984) experiential learning theory (Bartels and Wagenaar, 2018; Johnson et al., 2019; King, 2018; Lapum and Hume, 2015). They draw also on Schön (1983), practice theory (e.g. Nicolini, 2012; Wagenaar and Cook, 2011), open space learning (Monk et al., 2011), situated learning (Lave and &Wenger, 1991) and project-based learning (Dewey, 1916/1966). These theories rely on Western concepts, however, and Chen (2016, p. 73) adapts Kolb into a ‘pedagogy of unity of knowing and doing’ for the Chinese context. Some authors (e.g. CohenMiller et al., 2020) who advocate grounding learning in real-world problems do so based on intuition and experience more than theory.
The value of experiential learning is seen as the reciprocity between teaching and learning and a coming together of teachers and learners, theory and practice, in addressing the challenges of doing real-world research (Bartels and Wagenaar, 2018; Bogumil et al., 2017; CohenMiller et al., 2020), solving problems in real-world contexts and professional practice (Yang et al., 2020).
Experiential learning is valued particularly by teachers of qualitative research (and not just particular methods within this). It is argued to be suited to learning to be a researcher (Call-Cummings et al., 2019), particularly a qualitative (or mixed methods) researcher, which may be ‘a transformative process’ (Johnson et al., 2019; Lapum and Hume, 2015). Aesthetic as well as dialogic experiences are seen as useful for facilitating cognitive understanding (Chatfield et al., 2014; Lapum and Hume, 2015) and students being directly involved with research phenomena is regarded as critical (Patka et al., 2017). Experiential learning approaches fit the professional domains of some research learners, such as education leaders (Miskovic and Lyutykh, 2017), social workers (Natland et al., 2016; Yang et al., 2020), health professionals (Bryant et al., 2017), and counsellors (Patka et al., 2017) where constructivist paradigms are held. Less common, teachers of quantitative methods appreciate experiential learning through simulations, which immerse students in methodological concepts and tools and bring depth to students’ understanding of the processes they use (Carsey and Harden, 2015).
There is evidence that students can feel vulnerable and unsettled as they work through their experiences and periods of uncertainty in experiential learning (Bartels and Wagenaar, 2018; Bogumil et al., 2017; CohenMiller et al., 2020; Miskovic and Lyutykh, 2017). This is in part why experiential learning is also designed to be collaborative (CohenMiller et al., 2020) and the outcome of reduced stress as the learning progresses is reported (Chatfield et al., 2014). When efficacy data are reported, these indicate deep learning, including appreciation of the process of research and of the ethics of doing research (Chatfield et al., 2014).
Operationalising experiential learning approaches involves strategic placement of students into authentic research projects (Bryant et al., 2017; Natland et al., 2016) or strategic adjustment of the classroom environment in preparation for dialogue to collaboratively make sense of their experiences (rearranging the desks into a circle was the students’ first task for CohenMiller et al., 2020). Papers discuss strategies of practical reasoning (Chen, 2016) and of student groups undertaking research, negotiating the decision-making involved (CohenMiller et al., 2020; Johnson et al., 2019) and revising and building on previous assignments (Miskovic and Lyutykh, 2017). Strategic use is made of drama (Chen, 2016), other arts media including poetry, dance, film and story to give students experiences with data (Lapum and Hume, 2015), and photographs/photovoice (Patka et al., 2017) to involve students in the role of researcher, participant, learner and critical thinker. Unusually, Call-Cummings et al. (2019) and Miskovic and Lyutykh (2017) present their tactics and tasks for facilitating experiential learning in online teaching.
Student-centred Approaches
As with active and experiential learning, the rationale for adopting a student-centred pedagogic approach often reflects the good alignment with other approaches such as taking a standpoint approach (Andrzejewski and Baggett, 2020; Bogumil et al., 2017; Franco, 2016; Hesse-Biber, 2015), adopting collaborative learning (Simon, 2014), active learning (Hesse-Biber, 2015; Simon, 2014), or using scaffolding (Slayton and Samkian, 2017). Student-centredness combines with standpoint-led and experiential approaches when students who are learning research methods are prompted to contribute their own experiences, knowledge, and expertise, and get actively involved in their own development in critical thinking (Andrzejewski and Baggett, 2020; Bogumil et al., 2017). This can involve ensuring the contact is personally relevant (Bell, 2016).
Methods teachers value students’ prior learning and knowledge and seek to support iterative application of this as students’ understanding deepens (Hesse-Biber, 2015). They respect the practice-based knowledge of practitioner-scholars (Slayton and Samkian, 2017). Moreover, they are likely to hold a social constructionist belief that students create their own knowledge through inquiry and collaboration with peers (Franco, 2016), and they draw on sociocultural and constructivist learning theories that foreground supporting students to work both independently and collaboratively to construct their knowledge (Slayton and Samkian, 2017).
Teachers’ values underpin their student-centred pedagogic approaches and Andrzejewski and Baggett (2020, p. 866) write of a commitment to teaching for social justice and the positioning of ‘education as liberatory and classroom spaces as emancipatory’. More widely, student-centred methods teachers trust in the idea that students will expand their knowledge and skills best if they can connect the learning with their personal research interests (Franco, 2016) and their own personal philosophical stance (Andrzejewski and Baggett, 2020; Franco, 2016).
Teachers choose student-centred learning for its efficiency (Simon, 2014) as ownership of learning and projects brings with it better ability to defend and justify methods, deeper knowledge and stronger enthusiasm (Bell, 2016). Teachers value the cumulative learning that happens when starting from the learners themselves (Hesse-Biber, 2015). Students’ appreciation of a student-led approach though might be gradual after some initial discomfort (Andrzejewski and Baggett, 2020) or resistance to the challenges as Franco (2016) found. Students involved in student-led work report understanding the research process and challenges, and enhancing their insight into the way their biases shape their world view and approaches (Slayton and Samkian, 2017), questioning themselves (Hesse-Biber, 2015).
In terms of strategies, tactics and tasks associated with student-centred approaches, these are mostly applied in face-to-face, classroom-based work, with some online or apprenticeship-based mode. Ross and Call-Cummings (2020) discuss their strategy of the ‘fieldworking’ class or extended/recurring qualitative methods workshop (in person or online) as per student need, with students discussing and trying out fieldwork in the safety of a classroom environment where writing about the messiness is actively promoted and expected. A student-centred approach often involves peers working in groups on topics of interest to them in which they discuss ideas and gain regular peer feedback (Bell, 2016; Hesse-Biber, 2015), sometimes formalised as a ‘Socratic method’ of teaching through questions, group- and pair-work with continuous assessment and feedback (Simon, 2014).
Tactics and tasks include: weekly reflection journals used as a pair/small groupwork tool to highlight failures as experienced in fieldwork as well as thinking ahead scenarios (Ross and Call-Cummings, 2020); logs of experiences in learning about mixing methods and the concerns and issues in understanding the nuts and bolts of mixing methods (Hesse-Biber, 2015); assignments and writing exercises that help to personalise new knowledge (Bell, 2016; Franco, 2016; Hesse-Biber, 2015); personalised text/textbook selection (Andrzejewski and Baggett, 2020; Franco, 2016); and student-led reviews of research papers (Franco, 2016), methodological discussions (Andrzejewski and Baggett, 2020), and generation of observational and reflective data in their own fields (Andrzejewski and Baggett, 2020).
Regarding efficacy, data indicate this depends on students’ investment in exploring their own philosophical stance (Franco, 2016) whereby they co-construct knowledge of research from their reading, self-study, and discussion. The sociocultural context impacts efficacy, for example, Simon (2014) shows that transferring student-centred practices into the non-Western context of Hungary involves problems with satisfaction and learning. Bell’s (2016) natural experiment showed students performed better in a new intensive, more student-centred approach and format when compared to a previous format, but conclusions are limited by the student-centred approach and intensive format not being separable.
Other Approaches
The literature is not limited to these dominant and overlapping approaches. Papers also cover pedagogic approaches designed for postcolonial contexts/decolonizing methods (Kaomea, 2016; Parker et al., 2018) and feminist research (Beckman, 2014), plus approaches designed to fit the qualitative and post-qualitative (and posthuman, poststructural) content of research methods education. Such approaches involve teaching with standpoints or theory as much as with data, and the pedagogies are exploratory and emergent (Guyotte et al., 2020; Kuby et al., 2016; Kuby and Christ, 2019, 2020; Roulston and Shelton, 2015; Wolgemuth, 2016). These offer extreme examples of enacting or ‘living’ the theory in the pedagogy as students do research of a particular kind or be researchers of a particular kind and teachers theorise their practices and reflect on and with their materials.
Rather than remain distant from practice, these more unusual papers powerfully illustrate the interweaving of approach, strategy, tactics and tasks as the authors show the pedagogies (using vignettes, images, poetry and stories) as much as describe them. These emergent pedagogies are often student-centred, active, experiential and collaborative, thus not a complete departure from the approaches covered thus far, but their rationales and theoretical bases are different, which is important. Kuby et al. (2016), for example, refer to disrupting the teaching/learning binary and disrupting both teaching and research methods, and Parker et al. (2018) describe radical re-working of the research classroom.
Other approaches in the literature include the arts-based approach of Lester and Gabriel (2016) involving performance ethnography and influenced theoretically by performance-oriented scholars such as Denzin (2003). There is also Noy’s (2015) reflection on parallel approaches in teaching qualitative research and teaching martial arts, and Mulvihill et al.’s (2015) discussion of strategies to facilitate collaborative reflection when learning to analyse data. Thus, we see divergence as well as convergence in research methods pedagogy.
4. Discussion
Once described as missing (Earley, 2014; Wagner et al., 2011) or emerging (Kilburn et al., 2014), the pedagogical culture around research methods education appears to be flourishing in the form of widespread investigation and evaluation concerning how research methods are taught and learnt, with some (not yet systematic) debate and plentiful reflection. We found a 50% increase in papers, identifying the equivalent of around 30 papers per year in this period compared with Wagner et al.’s (2011), 20 per year. Wagner et al. (2011) regarded the wide dispersal of research methods pedagogic literature across journals as undermining the pedagogic culture. We found such dispersal in the literature pertaining to undergraduate study, where papers were often found in discipline-specific journals such as Teaching Sociology, Qualitative Psychology and Journal of Political Science Education as well as methodology journals. However, in doctoral/postdoctoral education we found that the methodology journals (particularly Qualitative Inquiry and The Qualitative Report) were becoming a natural home for this work, alongside yet dominating the higher education and discipline-based journals.
We did not systematically scrutinise rates of cross-referencing, but we did note readiness to engage with other scholars in this field (see e.g. the acknowledgements of Chen, 2016) and to build on the research methods pedagogy literature (see e.g. Carsey and Harden, 2015; Miskovic and Lyutykh, 2017). Nonetheless, Johnson et al. (2019) note that in mixed methods the pedagogical literature is limited mostly to curriculum design, with literature on practical teaching methods still embryonic (citing the authors in this review). Patka et al. (2017) claim ‘there is limited research on incorporating experiential learning in the teaching of research methods’ and Culkin (2018) makes a more general point about paucity. An image of a lack of pedagogical culture may not have caught up with the changing picture.
The mapping of the literature indicates that while growing, the spread of pedagogic work on research methods is not even: much more is written about the teaching of qualitative methods, with studies from the USA dominating and with most papers focused on the authors’ own teaching practice. The focus of the systematic review question steered us towards papers on teaching more than learning, with papers like Nind et al.’s (2020) research on student perspectives not meeting the inclusion criteria. Nonetheless, while Earley (2014, p. 248) found that ‘very little was presented that gives us a picture of what student learning looks like’, we found some very rich data on this, albeit mostly filtered through the lens of teacher-researchers. Occasionally teacher-researchers wrote papers together with student-researchers (e.g. Bogumil et al., 2017; Call-Cummings et al., 2019; Chatfield et al., 2014; Guyotte et al., 2020).
Data on the efficacy of research methods pedagogy also remains a gap, reinforcing the observed ‘paucity’ recognised by Johnson et al. (2019) among others. Nonetheless, case studies and reflective accounts of research methods pedagogy provide a rich reference point for teachers seeking inspiration or ideas such that they no longer have to rely on trial and error as Earley (2014) argued was the case. Evidence-based practice has to rely primarily on close-to-practice research designs and practitioner inquiry. This is because, while there is not the ‘substantial research base’ (Wagner et al., 2011, p. 85) of research conducted at a distance from practice or by researchers not involved in the teaching, there is a wealth of practitioner reflections. As Wyse et al. (2020, p. 4) discuss, this need not be seen as ‘less “scientific”’. If we are to take seriously the valuing of practitioner inquiry and close-to-practice research, we must acknowledge that by its very nature it ‘focusses on issues defined by practitioners as relevant to their practice’ (BERA, 2018), which might mean privileging a meaningful rationale over efficacy studies, with the former contributing to research capacity building (see Cooke, 2005).
5. Conclusion
This review builds on and updates recent evidence on teaching approaches. Kilburn et al. (2014, p. 198) observed that active learning in research methods education was ‘nigh on a pedagogical orthodoxy’ and that making the learning student-centred was an intuitive response to the challenges; this review reinforces these points. Equally, experiential learning continues to be a favoured approach. Papers from this recent period are sufficiently theorised to contribute to the knowledge base about these approaches in action. This may reflect the difference between this field and much of the research that Wyse et al. (2020) looked at, in that instead of there being a close relationship between the academics and the practitioners, in most of the studies in this review, the academics are the practitioners. While some papers refer to a study and research design and others offer reflective clarification and understanding of practices, all have potential to influence the practice of others.
This review, based on the volume and nature of the literature identified, suggests that the point made by Wagner et al. (2011) a decade ago – that literature on the ‘how to’ aspect of research methods is missing – is no longer true. There is a substantial literature now describing how research methods teachers go about their craft, some of it reinforcing earlier messages about what is important, some of it more innovative, emergent, even playful (e.g. Guyotte et al., 2020). It is significant that most of this literature reports on close-to-practice research, including action research, design-based research, case study and evaluation studies that are valued in the educational community (Wyse et al., 2020). It is people doing this kind of work that the National Centre for Research Methods is seeking to bring together, so that individual projects can remain close to practice, relevant to practitioners and become connected to each other and in dialogue.
The literature discussed in this paper evidences an interested body of methods teachers willing to write about their practices and their rationale for them. Inevitably, we identified papers where the in-classroom tasks, business-end of research methods pedagogy is missing (see ); there were papers that focus more on curriculum design aspects (e.g. Upadhyaya et al., 2015; Beres and Woloshyn, 2017; Woolf, 2014), or the detail of what teachers did (e.g. Johnson et al., 2019) or what the learners did (e.g. Bryant et al., 2017; Mulvihill et al., 2015), which might mean less emphasis the cohering strategy or approach (e.g. Miskovic and Lyutykh, 2017; Patka et al., 2017; Roulston and Shelton, 2015). Even so, in terms of the ‘how to’ element, much of the literature identified weaves together a story about approach, strategy, tactics and tasks as particularly well-illustrated by Chen (2016) and Lapum and Hume (2015).
As we move into a period of reflection on what has happened to our research methods and our teaching practices in the face of the Covid-19 pandemic, it will be fascinating to see how the research methods pedagogy literature develops and in what ways active, experiential, student-centred pedagogies survive the pivot to online teaching. The early signs are that there has been a growth of efficacy studies about teaching online; these will make a worthwhile focus for a future specific new review.
| study | study type | country | disciplinary context | level/participants | mode | approach | tactics & tasks | learner perspectives | effectiveness | |
|---|---|---|---|---|---|---|---|---|---|---|
| Teaching qualitative methods | ||||||||||
| Andrzejewski & Baggett (2020) | close-to-practice | USA | education | doctoral students | face-to-face | student-centred experiential reflective learning | ✓ | |||
| Bartels & Wagenaar (2018) | close-to-practice | Slovenia | unspecified | doctoral students | face-to-face apprenticeship | experiential other: dialogic | ✓ | ✓ | ✓ | |
| Bogumil et al (2017) | close-to-practice | Canada | sociology | postgraduate students | face-to-face classroom | other: arts-based | ✓ | ✓ | ✓ | |
| Boström (2019) | close-to-practice | Sweden | psychology | undergraduate/ postgraduate/ doctoral students | face-to-face classroom | active learning other: reflective learning | ✓ | |||
| Bourque & Bourdon (2017) | close-to-practice | Canada | multi-disciplinary | postgraduate/ doctoral/ postdoctoral students | blended classroom | active learning other: hands-on/hands-off | ✓ | ✓ | ✓ | |
| Bryant et al (2017) | close-to-practice | USA | nursing | doctoral students | face-to-face classroom | Experiential service learning | ✓ | ✓ | ✓ | |
| Call-Cummings et al (2019) | close-to-practice | USA | education | doctoral students/ professional educators | face-to-face apprenticeship | student-centred experiential other: reflective learning | ✓ | ✓ | ✓ | |
| Chatfield et al (2014) | research | USA | conflict analysis and resolution | doctoral students | face-to-face classroom | experiential other: arts-based | ✓ | ✓ | ✓ | |
| CohenMiller et al (2020) | close-to-practice | Kazakhstan | education | postgraduate students | unspecified classroom | experiential other: creative | ✓ | |||
| Culkin (2018) | close-to-practice | USA | education | faculty members | unspecified supervision | experiential | ✓ | |||
| Danquah (2017) | close-to-practice | UK | psychology | doctoral students | face-to-face classroom | active learning | ✓ | ✓ | ✓ | |
| Dyrhauge (2014) | close-to-practice | Denmark | european studies | undergraduate/postgraduate students | face-to-face classroom | active learning problem-based | ✓ | ✓ | ✓ | |
| Earley (2016) | close-to-practice | USA | unspecified | posgraduate/ doctoral students | blended classroom | active learning other: flipped learning | ✓ | ✓ | ✓ | |
| Franco (2016) | close-to-practice case study | USA | education (mostly) | doctoral students | face-to-face classroom | student-centred constructivist | ✓ | ✓ | ✓ | |
| Guyotte et al (2020) | close-to-practice | USA | unspecified | doctoral students | online | other: meanwhile pedagogy, intra-active | ✓ | ✓ | ✓ | |
| Hazzan & Nutov (2014) | close-to-practice | Israel | education in science and technology | postgraduate/ doctoral students | face-to-face classroom | experiential active learning | ✓ | ✓ | ||
| Hsiung (2016) | close-to-practice | Canada | unspecified | undergraduate/ postgraduate students/ novice researchers | online classroom | active learning experiential other: non-linear | ✓ | |||
| Kaomea (2016) | close-to-practice | Hawaii | unspecified | graduate students | face-to-face apprenticeship | collaborative | ✓ | ✓ | ||
| Kuby & Christ (2019) | close-to-practice | USA | unspecified | doctoral students | face-to-face classroom | other: intra-active, thinking with theory | ✓ | |||
| Kuby & Christ (2020) | close-to-practice | USA | unspecified | doctoral students | face-to-face classroom | other: intra-active, theory as practice | ✓ | ✓ | ||
| Kuby et al (2016) | close-to-practice | USA | varied | doctoral students | face-to-face classroom supervision | other: disruptive, transgressive | ✓ | ✓ | ||
| Lapum & Hume (2015) | close-to-practice | Canada | unspecified | graduate students | face-to-face classroom | experiential other: arts-informed | ✓ | |||
| Lester & Gabriel (2016) | close-to-practice | USA | education | postgraduate students | face-to-face classroom | other: arts-based | ✓ | ✓ | ✓ | |
| Mallette & Saldana (2019) | close-to-practice | USA | unspecified | doctoral students | face-to-face classroom | active learning Other: simulation Other: gaming | ✓ | ✓ | ||
| Miskovic & Lyutykh (2017) | close-to-practice | USA | education | doctoral students/ practitioner-scholars | online classroom | experiential other: qualitative research-based | ✓ | |||
| Mulvihill et al (2015) | close-to-practice | USA | education | doctoral students | face-to-face classroom | collaborative | ✓ | |||
| Noy (2015) | close-to-practice auto-ethnography | USA | not stated | undergraduate, postgraduate, doctoral students | face-to-face classroom | other - martial arts-based | ✓ | ✓ | ||
| Orange (2016) | close-to-practice | USA | education | doctoral students | face-to-face supervision | Active learning other: reflective learning | ✓ | ✓ | ✓ | |
| Patka et al (2017) | close-to-practice | USA | counseling/ psychology | postgraduate/ doctoral students/ professional counsellors | face-to-face classroom | experiential | ✓ | ✓ | ✓ | |
| Pfadenhauer et al (2018) | close-to-practice | Germany | public health | postgraduate/ doctoral students | face-to-face classroom | active learning | ✓ | ✓ | ✓ | |
| ROSS & Call-Cummings (2020) | close-to-practice | USA | unspecified | postgraduate/ doctoral students | unspecified classroom apprenticeship | student-centred standpoint-led | ✓ | |||
| Roulston & Shelton (2015) | close-to-practice | USA | unspecified | doctoral students | face-to-face classroom | standpoint-led | ✓ | ✓ | ||
| Simon (2014) (also quant) | close-to-practice | Hungary | political science | postgraduate students | face-to-face classroom | student-centred active learning | ✓ | ✓ | ✓ | |
| Slayton & Samkian (2017) | close-to-practice | USA | education | doctoral students/ practitioner-scholars | face-to-face classroom | student-centred active learning other: scaffolding | ✓ | |||
| Wolgemuth (2016) | close-to-practice | USA | education | doctoral students | face-to-face classroom | other: paradigm driven, disruptive | ✓ | ✓ | ✓ | |
| Yang et al (2020 | close-to-practice | USA/China | social work | doctoral students | face-to-face classroom | project-based experiential | ✓ | ✓ | ✓ | |
| Teaching quantitative methods | ||||||||||
| Bowers (2017) | close-to-practice | education | doctoral students/ practitioner-scholars | face-to-face classroom | active learning problem-based learning | ✓ | ||||
| Carsey & HARDEN (2015) | close-to-practice | USA | political science | postgraduate students | face-to-face classroom | experiential | ✓ | ✓ | ✓ | |
| Chen (2016) | close-to-practice | China | education | postgraduate students | unspecified classroom | experiential active learning | ✓ | ✓ | ✓ | |
| Simon (2014) (also qual) | close-to-practice | Hungary | political science | postgraduate students | face-to-face classroom | student-centred active learning | ✓ | ✓ | ✓ | |
| Teaching mixed methods | ||||||||||
| Beres & Woloshyn (2017) | close-to-practice | Canada | unspecified | postgraduate students | face-to-face classroom | collaborative experiential varied | ✓ | |||
| Hesse-Biber (2015) | close-to-practice | USA | varied/ healthcare | graduate students | face-to-face classroom | active learning student-centred, standpoint-led other: team-based | ✓ | ✓ | ||
| Ivankova & Plano Clark (2018) | close-to-practice | USA | doctoral students | face-to-face/online classroom | active learning | ✓ | ✓ | ✓ | ||
| Johnson et al (2019) | close-to-practice | UK | health sciences | postgraduate, doctoral, health professionals | face-to-face classroom | experiential active learning other: open space learning | ✓ | ✓ | ✓ | |
| Upadhyaya et al (2015) | close-to-practice | USA | public health | doctoral students/ professionals | face-to-face classroom/ apprenticeship | varied | ✓ | ✓ | ✓ | |
| Teaching unspecified/varied/other methods | ||||||||||
| Beckman (2014) | research | USA | clinical psychology | graduate/ doctoral /postdoctoral students | unspecified classroom | varied | ||||
| Bell (2016) | research | UK | business | doctoral students | face-to-face classroom | student-centred | ✓ | ✓ | ✓ | |
| Corti & Van den Eynden (2015) (data management) | close-to-practice | UK | undergraduate/ postgraduate/ doctoral students/ researchers | face-to-face classroom | active learning experiential reflective learning | ✓ | ||||
| Kilburn et al (2014) | review | N/A | unspecified | undergraduate/ postgraduate/ doctoral students/ researchers | varied | varied student-centred active learning experiential other: reflective learning | ✓ | |||
| King (2018) | close-to-practice | UK | not stated | postgraduate students | face-to-face classroom | student-centred experiential | ✓ | ✓ | ✓ | |
| Natland et al (2016) | research | USA | social work | postgraduate students/ researchers | face-to-face supervision | experiential | ✓ | ✓ | ||
| Parker et al. (2018) | close-to-practice | USA | interdisciplinary | postgraduate/ doctoral students/ professionals | unspecified classroom | varied | ✓ | ✓ | ||
| Saeed & Al Qunayeer (2021) | close-to-practice | Malaysia | languages and linguistics | postgraduate students | blended classroom | active learning | ✓ | ✓ | ✓ | |
| Tan & Hew (2016) | research | Asia | unspecified | postgraduate students | blended classroom | active learning | ✓ | ✓ | ✓ | |
| Woolf (2014 | close-to-practice | Canada | sport | postgraduate students | face-to-face classroom | problem-based | ✓ | ✓ | ✓ | |
6. Disclosure statement
No potential conflict of interest was reported by the author(s).