The selection of informants in kindergarten research in Norway: A critical review

Predetermined criteria

The selection of criteria for inclusion and exclusion in this study emerged from the research question and began with five main criteria for coverage. The first criterion is that the research was carried out in Norwegian kindergartens and produced PhD theses or peer-reviewed articles. Each publication was counted. Article-based PhD theses were counted as consisting of journal articles, where each article is an individual scientific contribution, while monographs were each counted as one publication. The second criterion for inclusion was that publications must use kindergarten staff as informants. The third criterion was that the publications were published between 2008 and 2017, since the research topic only concerns studies after the initiation of the MAFAL Project, which acknowledged the absence of assistants in kindergarten research. The fourth criterion was that the publications were written in Norwegian, Danish, Swedish or English, the predominant languages for empirical Norwegian kindergarten research. The fifth criterion concerned the relevant sources for the research. The database Oria was selected, as it includes NorArt, Idunn, ERIC, Academic Search, Sage, Taylor and Francis, Ebook Central and other sources. Searching in Oria ensured access to all databases that are available at Norwegian research libraries. In addition, another database was consulted – the Nordic Base of Early Childhood Education and Care (nb-ecec.org). These five criteria are presented in Table 1.

Search process

Searches were performed using different combinations of the following terms: barnehage* + forskning* OR forsking* + prosjekt* OR studi* OR empiri* + assistent* + phd. Seven searches combining the different search terms were performed in the Oria database, as presented in Table 2. Some of the hits in searches 1–7 were duplicates and were therefore excluded. Furthermore, a search of the nb-ecec.org databases was included, referred to in Table 2 as search 8. As this is a source for Nordic kindergarten research, all 278 hits for the variable Norge (Norway) were investigated manually. Some of the resulting hits were not peer-reviewed (for example, book chapters), so much of the work for this search involved sorting out which studies met the criteria for this study. Other hits duplicated searches 1–7 and were therefore eliminated. As a result of these procedures, 184 hits were excluded, so search 8 generated 94 new hits. Subsequently, a manual search was performed (search 9), based on the hits in searches 1–8 that appeared to be parts of PhD dissertations. Eight articles that had not come up as hits in the earlier searches were found in this way. Finally, different combinations were used in Oria corresponding to searches 1–7: kindergart* + research + empiri* + Norway OR preschool + research + empiri* + Norway OR kindergart* + assistant + empiri* + Norway OR preschool + assistant + empiri* + Norway OR kindergart* + assistant* + Norway OR preschool* + assistant* + Norway OR kindergart* + phd + Norway OR preschool* + phd + Norway. Searches 10–17 gave only 14 relevant hits. The search process and the sources used are presented in Table 2.

The results of the database searches included 184 hits in total (cf. Table 2), based on the inclusion and exclusion criteria set out in Table 1 and excluding duplicates. Articles that were not peer reviewed were excluded. The target publications were then printed out and assessed in full after all the searches were complete. After reading the full text of the source, 35 of the hits from searches 1–17 were found not to meet the inclusion criteria and were excluded. The number of relevant publications was therefore reduced to 149.

It was surprising that so many publications were excluded after the search process was complete, but according to Hart (2018), it is necessary to read and evaluate research findings analytically and critically. Even though the search process was thorough and comprehensive, this study may have missed relevant publications because of its inclusion criteria, the choice of databases or for other reasons. The data was collected by the first author of the study, while both authors participated in the analysis and writing stages. Appendix A presents the full list of publications retrieved from our sources, whereas Appendix B lists the publications that included assistants as informants.¹

Groups of informants among kindergarten staff

Figure 2 shows that the category of kindergarten teachers/pedagogical leaders was by far the largest group of informants engaged in Norwegian kindergarten research. In 21 publications (14.1%) the occupational background was unknown; these were coded simply as ‘staff’. In the other 128 publications, assistants were selected as informants in 43 publications (28.9%; see Appendix B). Kindergarten teachers and pedagogical leaders were selected as informants in 115 publications (77.2%), and chairs in 30 publications (20.1%). Some publications included different occupational groups as informants (cf. Fig. 3). Figure 2, however, shows each occupation separately. When coding the publications, more than one hit was registered per publication, as relevant.

Figure 3 shows that in the 43 publications where assistants participated as informants, they were never the only group of informants selected. Other groups among the staff were always engaged as informants as well, together with the assistants. Usually, the assistants were relied on as informants along with kindergarten teachers. However, in one publication, assistants and chairs were selected as informants (Tkachenko, Giæver, Bratland & Syed, 2015). All three staff groups were relied on together as informants in five publications (Børhaug, 2011; Carlsen, 2015; Nilsen, 2017; Seland, 2009; Storjord, 2008).

Research design in the target studies

The main methodological approaches in the publications where assistants participated as informants were qualitative. Assistants were engaged as informants in 32 qualitative, 8 quantitative and 3 mixed-method publications. Qualitative publications mainly used interviews, while the quantitative publications in this field all used surveys. Mixed-method studies used both interviews and surveys (Børhaug, 2011; Gradovski & Løkken, 2015; Storli & Sandseter 2017).

Furthermore, qualitative research designs were dominant in the sample of 149 publications generated by our searches. Of this total, 120 publications (80.5%) used a qualitative design, 19 publications (12.8%) used a quantitative design and 10 publications (6.7%) used a mixed-method approach.

Finally, we investigated to what degree those conducting research on kindergartens have kindergarten teacher backgrounds themselves. We conducted web searches to investigate the background of all first authors in the sample of 149 hits. Information on academic background was found primarily through employment relationships on websites of universities and university colleges. Persons not employed in this sector were found through manual searches, book reviews and other sources which presented the authors’ academic backgrounds.

For publications with more than one author, we identified and counted the first author and not the others. Among all the publications in the frame (N = 149), 63 were written by 44 researchers, all of whom have a background as kindergarten teachers. Some of the authors have written several publications (Appendix A). Of the 43 publications that used assistants as informants, 15 of them (34.9%) were authored by 13 kindergarten teachers (Appendix B). Otherwise, assistants were informants in 28.9% of the studies in the whole sample (N = 149). This shows that when kindergarten teachers were conducting the research, there may have been a tendency to use assistants as informants to a greater degree than other researchers in the field did, but the sample is small and it is necessary to conduct more research concerning this finding.

Main findings

In the timeframe of this review (2008–2017), assistants were never the sole informants in any of the publications. They were included as research subjects only together with other groups of staff members, mainly kindergarten teachers/pedagogical leaders. The resulting impression is that the absence of assistants as informants, which was the basis of the MAFAL Project, persists. The MAFAL Project, which explicitly investigated and compared responsibilities and cooperation between kindergarten teachers and assistants, produced 7 hits within the specified time parameters (Løvgren, 2012; Løvgren and Gulbrandsen, 2012; Smeby, 2011; Steinnes, 2014; Steinnes & Haug, 2013; Vatne, 2012; Håberg, 2015). Moreover, assistants were selected as informants in 36 additional publications; these represent 25.4% (N = 142) of the studies if the hits from the MAFAL Project are excluded.

Given that assistants are the largest occupational group among kindergarten staff, our finding that they were never the sole subjects of any research is striking. Following this observation, we argue that the representation of the various groups of staff members in Norwegian kindergarten research is skewed and, furthermore, that representation is clearly biased in favour of kindergarten teachers. As we discuss in greater detail below, this distortion may have several unfortunate consequences.

We argue that assistants are potentially very important as informants in kindergarten research because they are the occupational group which devotes the most time to direct interaction with the children (Glavind, Pade & Pade, 2000; Vassenden, Thygesen, Bayer, Alvestad & Abrahamsen, 2011). Pedagogical leaders spend less time per day with children than assistants do because several hours each work week are occupied with planning, in addition to participating in internal and external meetings (Nicolaisen, Seip & Jordfald, 2012). Overall, this leads to the impression that assistants offer continuous adult supervision for the children, while kindergarten teachers are more often absent during the day. Therefore, we argue for an enhanced appreciation of the understanding of the work and experiences of assistants, and believe that these should be studied on their own terms, not only in relation to other groups of kindergarten staff.

The under-representation of assistants as informants might express a hierarchical structure (Håberg, 2015) where assistants are perceived as relevant actors only to a limited extent. Enehaug, Gamperiene and Grimsmo (2008) point out that assistant jobs offer easy access to the labour market, with minimal education requirements and opportunities for part-time work. Perhaps assistants as a group are seen to have low status, but this is typical for kindergarten teachers as well (Kunnskapsdepartementet, 2014). In some of the publications in our study, assistants were used as informants in informal interviews in observation situations, but in formal interviews only kindergarten teachers were engaged as informants (Skreland, 2016). In another publication, the assistants, who were not informants, were more practically oriented and less pedagogically oriented than the kindergarten teachers (Eik, 2014). The under-representation of assistants might even be interpreted as an expression of an occupational conflict. In Appendix A, 21 of the 149 publications (14.1%), did not make reference to the staff’s professional backgrounds when they were used as informants. In Appendix B, 9 of the publications (N = 43) did not specify the number of assistants used as informants (Bjørgen, 2011; Figenschou, 2017; Granly & Maagerø; 2012; Hillesøy, 2016; Hillesøy & Ohna, 2014; Johansson, Emilson, Röthle, Puriola, Broström & Einarsdottir, 2016; Knudsen, Aglen, Danbolt & Engesnes, 2015; Larsen, 2014; Skreland, 2016).

Whether kindergarten teachers work as pedagogical leaders or chairs, they have appeared to researchers to be the most interesting occupational group to use as informants. The skewed representation of assistants and kindergarten teachers might also relate to the nature of the research questions studied. Such subjects as planning and evaluation of pedagogical work are primarily the responsibility of kindergarten teachers (Kunnskapsdepartementet, 2017), who might therefore be seen as the most relevant informants on such topics. In this study, we have categorised such publications as not relevant for assistants to participate in as informants (for example, Alvestad & Sheridan, 2015). However, assistants might also have interesting views and experiences related to planning and evaluation, which might complement the picture.

Appendix A shows that in 82 of the 106 publications (77.4%) that do not have assistants as informants, the topics under investigation might be equally relevant for assistants to offer their perspectives. Topics such as what is going on in the kindergarten, play, learning, interactions, gender and migrant children (see, for example, Bae, 2012; Kalkman, 2017; Kasin & Slåtten, 2015; Nygård, 2017) might generate a more diverse set of views if assistants were also included. Various occupationally specific challenges and dilemmas might be illuminated. On the other hand, topics like leadership, planning, professional development and kindergarten teacher education (for example, Bø & Hognestad, 2014, 2015, 2017; Moen, 2017; Skjæveland et al., 2017; Vatne & Gjems, 2014) are defined in this study as typical topics that might not readily accommodate assistants as informants. Publications where the research question is specific to designated occupational groups, such as chairs (for example, Børhaug & Lotsberg, 2014; Evenstad & Becher, 2015; Kirkhaug et al., 2012) or pedagogical leaders (for example, Keilman, 2017; Knaben & Abrahamsen, 2017; Ødegård, 2012) are not defined as relevant for assistants either.

In other countries, where assistants are in the minority among the staff complement, it is less surprising that they have played a limited role as informants in kindergarten research. In a Norwegian context, however, we argue that the marked lack of assistants raises serious methodological and epistemological questions, since the largest group of staff members is understudied. We will continue this discussion by reflecting on how the skewed representation of kindergarten teachers and assistants might affect scholarly knowledge about kindergartens. In this context, it is necessary to discuss the issue of validity.

Validity

This review synthetises evidence on the degree to which assistants have been engaged as research subjects in Norwegian kindergarten research, and does not, therefore, comment on the validity of individual publications. However, the current study enables us to reflect on the total body of research and discuss the accumulated contributions on this topic, wherein validity becomes a major issue.

Creswell and Miller (2000) have argued that validity does not refer to the data but to the inferences drawn from it. Moreover, they underlined that one key feature of validity is ‘how accurately the account represents participants’ realities of the social phenomena and is credible to them’ (Creswell & Miller, 2000, p. 124). Thus, they accentuated the importance of accurate representation of the research setting and how it is understood by actors themselves. Following this insight, we argue that the lack of assistants’ voices in Norwegian kindergarten research may negatively affect the quality and accuracy of the studies undertaken (Creswell & Miller, 2000). Furthermore, one might argue that the descriptions of data and findings will necessarily be of poorer quality in the total body of studies because the largest occupational group in the field is marginalised. From a phenomenological perspective, (Jacobsen, Tanggaard & Brinkmann, 2015) the implications of this include a likely loss of different perspectives, nuances, understandings and experiences and, furthermore, an epistemic loss of knowledge. Consequently, we argue that important gaps in kindergarten research in Norway may result from the skewed representation of the staff. It is not sufficient to build the knowledge of kindergarten almost exclusively on data from the smallest occupational group.

In their highly influential article on validity in qualitative research, Creswell and Miller (2000) described nine validity procedures. Since our focus primarily concerns the total body of evidence and not individual studies, we will narrow the discussion down to three procedures: member checking, thick description and triangulation (Creswell & Miller, 2000). First, Creswell and Miller emphasise member checking as ‘the most crucial technique for establishing credibility’ (2000, 127). This procedure entails presenting data and interpretations to the participants in the study ‘so that they can confirm the credibility of the information and the narrative account’ (2000, 127). As mentioned above, the main results of this study indicate that Norwegian kindergarten research in our time frame (2008–2017) does not adequately represent the population working in Norwegian kindergartens; the largest occupational group has only to a limited degree been included as research subjects. Moreover, we found evidence of conflation in 21 publications (14.1%), where informants were coded simply as ‘staff’. These publications merged the different occupational groups under the term ‘staff’ without distinguishing between assistants, kindergarten teachers and chairs. Conflation increases the possibility of bias and reduces the accuracy of inferences. Using the term ‘staff’ might distort the presentation of study results and disguise eventual findings that could show different occupational groups as having different roles and performing their tasks in different ways. For example, by designating informants explicitly as either assistants or kindergarten teachers, significant differences between kindergarten teachers’ and assistants’ work might appear. Håberg (2015) distinguished between different occupational groups, including in her study both visible practical work and invisible work, such as planning and reflection. Consequently, clear differences between occupational groups could emerge. This contrasts with the claim that kindergarten teachers and assistants in the Norwegian context are doing the same kind of work in an organisation characterised by flat structure (Aasen, 2010; Bøe, 2011; Nørregård-Nielsen, 2006). We conclude this paragraph by claiming that the lack of assistants in Norwegian kindergarten research obviously infringes on their involvement in member checking.

Second, Creswell and Miller (2000) also accentuated the role of thick description as a validity procedure. Thick description involves contextualising the people and sites studied, giving detailed, deep and dense descriptions of the setting, participants and themes. The purpose of this approach is to create verisimilitude, ‘to produce for the readers a sense that they have experienced, or could experience, the events being described in a study’ (2000, p. 129). Creswell and Miller argued that thick descriptions ‘enable readers to make decisions about the applicability of the findings to other settings or similar contexts’ (2000, p. 129). Following this insight, we emphasise that the representation of Norwegian kindergarten staff we have documented is evidently at odds with the ideal of thick description. Furthermore, we argue that this skewed representation may threaten the transferability or generalisability of the results.

Third, Creswell and Miller have emphasised triangulation as another procedure to enhance validity. They described triangulation as ‘a search for convergence among multiple and different sources of information to form themes or categories in a study’ (2000, p. 126). They underlined the use of multiple methods as a technique to provide more valid narrative accounts. Furthermore, triangulation is the rationale for mixed-method research designs: such designs enable complex and multifaceted investigations. We found that only 7% of the studies investigating the work of assistants used mixed-method designs. Meanwhile, in line with several earlier Scandinavian research reviews, our review has shown qualitative designs to represent the most common approach in kindergarten research (Bondebjerg et al., 2019; Bondebjerg et al., 2018; Bondebjerg et al., 2017; Højen-Sørensen et al., 2016; Wendt & Jørgensen, 2015). For example, the latest review (Bondebjerg et al., 2019) dealt with the year 2017 and found that 79% of all studies (N = 121) had a qualitative design. Our study of assistants as informants arrived at the same tendency (80.5%, N = 149).

More empirical knowledge is needed about the dynamics of kindergartens and their context, along with more sophisticated research designs. The role of kindergarten in Norway has changed, from being an opportunity experienced by some of the children from the age of one until they start school at six (60% in 2000) to becoming a normal and nearly universal element of early childhood (92% in 2018; SSB, 2019). This high percentage of participants has been highlighted from a policy perspective as an opportunity to give children who are not enjoying stimulating and supporting home environments experiences that promote learning and development (Kunnskapsdepartementet, 2019). The kindergarten’s role – to compensate for social inequality and give all children an equal chance when starting school – depends on a high-quality learning environment (Aukrust & Rydland, 2009); therefore, this knowledge base is of great importance.

As shown above, a clear majority of Norwegian kindergarten research uses qualitative methods. In this research, the selected groups of respondents are typically small and strategic, and researchers emphasise contextualisation and focus on unique characteristics (Karpatschof, 2015). However, qualitative studies do not offer the opportunity to generalise or point to causality in ways that could paint a broader picture of the kindergarten context (2015). Qualitative research is more vulnerable to sampling error, that is, ‘error in the findings deriving from research due to the differences between a sample and the population from which it is selected’ (Bryman, 2008, p. 168). The predominant use of qualitative approaches in kindergarten research amounts to adding many small pieces to a puzzle. The problem is that these pieces might, to a large extent, be placed on top of each other, while not contributing to completing the picture. In closing, many of the publications that included assistants as informants have had methodological limitations. Three or fewer assistants were consulted as informants in eight publications, a very small sample (cf. Appendix B). Moreover, nine of the publications did not quantify the number of assistants included as research subjects (cf. Appendix B); the latter methodological choice threatens the transparency of the research process.