Evaluation of the Welsh Language Sabbatical Scheme for education practitioners: summary

This report presents the findings of an evaluation of the Welsh Language Sabbatical Scheme for education practitioners ('the Scheme'). Arad, in partnership with researchers from Iaith Cyf., was commissioned to complete the evaluation, and the evidence was gathered between March 2019 and March 2020.

The Scheme offers periods of intensive study away from the classroom, for education practitioners to develop Welsh language skills and gain confidence in bilingual and Welsh- medium teaching methodologies. A programme of courses is delivered across Wales under a contract with Welsh Government. During the period of the evaluation, namely the academic years between 2013/14 and 2018/19, 1,299 practitioners participated in one of the courses.

The courses are available at different levels:

• Mynediad/Entry, a 5-week full-time course for teaching assistants working in English-medium primary schools
• Sylfaen/Foundation, an 11- week full-time course for teachers in English-medium primary schools
• Uwch/Advanced, a part or full-time course for teachers or teaching assistants, who are already Welsh speakers
• Canolradd/Intermediate course for practitioners in English-medium primary schools (these courses are referred to as 'block courses')

There is a year-long full-time course available since September 2007 for teachers in English-medium primary schools, namely Welsh in a Year. 

The Sabbatical courses are delivered by three training providers under a contract with Welsh Government. The three contracted providers delivering during the period in question were Bangor University, Cardiff University and University of Wales Trinity Saint David.

During the 2018/19 academic year, there was a budget of £3.38m to deliver the Scheme, and in that academic year, block courses were provided at a range of levels with 143 participants, and five Welsh in a Year courses in four locations across Wales with 68 participants.

The aim of the evaluation was to assess the impact of the Scheme and examine two elements:

• how, and to what extent the Sabbatical Scheme contributes to change in the way Welsh is taught or used as a medium of teaching in schools
• the contribution of the Sabbatical Scheme to Welsh-language or Welsh-medium professional development provision for practitioners

The evaluation used a combination of methods to examine the research questions. The methodology included:

• desk-based research
• interviews with officers from the four regional education consortia, Welsh Government and training providers
• telephone interviews with 46 practitioners who had participated in the block courses and eight teachers who had attended a Welsh in a Year course
• telephone interviews with a sample of 26 headteachers and senior managers from whose schools practitioners had followed a Sabbatical course

The fundamental theory underpinning the Scheme is that practitioners' Welsh language skills and their use of teaching methodologies need to be improved, and that Sabbatical schemes away from the workplace are an appropriate way of improving these skills.

The theory of change prepared as part of the evaluation identifies the steps that lead from planning at a local, regional and national level to the associated activities of recruiting practitioners and delivering training, and then to the anticipated outputs of those activities: an improvement in the Welsh language skills and Welsh language teaching skills of the practitioners who attend the courses. The theory demonstrates that improving these skills should contribute to longer-term effects such as improving the teaching of Welsh in schools, increasing the number of practitioners who can teach Welsh and through the medium of Welsh, and increasing the number of Welsh speakers. 

The theory of change identifies the assumptions or conditions required to be at play to attain these outputs and results. The theory also considers the external factors, outside the scope of the Scheme's direct control (including the wider national policies and strategies) that can influence progress along these stages.

Planning and strategic use of the Sabbatical Scheme 

At a national level, the Welsh Government strategy for the Welsh language, Cymraeg 2050: a million Welsh speakers, the introduction of the new curriculum, and the professional standards influence the need to improve the Welsh language skills of the workforce and therefore the need for the training available through the Scheme.

Regional officers use a variety of methods to collect information about the likely need and demand for training courses among practitioners in their area. However, it became apparent from the evidence collected, that data available at a local and regional level depends on the information collected by individuals within the consortia and local authorities. The quality and nature of this data are not consistent, and in certain cases are anecdotal in nature. There is no consistent record of this information, and it is not widely shared.

Recruitment

Consortia officers and local authority athrawon bro are chiefly responsible for engaging with schools and recruiting practitioners to the courses. The recruitment process, though sometimes challenging, was found to be targeted towards those schools where the greatest need for training has been identified. The recruitment process is reliant on ensuring that there is both the time and the resources to engage with schools, and consortia officers reported that the time and resources available for them to focus on this recruitment process were sometimes lacking. Some stakeholders reported that a lack of communication and information-sharing between partners had posed barriers in the past.

Although stakeholders interviewed identified only a small number of examples of challenges encountered in the recruitment process, they highlight the confusion that can arise when there are a number of institutions engaging in similar processes by way of marketing and recruitment. On the basis of evidence gathered, it appears that communication processes and information-sharing between all institutions involved in the recruitment process could be improved.

Deciding to participate

The need to develop skills that would support some of the school's strategic Welsh language aims (and particularly in preparation for the new curriculum) was the main reason identified by headteachers for encouraging practitioners to attend either the Entry or the Foundation level course. The reasons cited by headteachers for participating in Advanced courses were to a large extent about the need to raise individual practitioners' language skills. Headteachers who had supported a Welsh in a Year practitioner tended to share examples of where they as a headteacher, or other members of the senior management team, had been proactive and had placed greater emphasis on selecting (or refusing to allow) practitioners to attend the course. These findings suggest that headteachers consider that the opportunities for upskilling offered by the Scheme correlate with their school's strategic Welsh language needs and that they see the training offered through the Scheme as a way of addressing them.

Practitioners' motivation to apply varied according to the course level, but most practitioners, across courses of all levels, cited reasons relating to their desire to improve their Welsh language skills as part of their own professional development.

View on the courses

There was a strong consensus among practitioners and headteachers that the structure and content of the courses was suitable for them. A large number (spontaneously) stated their view that the teaching provision was also of a very high standard. Over half the practitioners interviewed (from courses of all levels) felt their course was very intensive, but in almost all cases, practitioners stated that their experience, though challenging, had been worthwhile. Training providers and regional officers felt that the content and structure of the courses suited practitioners' needs, but they raised a few points about how, in their view, the provision currently available could be improved.

All regional officers interviewed, in addition to some providers and headteachers, were eager to consider other and alternative methods of developing Welsh language skills. The ideas offered by some raise questions about the understanding among the different stakeholders of the scope of the Scheme. It is clear that there is a demand for more Welsh language training and support for practitioners, in addition to what is currently available. However, consortia already have a role in delivering this type of less intensive training to supplement the Scheme. Based on the evidence gathered, it appears that further consideration may need to be given to how practitioners and headteachers understand and use the Scheme, rather than the Scheme needing to change.

Follow-up support after the courses: regional consortia

The evaluation considered how much support schools and practitioners received as part of the Scheme after completing the course, and what other activities practitioners and schools undertook to support the skills developed during the course. 

The role of the consortium is to provide follow-up support for the courses, and they do this via the support of schools' athrawon bro/advisory teachers. Funding this provision, however, is outside the Scheme's contract, and it is for each individual consortium to determine what they fund and how they do so. The regional consortia can use the Regional Consortia School Improvement Grant to fund this provision.

The majority of practitioners interviewed stated that they had received no support from anyone outside school following the courses. Some practitioners and headteachers, explained, however, that they had received support to implement what had been learned on the course, including school visits from athrawon bro or a consortium officer to observe and offer feedback on lessons, language refresher lessons and planning support. Of the 54 practitioners interviewed, 20 stated that they had been supported by their headteachers or other colleagues in their school to support them in implementing the Welsh language skills that had learned. Examples of the support varied from Welsh-speaking colleagues making a concerted effort to speak Welsh to them outside the classroom, to specific sessions with the headteacher or during in-service training days.

Most practitioners and headteachers stated they would welcome further support after the course. In the case of the Welsh in a Year course in particular, practitioners, headteachers and providers shared the same aspiration of the need, in their view, to formalise slightly the after-care available in the period following the course, possibly as part of their terms of undertaking the course. A number of interviewees felt this after-care element should be included as part of the Scheme. This suggests that practitioners and headteachers are misinterpreting the role of the Scheme as being one that supports practitioners' continuing development of skills and teaching practices.

Continuing Professional Development

It is neither the role of the Scheme nor the responsibility of regional consortia alone to support the continuing development of individual practitioners' Welsh language skills. Individual schools and practitioners have an individual responsibility for their own continuing professional development. There was clear evidence in some practitioners' application forms for the Scheme of their intention to use the courses as part of their personal professional development. However, practitioners could offer very few examples during interviews, and only a small number stated that they had continued to seek opportunities to develop their Welsh language skills and bilingual and Welsh-medium teaching methodologies after completing the course. 

Several practical reasons were given for not developing their skills further (e.g. time pressures, responsibilities outside school, change of role in school). Nonetheless, the evidence suggests that maintaining the skills has not generally been a priority for them as practitioners nor for their schools during the years following the course. On the basis of the evidence gathered, it appears that a number of practitioners and their headteachers see their participation in the Scheme as being the only step required in relation to their Welsh language skills rather than merely one step within continuing professional development.

Impact of the courses and the Scheme: impact on language skills

Practitioners' views were sought on how and to what extent they implement the skills and teaching methods developed, and the effect this has had on the way they teach Welsh or use Welsh as a medium of teaching. All practitioners interviewed stated that their Welsh language skills, in their view, had improved. Most practitioners reported that their confidence in using those skills had developed or improved as a result of taking part in the Sabbatical course.

The assistants who attended the Entry level course stated that they felt they had developed sufficient Welsh language skills to enable them to use these skills effectively in the classroom. Most practitioners who attended the Foundation course stated that they believed their skills had improved and their confidence in Welsh increased following the course. All interviewees who attended the Advanced courses stated that they believed there had been a significant increase in the accuracy of their language, particularly their written language.

The examples shared of increased language skills were more numerous among those who had followed the Welsh in a Year course. Most practitioners on the Welsh in a Year course felt they were almost fluent in Welsh after the sabbatical period and as a result of the course. This was highlighted in the fact that six out of eight of those interviewed chose to hold the interview entirely through the medium of Welsh. These practitioners offered examples of how the increase in their Welsh language skills and vocabulary had enabled them to deliver Welsh-medium lessons across a number of subjects, and to be confident in marking pupils' written and oral work in their schools.

Although the practitioners considered that their language skills to have increased since attending the course, it was not possible to attribute practitioners' linguistic ability at the end of the course entirely to the training received. Practitioners' linguistic ability at the end of the course was partially dependent on their linguistic ability at the beginning of the course.

Impact of the courses and the Scheme: impact on teaching methodology

In addition to improving practitioners' language skills, the Sabbatical courses provide participants with training on teaching methodologies. Practitioners on the Foundation course shared most examples of how they had adapted and changed their teaching methods after completing the course, and they stated that the methods developed in terms of how to teach Welsh as second language had been particularly useful for them and had enabled them to introduce more Welsh in to other subjects across the curriculum.

Every teaching assistant who had been on the Entry level course stated that they had introduced more Welsh in to the classroom as a result of the course, and all those who participated in the Welsh in a Year course offered examples of how they had introduced more Welsh across the curriculum in their classes. Although practitioners on the Advanced level course already used Welsh, they felt that the course had improved the accuracy of their language, and in their view, the quality of delivery of their lessons.

Although the examples identified above provide anecdotal evidence relating to the skills developed and how they are used, there is no wider evidence of how the skills developed by all practitioners who attended the courses are used or about the difference they make. Schools are expected to identify evidence in their school development plans of how this training has made a difference. However, it does not appear from the interviews held with consortia officers that any such evidence is produced or gathered at a regional or national level.

Impact on the use of Welsh across the school

Most practitioners and headteachers interviewed stated that they believed there had been an increase in the informal use of the language outside the classroom, and more broadly across the school since practitioners from their school had been on one of the Entry, Foundation or Welsh in a Year courses. Among the examples offered were increased use of incidental Welsh when talking to colleagues and pupils outside the classroom, more Welsh-medium visual displays, and more Welsh-medium work in the classroom.

There were numerous examples of increased use of incidental Welsh by practitioners after attending the Sabbatical courses, but no clear evidence that the Scheme was what prompted the increase. According to headteachers, there are factors, such as preparation for the new curriculum, some schools' involvement in the Welsh Language Charter/Cymraeg Campus, or the practitioner's personal motivation, that have influenced this increase also. However, the Scheme appears to have facilitated the increase, though other factors have also influenced the need to aim for the increase. As such, it is possible to conclude that these other factors, and the Scheme, reinforce each other in contributing to the change in the way in which Welsh is taught and used in schools.

Headteachers and practitioners were asked to indicate whether they felt taking part in the Scheme had influenced their school's attitude and ethos towards the Welsh language. Most headteachers and practitioners deemed that their schools already had a positive attitude and ethos towards the Welsh language. However, a small number reported that the Scheme had had a positive influence on their school's ethos and attitude, but that the Welsh language was increasingly becoming a priority for a number of reasons including the introduction of the new curriculum, responding the recommendations following an Estyn inspection and participation in the Welsh Language Charter programme.

Impact on practitioner leadership

One of the outcomes the Scheme seeks to achieve is that practitioners demonstrate greater leadership in relation to the Welsh language within their school upon their return. In order to investigate whether practitioners had done this, headteachers and practitioners were asked whether individuals took on additional responsibilities within their school as a result of the course. A number of practitioners referred to examples where they now played a prominent role in arranging and running activities in Welsh and encouraging enthusiasm in others. 

A greater number of examples were provided by those who had attended the Foundation course of activities or arrangements in their school for which they had taken the lead since attending the course. These included: undertaking the role of Welsh language coordinator at their school; supporting the current Welsh language coordinator; starting a Welsh club and establishing a parents' group.

It appears from this small sample of interviews that some practitioners who took part in the Scheme have taken on new responsibilities which influence the teaching of Welsh across their school. However, it is not clear from this evidence whether the additional responsibilities taken on by these individuals were as a direct result of their participation in the Scheme. Some practitioners had already taken on these responsibilities before attending the course, and others had attended the course in preparation for the additional responsibilities they were expected to undertake after completing the course. Nonetheless, in all cases, practitioners thought they were better able to undertake those responsibilities as a result of attending the course.  

Sharing skills and experiences

Headteachers were asked whether there were any arrangements in place for practitioners to share their experiences and new skills with their colleagues after returning to school. There were some examples where practitioners had shared ideas and experiences with colleagues during staff meetings or during day-to-day networking within the school.

However, these examples were in the minority and limited mainly to the practitioners who had participated in the Welsh in a Year course. Most headteachers interviewed stated there were no specific arrangements in place at their school to enable participants in the Scheme to share the experiences across the school.

Stakeholders were also asked whether there were procedures in place for practitioners to share their experiences more widely beyond their own schools after the course. Consortia officers talked about conferences and events where those who had been involved in the Scheme had had an opportunity to speak about their experiences. Only practitioners who had attended the Welsh in a Year course gave examples of how they had shared their experience and teaching skills with other schools.

The evaluation report was completed during the COVID-19 pandemic, based on fieldwork undertaken before the pandemic began. The climate therefore in terms of delivering the Scheme has changed during recent months, with a number of courses being postponed for practical reasons, and others being converted into online courses. In considering the conclusions and recommendations that follow, it is important to bear in mind that the context in terms of implementation has changed.

Overall findings and Strategic use of the Sabbatical Scheme

The policy context highlights the need to improve teachers' Welsh language skills in line with the requirements of the professional standards and the new curriculum. Welsh Government has also set an ambitious target of reaching a million Welsh speakers by 2050 and has emphasised the importance of the education sector in achieving this aim. As a result, Welsh needs to be taught to all pupils, and in order to achieve this, there is a need to ensure that practitioners possess the skills to do so. This evaluation concludes that the Scheme makes a positive contribution to practitioners' Welsh language skills, and that there continues to be a need for the Scheme. 

Recommendation 1: Welsh Government should continue to fund and support the delivery of the Welsh Language Sabbatical Scheme as part of a programme of professional learning for school practitioners.

Recommendation 2: in promoting the Scheme and recruiting practitioners, regional education consortia officers should draw schools' attention to how the Scheme contributes to realising the aims of Welsh Government's education and Welsh language strategies and to enabling practitioners to meet the professional standards.

Although a number of schools identify the need to improve practitioners' Welsh language skills and bilingual or Welsh-medium teaching methodologies in their school development plans, there does not appear to be a clear requirement on schools to provide further evidence as to how, or to what extent, the skills and methodologies have been put into practice after practitioners' sabbatical has ended. In order to assess the impact of the Scheme in a more systematic way, a more specific process of recording the use of skills back at the school and sharing that information with the consortia and Welsh Government would be essential. 

Recommendation 3: regional consortia should ensure that schools that release practitioners to attend courses systematically report on how practitioners use their new skills after they return to school at the end of the course. Regional consortia should support schools to include this evidence in their school development plan and in discussions with the challenge advisors.

The Scheme's contribution to developing practitioner skills

The evidence gathered during the evaluation suggests that the Welsh language skills of the practitioners interviewed had improved, in their view, as a result of attending one of the Sabbatical courses. Most practitioners and headteachers recognised the need to maintain and develop Welsh language skills and teaching methodologies further after completing the course, and many stated they wanted more support. It would appear that many headteachers, practitioners and stakeholders expect this continuing development support to be provided via the Scheme, in a supplementary package, although this is neither part of the Scheme’s design nor part of the contract to deliver the Scheme’s courses. It is the consortia’s role, outside the Scheme’ contract to provide follow-up support to practitioners after the courses.

Recommendation 4: regional consortia should share clear information with participants on the nature and source of follow-up support available on the conclusion of the course. Expanding on the work that has begun on an after-care package, this task could include mapping the follow-up support and training available across each region and developing a clearer understanding of the follow-up support needs.

Recommendation 5: regional consortia should extend their support for practitioners and schools by offering further support on how to plan to make the best use of the skills developed by practitioners during the Sabbatical courses.

The findings suggest that further consideration is needed of how the Scheme fits with school development plans, and the role of the Scheme within the range of professional learning provision for practitioners. In addressing this further, the role of the practitioners themselves would need to be emphasised, along with the role of school leaders in taking responsibility for recognising and attending to the continuing professional development needs of their staff, rather than expecting that taking part in the Sabbatical Scheme in itself to be the only solution.

Recommendation 6: Welsh Government and regional consortia should emphasise that Sabbatical Scheme courses are one form of training, among an array of professional learning possibilities. This should be done to encourage and support schools and practitioners to consider the Scheme as one method of supporting the development of practitioners' Welsh language skills and bilingual and Welsh-medium teaching methodologies and not the sole method.

The Sabbatical Scheme's contribution to school teaching practices and curricular provision

Practitioners who attended the Foundation, Advanced and Welsh in a Year courses stated that they had developed new teaching practices as a direct result of the course. Practitioners who had participated in the Entry level course stated that they had learnt more about the rules of writing the language and had developed in confidence. Evidence was also gathered by practitioners and some headteachers to show how these skills had been put into practice by those who had attended the course.

For the Scheme to succeed, practitioners must be able to share the new teaching practices with colleagues in their schools at the end of the course. There were some examples of practitioners sharing their knowledge of the teaching practices and resources developed during the sabbatical period with colleagues within their school and beyond, but this sharing did not happen consistently.

Recommendation 7: schools, local authorities and regional consortia should support practitioners to share teaching practices and examples of implementing their new skills with colleagues and other schools in order to extend the reach of the Scheme's impact.

Evidence for evaluation purposes

The evidence gathered for the evaluation provides information relating to the impact of the Scheme on practitioners and schools in the short term. It does not enable us to draw firm conclusions as to the Scheme's contribution to longer-term outcomes. To reach these conclusions, further quantitative data would need to be collected and reviewed. The process of developing the theory of change highlighted some key data areas that would be needed, in addition to some gaps in the data currently available at a national and regional level.

Recommendation 8: Welsh Government should consider undertaking quantitative research to gather information from beneficiaries of the Scheme, practitioners and schools alike, to contribute to a fuller picture of the Scheme's impact.

The evidence gathered through the evaluation process suggests that the theory of change offers an accurate reflection of the logic flow between the Scheme's inputs, outputs and outcomes and its contribution to longer-term outcomes. The process of analysing the theory has revealed certain gaps in data: the evidence on which the evaluation findings are based is, on the whole, limited to observations and examples provided during interviews. The conclusions could have been strengthened had documentary evidence been available, including data against the indicators of practitioners' use of skills after returning to school (see Recommendation 3 above).

Recommendation 9: Welsh Government should address refining the theory of change for the Sabbatical Scheme by (i) including more stakeholder input; (ii) making the widest possible use of data already available in terms of workforce skills capacity (ability in Welsh and to teach through the medium of Welsh) and (iii) dealing with the gaps in data and evidence that have been highlighted in this evaluation to ensure a fuller understanding of the Scheme's contribution to longer-term outcomes.

Full Research Report: Evaluation of the Welsh Language Sabbatical Scheme for education practitioners

Cardiff: Welsh Government, GSR report number 15/2021

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact research.welsh@gov.wales

Image

 Digital ISBN 978-1-80082-935-0