Evaluation of the Violence Against Women, Domestic Abuse, and Sexual Violence Survivor Engagement Pilot Panel: phase 2 (summary)

The Violence Against Women, Domestic Abuse and Sexual Violence (Wales) Act 2015 (‘the Act’) was Welsh Government’s formal commitment to tackling violence against women, domestic abuse and sexual violence (VAWDASV) through protection, prevention and supporting those affected by these types of violence and abuse.

The Act acknowledged that although women are statistically more likely to experience VAWDASV, anyone can be affected including: men; people from black, Asian and minority ethnic (BAME) communities; people from the LGBTQ+ community; disabled people; younger people; and older people.

The Act is supported by the Wellbeing of Future Generations (Wales) Act 2015, which states that freedom from abuse and violence is a key component of wellbeing. These commitments have led to a focus on delivering meaningful and sustained engagement with survivors of VAWDASV.

The Internal Research Programme (IRP, Welsh Government) was commissioned by the policy team for VAWDASV (Welsh Government) in October 2018 to conduct a small-scale research project to support the creation of a Survivor Engagement Framework for VAWDASV.

There were two phases to the research, which ran concurrently. Phase 1 was focused on how Welsh Government should best engage with survivors of VAWDASV from underrepresented groups. Phase 2 focused on evaluating a pilot Survivor Engagement Panel. This executive summary details the Phase 2 element of the research.

The main objectives of this research were to:

• explore the views and experiences of the target populations with respect to previous participation and effective models of participation
• understand the nature, focus and provision of support required to facilitate the participation of the target populations
• develop evidence-based data collection methods and mechanisms to support the implementation of the National Survivor Panel
• develop an evidence-based outcomes framework and outcomes measures sufficient to measure the effectiveness of the National Survivor Panel
• test and analyse the effectiveness of the National Survivor Panel

The methods for evaluating the pilot survivor engagement panel included three Theory of Change workshops, three pilot panel sessions and semi-structured interviews with Panel participants.

There were three Theory of Change workshops (one with survivors, one with external stakeholders and one with policy officials) completed as part of this phase 2 research. The output of the workshops was a programme logic model (see section 3 of the final report), the purpose of which is to provide a framework through to assess whether the intervention had been implemented as planned and to interrogate the outcomes and impacts of the logic model.

The pilot panel sessions were attended by ten recruited survivors of VAWDASV and took place between September and November 2019. The aim of these was to provide insight into how a survivor engagement panel could be organised and discussions could be facilitated. One objective of the National Strategy 2016-2021 was chosen per session to focus on, and three of the six objectives in the strategy were discussed in total. Semi-structured interviews were conducted with panel members, stakeholders and policy officials to explore how successful the pilot panel sessions had been. A total of 16 interviews were completed. Interviews with panel members and policy officials focused on how the pilot had worked and what improvements could be made for a longer term panel. It was also discussed how best to undertake survivor engagement for the future and measuring the impact of this. Interviews with stakeholders explored the nature of their involvement with the pilot panel, what they had found to be effective with survivor engagement, and why survivors from underrepresented groups do not engage with Welsh Government. 

Findings are based on evidence gathered via the three panel sessions and interviews with survivors, external stakeholders and policy officials.

Recruitment to the panel

Panel members reported that invitations to the survivor’s panel were clear in terms of basic information but could have provided more detail about what was expected of them during the session, what was hoped would be achieved from the sessions and the outcomes.

Panel members would have liked it to be made clear in advance what remuneration they would receive for taking part in the panel, e.g. travel expenses, child care costs and compensation for loss of earnings. Clarity on this would have made committing to attendance at the panel easier. Some panel members also felt that payment for attending the sessions would demonstrate that the Welsh Government valued their input.

It was suggested that relevant material sent in advance of the panel sessions would have been beneficial for members to stay focused on the topic at hand and make the most constructive use of their time.

To help prospective panel applicants know exactly what was required of them during the panel sessions, pilot panel members suggested that a ‘job description’-type guidance document should be provided as part of the recruitment process. This would also help to explain how participation will contribute to a survivor engagement panel. Both panel members and policy officials recognised the lack of diversity on the pilot panel, and it was felt that a guidance document for potential participants could help to achieve more representation.

Using stakeholder organisations as an outreach to help build trust and secure buy-in was seen as beneficial for increased diversity and representation in any future panel. Other suggested methods for improving representation on the panel included using social media to target groups who are not already engaged with organisations and services.

The Pilot Panel

A commonly reported complaint about the panel sessions was that they were not long enough. The panel sessions for the pilot were 2.5 hours long each, but it was felt that this was not enough time to explore the complexity and depth of the topic fully.

Policy officials thought that, on reflection, the topic areas assigned to each of the sessions covered too much for the amount of time available. In response, one suggestion was to vary the lengths and frequency of the sessions to align with the scale of the topic being discussed, as this would allow for more complex objectives to be discussed in depth and more manageable issues could be covered in shorter, less frequent sessions.

The pilot panel sessions were held at around midday and feedback from the panel members indicates that this time of day was good in terms of travelling to the venue and making arrangements for caring responsibilities where required.

There was debate among stakeholders and panel members about whether the Welsh Government offices were an appropriate venue to hold the panel sessions. Some members felt that using the office demonstrated the high status of the panel, whereas others felt it was too formal. Being escorted around the offices by a government official made some feel anxious. There were also questions around accessibility, as the Cathays Park office where the pilot was held is in Cardiff city centre. Prospective participants from more rural areas of Wales may struggle to travel to Cardiff where public transport is not available or appropriate. It was suggested that panels were held across different locations in Wales to overcome this issue.

Group dynamics

Panel members agreed the pilot sessions were well-facilitated and organised. It was recognised that facilitating a group where emotive topics are discussed can be difficult, but panel members felt that everyone was given proper chance to participate.  

It was questioned whether a mixed-gender panel was appropriate for discussing VAWDASV related topics by both panel members and stakeholders, as many survivors have been impacted by male violence against women. However, it was also highlighted that both women and men can experience these types of violence and that men’s experiences should not be excluded.

Some comments made during the panel sessions were triggering or caused discomfort for other survivors. Some panel members felt that having the chance to voice these concerns with facilitators after a panel session would be beneficial in terms of monitoring group dynamics at future sessions.

Panel members believed it was important that Welsh Government recruits survivors who are further distanced from their experiences of abuse, as being in the middle of the situation could make participating in a panel extremely difficult and be detrimental to recovery.

Topics discussed

Panel members and stakeholders welcomed the discussion of the National Strategy objectives, which were all seen as important and relevant.

Some panel members felt that the National Strategy did not address stopping or reducing VAWDASV, only responding to it. It was suggested that an additional objective should be included to reflect preventing VAWDASV.

The topic of the police and criminal justice system was raised a few times as one that should be included in panel sessions. Many panel members reported negative experiences with the police and family courts, so felt this was an important factor in tackling and responding to VAWDASV.

Panel members expressed that they would like the panel to contribute to more senior level decisions, including funding and budgeting. Policy officials agreed that panel input was important when making decisions about spending.

It was suggested that the agenda and aims of panel sessions should be survivor-led as those with lived experience are best placed to know what VAWDASV survivors need and want.

Some stakeholders said that they would like awareness and education around coercive control to be a panel topic, to help the public understand that physical violence is not the only form of abuse. It is believed that there is still work needed to help victims recognise coercive control as a form of abuse and helping them seek appropriate support.

Permanent panel

All research participants felt that a long-term, permanent panel was worthwhile. Whilst all of the panel members involved in the pilot wanted to be permanent members, they also recognised that changing membership occasionally would be beneficial for inclusivity and representation amongst VAWDASV survivors.

It was felt that online engagement methods should be offered to VAWDASV survivors, as not all are able or want to physically attend group sessions.

The pilot panel members all said that face-to-face engagement would be their preferred contact method. It was suggested that online surveys are a ‘tick box’ exercise that lacked any real depth into issues experienced by survivors. It was also felt that surveys can become boring and lead to disengagement.

Panel members were open to virtual meetings as an alternative to face-to-face, but were clear that face-to-face was their preference. There was concern about confidentiality with virtual meetings.

Some panel members were uncomfortable with the idea of other online engagement methods, such as forums, where messages are kept online and it is unknown who exactly is accessing them. The risk of perpetrators infiltrating these sorts of platforms was felt to be very real, and therefore would need to be closely monitored if offered as an engagement method.

Consideration around bringing together various engagement methods was discussed, for example combining virtual and face-to-face meetings. There was concern that people dialling in virtually may be put off future engagement if they experienced technical issues during the session.

Both stakeholders and panel members suggested that there could be some benefit in having not only a national panel but also regional panels. It was perceived that survivors from other parts of Wales may not be able to access face-to-face panel meetings if they were only ever held in Cardiff, as was the pilot. They also felt it was important to recognise how regional differences affect the needs of VAWDASV survivors.

Supporting panel members

It was recognised that supporting the wellbeing of panel members was important given the sensitive and traumatic nature of the subject. It was said that being triggered by these discussions was inevitable. Some felt that having a counsellor available would be helpful, while others did not think it would be useful as they already had their own support systems in place. Having post session check-ins between facilitators and panel members was suggested as an alternative form of support to counselling.

The method of engagement was considered important in how support should be offered to participating survivors. It was felt by panel members and stakeholders alike that face-to-face engagement is the most appropriate method for building trust and providing a safe space for sharing information. There was concern that online methods would not allow for the same level of support.

Ongoing communication with panel members

Panel members agreed that having feedback regarding ministerial decisions relating to their input is important, particularly where a decision is made that goes against panel advice. Understanding the rationale behind decisions is a way in which Welsh Government can demonstrate that panel input is not simply tokenistic.

Stakeholders stated they would like to be kept aware of what was being discussed during panel meetings in case it fed into their organisation’s work.

Measuring the success of a permanent panel

Panel members felt strongly that the engagement panel should be survivor-led and success would be achieved through their personal experiences influencing VAWDASV policy. Meaningful demonstration by Welsh Government that the voices of survivors are taken seriously when formulating policy was considered a crucial marker of success.

It was suggested that the panel should identify gaps in services and support available to survivors, and then advise on how to address those gaps with the appropriate support.

Some suggested that success of a long-term panel could be measured by survivors reporting that they feel more supported and safe as a result of the policies that were advised by the panel.

Policy officials and stakeholders discussed managing the expectations of survivors and panel members as to what Welsh Government can realistically achieve. There was frustration among panel members during the pilot sessions about the limited powers that Welsh Government has, for example over the criminal justice system which is a reserved power. It was discussed how Welsh Government can realistically hold perpetrators to account when they do not have such powers.

Permanent Panel options

Based on the findings from both the pilot panel and interviews with panel members, stakeholders and policy officials, four options were determined for a long-term survivor engagement panel.

Option 1

Face-to-face panel consisting of 10 to 12 members with rotating recruitment every 12 to 18 months. Panel members have the opportunity to lead the panel, set the agenda and objectives. This option is low cost and policy has adequate resource to deliver. However, the COVID-19 pandemic restrictions will need to be considered for any face-to-face meetings going forward. There is a risk with this option that survivors who are unable or do not want to engage face-to-face will be automatically excluded and thus underrepresented.

Option 2

Face-to-face panel combined with online engagement including surveys, webinars and discussion boards, allowing for flexible use and greater accessibility for those who cannot or do not wish to attend face-to-face meetings. This option may allow for more representation from more marginalised groups. However, there is less available capacity within the VAWDASV policy area to co-ordinate both face-to-face meetings and online engagement activities, and potential participants may be put off by the perceived security risk of submitting information online.

Option 3

Specialist short-term sub groups/panels, which work towards smaller outcomes through more flexible groups in terms of size, location and duration. Specialist panels would allow for increased diversity amongst survivors, meaning more varied representation of experience and input towards policy objectives. Having flexible locations also increases accessibility for a more inclusive and pan-Wales approach. This option carries the risk of being resource-intensive for two reasons: more recruitment would be required to ensure specialisms are fulfilled and having numerous groups may make coordination and outcomes more difficult to manage.

Option 4

Three regional panels and ad hoc online consultation. This option would reflect regional contexts and provide increased accessibility for survivors across Wales in terms of geography, access to transport, and choice of Welsh-language facilitation of face-to-face meetings, when compared with a Cardiff-based panel. The ad hoc online consultation aspect would be more secure, with less risk of perpetrator infiltration at one-off activities. This option is more resource-intensive due to the running of three panels across Wales and requires more planning in terms of coordinating outcomes, feedback between panels and facilitation.

Based on the findings of this research, a number of recommendations are made.

The policy team should consider clearly outlining the following aspects of the panel to prospective panel members prior to their committing to participating.

• Terms of Reference of the panel.
• Description of the panel member role.
• Eligibility criteria for panel membership.
• Indication of what the panel is required to do and the anticipated outcomes, to be negotiated once the panel is recruited.
• Clarity on remuneration for panel participants, for example, T&S only, compensation for lost earnings etc.

Panel sessions should be held in a neutral venue.

Sessions of the permanent panel should be clearly structured, ideally with the provision of relevant reading, an agenda and a clear outcome stated prior to the session taking place.

The policy team should consider the following suggestions from panel members and stakeholders.

• Vary the regularity and length of the sessions to align with the scale of the topic.
• Hold panel sessions in different areas of Wales to allow and encourage those based beyond South Wales to participate.
• Hold an introductory session to allow panel members to meet and get to know one another. This would allow for Welsh Government to meet panel members, and for panel members to get to know one another, sharing their experiences before the first panel meeting. The introductory session should also outline which aspects of VAWDASV the Welsh Government can effect i.e. only aspects which are devolved.

The policy team should consider further engagement with stakeholder organisations to raise awareness of the panel and understand the barriers to engagement from the survivors they support, with the objective of improving buy-in to the panel within those marginalised groups.

Develop a clear social media strategy for engaging those currently not receiving support from services.

Welsh Government officials should ensure that the relationships between the panel and the expert stakeholder group are clear from the outset.

The policy team should consider Option 4 as the approach for future survivor engagement.

The policy team should undertake a full evaluation of the permanent panel.

Report authors: L Entwistle and J Coates

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Dr Jo Coates
Email: rhyf.irp@gov.wales

Social research number: 57/2021
Digital ISBN: 978-1-80195-814-1

Image