Managing technology that manages people: a Social Partnership approach to algorithmic management systems in the Welsh public sector

On behalf of the Workforce Partnership Council (WPC), this document provides guidance to managers, trade unions and other worker representatives on the use of algorithms and artificial intelligence systems in devolved public sector workplaces for the management of staff and their work. The document considers the opportunities and risks related to technological systems that are used to manage people at work – for example, in recruitment, task allocation, shift allocation or staff monitoring – and provides general principles that should guide representatives of employers and worker representatives in engaging with dialogue and decision-making on this topic. 

The WPC represents the interests of devolved public sector employers, trade unions and the Welsh government.  Their endorsement of this guidance means that it should be followed in all devolved public sector bodies.  Unions and managers should establish sectoral or local agreements on how best to implement this guidance. The WPC may engage in further work to evaluate how effectively this guidance is being implemented after its publication. The guidance may also be reviewed and amended in the light of changes in the use of technology and its capabilities, as well as the experience of public sector bodies and their unions in implementing its content. This guidance document will be put before the WPC for agreement in November 2024. The document can be considered up to date at that point. 

Public sector bodies have a legal duty to operate in social partnership with their unions, including in matters relating to the management of their workforce. Where public bodies fail to abide by this duty, such a failure can be reported to the Social Partnership Council. 

The document does not explicitly cover the use of generative AI tools by staff, although the WPC may decide to issue guidance on this matter in future.  The WPC’s Principles for Digitalisation should be followed in the meantime, namely:

• employee voice and participation
• flexible and secure job change
• opportunity for progression and growth
• health, safety and well-being
• respecting workers' rights. 

The WPC is grateful to Dr Philippa Collins, Senior Lecturer in Law at the University of Bristol, for her significant contribution to the preparation of this document.

The aim of this document is to provide information and guidance to managers and trade unions regarding the use of algorithms to manage staff in the public sector in Wales. The integration of new technologies into our working practices presents a significant opportunity to improve how we perform our roles and to deliver broader benefits to workforce wellbeing, from service delivery, to policy-making or administration, if this process is undertaken responsibly and fairly. This document sets out how employers and workers can collaborate effectively together to ensure that the Guiding Principles set out below are respected and embedded at every stage of the process.  

This document first sets out the context to the guidance, providing definitions, case studies and an overview of the potential benefits and risks of using algorithmic management systems.  It then sets out guidance, divided into three sections: overarching Guiding Principles that should be followed at every stage of the process, steps to be completed before implementation and requirements for after implementation. 

A full glossary can be found at the end of the document. The key definitions used in this document are as follows: 

‘Algorithmic management system’ refers to any system that uses computational processes to take or support decisions relating to the management of employment or work. An algorithmic management system includes some aspect of automation and may include processes based on machine learning, statistical analysis or artificial intelligence. An algorithmic management system may be implemented to undertake or support one management function, such as recruitment or the organisation of work, or a system may undertake or support a series of management tasks. 

‘Social partners’ refers to representatives of the government, representatives of the employer(s)/management and representatives of the workers/staff, whether at national or organisational level.

How can algorithms be used to manage staff?

An ‘algorithm’ is a term generally associated with computer programming that refers to any rule or instruction or set of rules. In the context of an algorithmic management system, the algorithm is then applied to data about the workforce, the workplace, the work or an individual worker to produce an output. The output might be an instruction, a recommendation to a manager or the triggering of another sequence of algorithmic decision-making. In some cases, an algorithmic management system relies upon artificial intelligence. Artificial Intelligence (AI) refers to the use of digital technology to create systems capable of performing tasks commonly thought to require intelligence. 

Both AI and algorithmic management systems rely on the input and processing of data, often vast amounts of it. Modern technology also allows this data to be increasingly granular, pinpointing a worker’s activities in real-time for example. Data about past events or interactions can also be used to model and predict the likelihood of events in the future. For example, if AI detects patterns of common behaviour or qualities in previous successful hires, these patterns can then be applied to predict which of a pool of candidates is most likely to be successful based on their fit with those patterns.  

AI and algorithmic systems are already being deployed throughout the lifecycle of employment. Recruitment, from sourcing appropriate candidates to “reading” their CVs to conducting interviews and ultimately to ranking candidates based upon their suitability for an organisation, can be mediated or even performed by AI. Once within an organisation, whole aspects of managerial functions can be supported or delegated to algorithmic management systems. For example: 

• Organisation of shift and working patterns 
• Work allocation and team creation 
• Issuing instructions and sequencing tasks across the workforce 
• Determining and setting out productivity or performance expectations 
• Training  
• Monitoring productivity and performance, including triggering capability or disciplinary procedures 
• Issuing disciplinary sanctions  
• Termination of employment.  

AI can also engage in other activities, such as monitoring the health of a workforce or a worker, or engaging in emotion recognition that assesses the individual’s emotional and psychological state.  

Early examples of algorithmic management were often drawn from the platform or gig economy and employers such as Uber or Deliveroo or from technologically advanced companies like Amazon. However, algorithmic management can now be seen in almost every working context in some shape or form.  For example, Home Care Direct is used in Ireland to connect relatives of people requiring care with care workers. In Caerphilly, hand-held devices have been adopted in domiciliary care, which allows actual routes taken and their timings to be recorded. In turn, the schedules are more accurate and improved. Unions have been involved to ensure that staff’s concerns about the potential for intrusive monitoring are resolved.  In Neath Port Talbot, staff and specialist IT contractors co-created systems to automate some of the HR team’s high volume processes. The use of the system reduced time spent doing repetitive tasks and created new job opportunities. 

What does research tell us about the opportunities and risks of using algorithmic management systems?

Opportunities

These examples tell us about some of the advantages of using algorithmic management systems and how public services can grasp the opportunities presented by new technologies to innovate, to respond to changing needs and to seek efficiencies in order to continue to provide a sustainable service to the public. This list is not exhaustive.  Current research indicates that opportunities for our public services include:

1. Increased efficiency and productivity: Time can be freed up from repetitive tasks, tasks can be planned and allocated more accurately and on the basis of better quality information, and processes can be made more efficient. 
2. Better data-driven decision-making:
   1. Reduction of biases in decision-making by relying on data rather than subjective human judgment, meaning potentially fairer evaluations and promotions based on performance metrics
   2. Algorithmic analysis of trends and patterns can help predict future needs, such as staffing requirements, and make proactive adjustments.
3. Efficiencies in processes and teams can contribute to improved service delivery and experience for members of the public, for example, by better matching staffing requirements to predicted levels of need from the public.

Risks

Research also highlights several concerns regarding the implementation of algorithmic management systems. These risks include: 

1. Difficulties creating genuine transparency around the build/procurement, adoption, implementation and oversight of algorithmic management systems. This is because of the technical complexity of the systems and a lack of detailed understanding and expertise regarding the systems amongst the wider workforce. The challenge is further compounded by the commercial sensitivity of some of the systems themselves, particularly where they are procured from third party companies.
2. Concerns regarding the right to respect for private life and data protection and the right to non-discrimination have been highlighted. These concerns arise because of the data collection and processing that underpins algorithmic management systems, the potential to blur boundaries between on-duty and off-duty time, and the potential for such systems to replicate, or even create, patterns of unfairness or unequal treatment of individuals. 
3. The cumulative effects of algorithmic management, where not implemented responsibly, can damage the health, safety and well-being of workers: expectations and the pace of work intensify and the experience of being monitored closely has a negative impact on mental health. 
4. Where human decision-making is supported by algorithmic management systems, we should also be mindful of the risk of a decline in the place of human judgment, human expertise and human connection in our working relationships.

Case studies 

Employers and unions have succeeded in negotiating agreements on the use of AI and algorithmic management systems.  Some good examples come from neighbouring European countries.   

Norway: A national agreement in Norway has been negotiated between the main trade union and employer bodies, the Confederation of Norwegian Enterprise and the Norwegian Confederation of Trade Unions.  The agreement was achieved in 2018 and addresses the opportunities and challenges of using AI at the workplace, emphasising the need to comply with existing legal rules. The agreement states that employee privacy and dignity are paramount and requires involvement of employee representatives, as well as efforts to prevent bias. The agreement requires discussions with trade union representatives in the event of important changes. Under the agreement, employers must keep employees informed through shop stewards on plans and decisions regarding control measures, which can be based on technological, financial, safety, and health considerations.  

IBM in Germany: In July 2020, IBM, the multinational technology firm, struck an agreement with its trade union on the introduction and use of artificial intelligence systems within the company.  The agreement includes provisions relating to:  

• transparency of AI systems 
• explainability of their decisions 
• ensuring that humans are responsible for decision-making 
• compliance with non-discrimination and fairness legislation 
• conducting risk assessments.  

These examples show that employers and worker representatives and unions can work together to reach agreements that permit employers to use new technologies, agree common principles that guide their use and recognise the need for safeguards to protect the workforce.  

The WPC understands the need to balance the opportunities of adopting new technologies in our working practices with the risks and challenges that we may face in doing so. The WPC adopts the following set of general principles that should be followed by social partners throughout negotiation, decision-making, deployment and ongoing evaluation of any algorithmic management systems. 

1. Social partnership. This aligns with the provisions of the Social Partnership and Public Procurement Act 2023 and long-established ways of working in Wales.

2. The preservation and prioritisation of human oversight and human interaction. There must be human oversight of all strategic decisions taken about the use of algorithmic management systems (a “human in command”) and clear lines of responsibility to management drawn for any decision taken. Human interaction should also be required in the day-to-day running and decision-making of a system (a “human in the loop”).   

3. Adherence to the Welsh Government’s definition of Fair Work, especially in relation to equality.  Fair Work is the presence of observable conditions at work which means workers are fairly rewarded, heard and represented, secure and able to progress in a healthy, inclusive working environment where rights are respected.

4. A commitment to building capability regarding algorithmic management systems across the workforce and at all levels. 

5. A commitment to the protection of jobs, the creation of jobs, and investment in the workforce.

In this section, the guidance focuses on activities that should be undertaken before a system is implemented. The aim of these activities is to foster transparency and communication from the very first stages of the process and to give the best possible chance of ensuring that the Guiding Principles set out above are respected and reflected in the choices ultimately made in the public sector. 

Audit of existing systems 

The first stage of a conversation between social partners about algorithmic management should be a full audit of where algorithmic systems are already in use. 

Public sector bodies are encouraged to adopt the Algorithmic Transparency Recording Standard (ATRS), which provides a standardised way of recording and sharing information about the algorithmic tools they use in an open, understandable, easily accessible, and free format. It records information about: 

• how the organisation’s algorithmic tool works 
• how it’s incorporated into their decision-making process 
• what problem they’re aiming to solve by using the tool 
• their justification or rationale for using it 
• who owns and has responsibility for the tool
• the results of risk assessment processes that have been undertaken.

The outcome of this audit should be a comprehensive record - a register - of all the algorithmic management systems being used by an organisation. This record should be updated with any changes and should be made freely available to all members of staff.  

This audit process may disclose the existence of systems about which the social partners have not engaged in any dialogue and about which there is not clear agreement. In this case, the social partners should complete the steps set out herein, including making any adjustments to the use of established systems as necessary, and ensure that the guiding principles set out above are respected in the use of current systems. 

Build or buy? Before in-house development and / or procurement begins

Organisations should undertake a readiness assessment before making a decision to begin either a procurement (the purchase of a system from a supplier) or build (the development of a system within the organisation) process relating to an algorithmic management system. Organisations are encouraged to engage with the Office for AI’s Guidelines for AI Procurement and the Data Ethics Framework, provided by the Central Digital and Data Office, as useful guides to this process. 

We would encourage public sector organisation's to utilise all available guidance on the cyber security of AI from the National Cyber Security Centre (NCSC), particularly their 'Guidelines for secure AI system development'. A key priority of the Welsh Government’s Cyber Action Plan for Wales is to protect public services. Welsh Government convenes a Cyber Programme Board which comprises relevant Welsh Government policy leads, the Centre for Digital Public Services (CDPS) and Chief Digital Officers (CDOs) for the Welsh Government, Local Government and Health and Care

In the earliest stages of considering the procurement or building of an algorithmic management system and in the context of wider multidisciplinary engagement, social partners should engage with questions including: 

• is there evidence of a need for a technological solution for the problem? Does technology present an opportunity for innovation in the way we work? 
• is the task one that is suitable for being supported or assisted by the use of an algorithmic management system? 
• is the organisation as a whole ready to undertake the change proposed? 

This last question should be considered from a variety of angles, including IT infrastructure, data infrastructure and workforce capability and culture. 

There may be some uses of algorithmic management systems that are considered inappropriate, unproven, a disproportionate infringement upon the rights of individuals or creating risks that cannot be adequately mitigated. Social partners may seek to agree a list of such use cases in advance, as well as adding to that list over time as technologies develop and risk assessments are conducted. 

An indicative list of such technologies include those relying on invasive monitoring, emotion recognition technologies or biometric surveillance; any systems that automate any part of disciplinary or capability processes or any other decision of equivalent detriment; systems that seek to predict or influence workers’ enforcement of their legal rights, and systems that seek to seek to influence an individual’s behaviour or place undue pressure on an individual to act to their own detriment or to put their health or wellbeing at risk.

Due diligence and consultation during procurement and / or build

Organisations will have extensive expertise in relation to the procurement or building of systems for use within the public sector. As with the other stages of the process mentioned within this document, social partners should work together to ensure that the Guiding Principles highlighted above are respected where the system will perform management functions. Social partners should also be open to opportunities beyond those identified at the first stage for improving the processes and systems in place. 

Organisations may find that the use of the Algorithmic Transparency Recording Standard (ATRS) would be useful in structuring dialogue with unions and the wider workforce regarding the procurement or build of new algorithmic management systems. Consultation during the drafting of a Standard would allow transparency about decision-making at an early point in the process and a collaborative approach to assessing and addressing any risks that are identified. 

In particular and again in the context of wider multidisciplinary engagement, social partners should be encouraged to prepare a Data Protection Impact Assessment, an Equality Impact Assessment and an Algorithmic Impact Assessment and share the results of these assessments in the final Standard relating to a particular system. Helpful resources for social partners have been included in the Further Resources section below. These impact assessments will enable social partners to identify a range of risks and to consider how these risks should be addressed. Progress towards adoption and implementation should only continue if any risks can be removed or managed to such a degree that the social partners consider them acceptable.

What risks should be considered?

Due consideration should be given to the Guiding Principles set out in this document, where they are not covered by these risk assessment processes. The regulations and rights that social partners should consider for the purpose of a risk assessment include: 

• the Health and Safety at Work etc. Act 1974, 
• the Human Rights Act 1998, 
• the Equality Act 2010, 
• the Data Protection Act 2018 and the UK General Data Protection Regulation, 
• unfair dismissal law, 
• working time and minimum rage regulations, 
• trade union regulations and 
• individuals’ rights under their contract. 

In the process of drafting a risk assessment, the social partners (including unions, the employer and workforce managers) should also consider broader concerns, such as the possibility of work intensification, deskilling of the workforce, the loss of expertise and the sidelining of human judgment. Systems should be procured or built in a way that minimises any such risks identified. 

Where a system is being procured, consideration should be given to the end of a contract with a supplier and the sustainability of operations beyond the duration of a particular contract. 

The role of worker representatives in procurement and/or build

Worker representatives are key stakeholders in the procurement or building process. They should be encouraged to provide their expertise and feedback and contribute to ongoing two-way communications. Worker representatives have an important role to play in enabling the wider workforce to understand and offer feedback in relation to any planned changes, managing the expectations of the wider workforce and facilitating and supporting implementation processes if a new system is adopted. Managers responsible for a procurement or build process should put forward any queries raised by worker representatives to a system’s vendor/builder or facilitate direct communication between their respective contacts. 

It must be recognised that worker representatives may require external advice or training to ensure they are adequately skilled to participate effectively in the procurement or build processes associated with an algorithmic management system. Consideration should be given to the time and funding required in order to achieve this. 

Staff training

One aspect of implementation that should be discussed well in advance of implementation is staff training. Social partners should work together to identify where training needs may arise, particularly for anyone whose decisions will be supported or informed by an algorithmic management system. Managers particularly should receive thorough training on the appropriate uses, the functions and functioning of a system, as well as ensuring they understand the limits of any systems that are introduced. Social partners might consider that more general training for the whole team is desirable to achieve wider transparency and understanding regarding the relevant systems.

As mentioned above, the Guiding Principles set out in this document should be reflected throughout the lifecycle of a system. The need for co-governance and responsible use does not end once the system is first adopted and implemented. This section sets out key monitoring and governance measures that employers and representatives should agree on and comply with after initial implementation.  

Ongoing monitoring of systems

Algorithmic management systems must be subject to effective monitoring. Designing a system of monitoring that works for both the organisation and its staff is one part of the process that should be undertaken before implementation, at the due diligence stage. Points of ongoing evaluation of the system could include:

• whether the system is achieving its intended purpose and its effects on the work and working conditions within the organisation
• whether any agreed risk mitigation measures are in place and whether they are effective
• whether any unforeseen risks or impacts have emerged that require mitigation or other action 
• whether technical assessments of functioning and bias audits are producing satisfactory results
• whether agreed principles between the social partners are being upheld by the system and its use in practice. 

Feedback from workers and workplace representatives should be regularly sought, collated and used to shape the ongoing use of any system. Workers should be able to raise concerns without being concerned about any negative personal impact of doing so and, where necessary, concerns about algorithmic management systems should be escalated quickly either within or beyond the organisation. 

Workplace representatives should have periodic access to the system to maintain transparency and to monitor how organisations are using the system, including examining decisions made or supported by the system. 

Where desirable and feasible, organisations should discuss development opportunities to improve the system in response to concerns or impacts highlighted by the post-implementation monitoring with suppliers. Organisations must remain open to pausing or halting the use of the system entirely where risks cannot be managed adequately. 

Human-in-command and human-in-the-loop

Human oversight of algorithmic systems is an important principle.  A senior manager should have designated and meaningful responsibility for any algorithmic management system and decisions made on the basis of its outputs.  Any designated manager should have (at least) adequate training and understanding of the system; an awareness of the tendency towards automation bias; the ability to interpret outputs correctly; the capacity to overrule, disregard or not rely on the system, and the capacity to intervene in its operation to end processing in a safe way. The designated manager should be in regular contact with workplace representatives. 

It should also be clear who within or outside the organisation has responsibility for maintaining the system and who has the ability to change and modify the system. 

In addition to general oversight and accountability, the system should be designed and implemented in a manner that ensures that decisions that have significant effects for an individual are subject to active human involvement. This is often known as keeping a “human in the loop”. 

Safeguarding individuals’ rights 

Organisations must ensure that individuals’ rights are protected throughout the lifecycle of a system. This includes the full range of rights mentioned above in the due diligence section, but particular attention should be paid to the following: 

• the right to receive information about data processing, particularly regarding the use of automated decision-making by an organisation
• the right to request access to the individual’s own data held by the organisation, as well as to rectification
• the right to request the erasure of data which is inadequate, irrelevant, or no longer relevant
• the right to request a review of any automated decision made about an individual. 

Social partners may also wish to agree additional rights and safeguards for individuals in the context of decision-making using algorithmic systems. For example:

• the right to raise concerns about a system without suffering any detriment in response. 
• the right to not suffer any detriment as a result of relying upon an algorithmic management system. 
• the right to receive a personalised explanation of any significant decision made about an individual that relied upon an algorithmic management system in advance of that decision taking effect. 

Information sharing and further updates 

Social partners should share information about the system and its use as much as is reasonably possible to do so. Where the organisation proposes to make changes to a system or updates are necessary, they should ensure that the Guiding Principles set out above are adhered to and commence consultation with trade unions and workplace representatives as soon as possible. 

• The Central Digital & Data Office Data Ethics Framework. 
• The Department for Science, Innovation and Technology Responsible AI Toolkit, including the Algorithmic Transparency Recording Standard Hub. 
• The Department for Science, Innovation and Technology, Office for Artificial Intelligence and Centre for Data Ethics and Innovation Guide to using artificial intelligence in the public sector. 
• The Department for Science, Innovation and Technology, Office for Artificial Intelligence, Department for Digital, Culture, Media & Sport and Department for Business, Energy & Industrial Strategy Guidelines for AI Procurement.
• The Equality and Human Rights Commission Guidance on Artificial intelligence: Meeting the Public Sector Equality Duty and Case Studies of good practice by local authorities. 
• The Information Commissioner’s Office Guidance on Data Protection Impact Assessments, Best Practice for data protection-compliant AI and Explaining decisions made with AI. 
• The Institute for the Future of Work’s Guidance on the Good Work Algorithmic Impact Assessment and a Report on the Algorithmic Impact Assessment’s developments and limitations. 
• The National Cyber Security Centre AI and cyber security: what you need to know.
• The National Cyber Security Centre guidelines for secure AI system development. 
• The Parliamentary Office of Science and Technology Explainer on Artificial intelligence. 
• The Welsh Government Cyber action plan for Wales. 
• The Welsh Government Guide to Fair Work. 
• The Welsh Government Report on The Future of Work: the impact of innovative technology on the workforce.
• The Workforce Partnership Council Agreement on Partnership and Managing Change, including the Principles for Digitalisation. 

• Adams-Prassl, J. (2019) ‘What if your boss was an algorithm? Economic Incentives, Legal Challenges, and the Rise of Artificial Intelligence at Work’, Comparative Labor Law & Policy Journal 41, 123-146. 
• Brunnerová, S. et al (2024) Collective Bargaining Practices on AI and Algorithmic Management in European services sectors (Friedrich Ebert Stiftung Competence Centre on the Future of Work).
• Collins, P. (2024) ‘Managing technology that manages people: Regulatory Strategies for the UK’,University of Bristol Law School Working Paper, 1-15.
• Collins, P. and Atkinson, J. (2023) ‘Worker voice and algorithmic management in post-Brexit Britain’, Transfer: European Review of Labour and Research, 29, 37-52. 
• De Stefano, V. and Taes, S. (2021) Algorithmic management and collective bargaining (ETUI Foresight Brief).
• Kelly-Lyth, A. and Thomas, A. (2023) ‘Algorithmic management: Assessing the impacts of AI at work’, European Labour Law Journal, 14, 230-252. 
• Molina, O. et al (2023) ‘It takes two to code: a comparative analysis of collective bargaining and artificial intelligence’, Transfer: European Review of Labour and Research, 29, 87-104.
• Rolf, S. (2024) AI and Algorithmic Management in European Services Sectors: Prevalence, functions and a guide for negotiators  (Friedrich Ebert Stiftung Competence Centre on the Future of Work).
• Wood, A. J. (2021) Algorithmic Management: Consequences for Work Organisation and Working Conditions (JRC Working Papers Series on Labour, Education and Technology, European Commission, JRC124874). 

These definitions are drawn from Artificial Intelligence: An Explainer (2023, POSTbrief 57) provided by the UK Parliamentary Office of Science and Technology and The Alan Turing Institute’s Data Science and AI Glossary that is available online. 

Algorithm

A set of instructions used to perform tasks (such as calculations and data analysis) usually using a computer or another smart device.

Algorithmic management system

Any system that uses computational processes to take or support decisions relating to the management of employment or work. An algorithmic management system includes some aspect of automation and may include processes based on machine learning, statistical analysis or artificial intelligence. An algorithmic management system may be implemented to undertake or support one management function, such as recruitment or the organisation of work, or a system may undertake or support a series of management tasks. 

Generative AI

An AI model that generates text, images, audio, video or other media in response to user prompts. It uses machine learning techniques to create new data that has similar characteristics to the data it was trained on. Generative AI applications include chatbots, photo and video filters, and virtual assistants.

Human-in-command

A situation where a system is embedded within a clear system of human oversight, responsibility, accountability and governance measures. 

Human-in-the-loop

A system comprising a human and an artificial intelligence component, in which the human can intervene in some significant way, e.g. by training, tuning or testing the system’s algorithm so that it produces more useful results. It is a way of combining human and machine intelligence, helping to make up for the shortcomings of both. Having a human-in-the-loop prevents fully automated decision-making. 

Machine learning

A field of artificial intelligence involving computer algorithms that can ‘learn’ by finding patterns in sample data. The algorithms then typically apply these findings to new data to make predictions or provide other useful outputs, such as translating text or guiding a robot in a new setting. Medicine is one area of promise: machine learning algorithms can identify tumours in scans, for example, which doctors might have missed.

Social partners

Representatives of the government, representatives of the employer(s)/management and representatives of the workers/staff, whether at national or organisational level.