9 Trials and testing

Zoom in on image

9.1 The current situation

Innovation can be complex and entail risk. Trials and experimentation can be expedient approaches to managing these aspects of innovation.

Trials are often carried out within a defined time frame and a delimited area, where those involved are able to test the effect of new solutions in a realistic environment. In this way, the costs and risks can be reduced compared with a situation where all municipalities or public agencies have to do the same. Using control groups and/or impact evaluations makes it possible to identify which solutions have the desired effect and which should be discontinued. In larger studies, it will be necessary to include researchers in this work. This will enable everyone involved to generate knowledge and learning that others can use, and form the basis for assessing whether or not the solutions should be implemented and spread on a larger scale.

Different concepts and methods exist for testing new solutions. In the Nordic context, the terms trial and experimentation are often used interchangeably. In this white paper, the term trial is primarily used about testing new solutions.

Trials have be conducted for a long time in Norway, but no comprehensive overview of trials, test schemes or lab activities in the public sector is currently available. Concepts and schemes are also under development, both in Norway and the rest of the world.

9.1.1 Legal framework for testing new solutions

All types of trials and testing of new solutions established in the public sector are subject to the same laws and regulations that regulate public bodies. Some trials are conducted pursuant to relevant statutory provisions that regulate trials since they, on certain conditions, provide for the possibility of granting exemptions from applicable laws or regulations. Some statutory provisions allow for dispensations from other requirements in order to be able to test new things.

In many cases, it is possible to try new solutions without needing an exemption from laws or regulations, for example testing new ways of organising a public body’s internal work or new digital solutions that make case processing more efficient. In the cooperative development program Sammen om en bedre kommune (‘Together for a better municipality’), municipalities tested the use of alternative rota systems to increase the proportion of full-time positions. The percentage of a full-time position was an indicator measured before, during and at the end of the program. The municipalities reported which solutions they tested and the results they achieved. The testing did not require a legal exemption, although some alternative rota systems required approval from the social partners.

Regulatory sandboxes

The term regulatory sandbox is used to describe ways of testing new technologies, statutory regulations and business models within a given framework. The concept regulatory sandbox is especially common in the financial sector, where selected institutions have been given an opportunity to test given products, technologies or services on a limited number of customers and for a limited period under close follow-up by the supervisory authority. It can also concern more extensive measures carried out under close follow-up and guidance from, for example, the supervisory authorities in a specific area. The Financial Supervisory Authority of Norway accepted applications for a regulatory sandbox for financial technology from December 2019 (Box 9.1).

Exemption clauses

Another way of testing new solutions is to use exemption provisions in applicable laws relating to, for example, requirements concerning organisations or their activities. The Act relating to the Testing of Autonomous Vehicles is an example of a law that provides for the possibility of testing new solutions. Another example is Section 25 of the new Harbour and Fairways Act, which provides for new technology such as self-driving, autonomous ships. The section opens for the possibility of being granted permission for autonomous coastal voyages and exemption from the pilot requirement. We have long experience of codified exemptions of this type from other requirements in a law. It is a useful means of enabling testing that avoids having to go via the Pilot Schemes in Public Administration Act or trial provisions in special laws.

Some special laws have separate provisions relating to trials. One example is the Education Act, which permits deviations from the rules for a fixed-term educational or organisation trial. The Election Act contains a special provision on trials when holding elections. The Child Welfare Act contains a trial provision that allows exemptions from the statutory duty of secrecy in several special laws.

In some cases, it may be relevant to amend laws or regulations to include a special trial clause in the sector regulations based on specific cases and expert assessments, in a manner that makes trials possible.

The Pilot Schemes in Public Administration Act

The most important act relating to formal trials is the Pilot Schemes in Public Administration Act (Act No 87 of 27 June 1992). The purpose of the Act is, through pilot schemes, to develop functional and efficient forms of organisation and operation in the public administration, and an appropriate division of duties between public administrative bodies and between public administrative levels. The Act is intended to improve the public provision of services to citizens, and ensure the best possible use of resources and good democratic forms of government.

It allows the state, county and municipal administrations to conduct pilot schemes upon application. It allows them to depart from the main rules for how public agencies are required to organise their activity and perform their duties. It also allows them to depart from the provisions concerning the division of duties between these bodies.

The pilots must be appropriate and professionally well founded. They apply for a period of up to four years, although it is possible to extend the period. The pilots must aim to achieve the purposes set out in Section 1 of the Act, which states that the aim is to test whether the scheme is sufficiently expedient that a permanent scheme is desirable. The pilot must also include a good evaluation scheme.

Pilot schemes will not be approved if they entail departing from the fundamental provisions of the Local Government Act with respect to the organisation of municipalities and county authorities or rules of procedure, or that would otherwise entail restricting the rights or extending the duties of any individual. Nor will pilot schemes with substantive new content be approved pursuant to the Pilot Schemes in Public Administration Act, i.e. pilots that entail an out and out dispensation from the law.

More detailed rules must be adopted for how the individual pilot scheme is to be conducted. The municipal council must adopt regulations for pilot schemes in the local government sector, which must then be approved by the Ministry.

9.1.2 Examples and experience

Conducting trials is not a new endeavour in Norway or the other Nordic countries. Trials have been carried out in the local government sector in areas including direct elections of mayors, granting extended powers to the mayor, voting rights for 16-year-olds and the political organisation of municipalities. The trials relating to direct election of mayors and voting rights for 16-year-olds were discontinued and not proposed for continuation as permanent solutions. The trial concerning granting extended powers to mayors resulted in new provisions being introduced in the Local Government Act of 2018. The trial concerning the political organisation of municipalities was one of the main reasons why the Local Government Act of 1992 introduced the possibility of municipalities using a parliamentary model rather than the alderman model.

The trials relating to elections that have been carried out in recent elections have formed an important basis for considering whether the solutions should be introduced across the country (Box 9.2).

The Ministry of Local Government and Modernisation started a trial in 2019 of anonymous job applications in the state sector. The trial aims to document whether the use of anonymous applications leads to more immigrants being invited to a job interview and offered a position with a public sector employer. All ministries have selected one subordinate agency to participate in the trial. The trial started in spring 2019 and was concluded in 2020, with a subsequent evaluation. The trial forms part of the Government’s integration reform. The purpose of the integration reform is to increase immigrants’ participation in small and large communities in society. The most important aim of the reform is to help more people into employment.

Experimental Finland

In 2016, the Finnish government established the program Experimental Finland, with the objective of promoting an experimental culture in the public administration. The initiative, based at the Office of the Prime Minister, comprised three priority areas:

• to express a clear political wish for a more experimental public sector

• to organise training in experimental work methods

• to conduct experiments in strategic areas for the Government

Many different experiments were conducted and they resulted, among other things, in a guide on how to support projects that conduct trials. The different parts of the program have now been continued in another form, now that experimentation has come to form a larger part of the public administration’s work methods.

Free commune experiment in the Nordic countries

Since the mid-1980s, free commune experiments have been conducted a total of six times in Sweden, Denmark, Norway and Finland.1 The term ‘free commune’ has been used to denote municipalities that are granted exemption from selected laws and regulations through a special legal provision. The free commune experiments have been based on the premise that the generation, testing and diffusion of innovations take place at the intersection between state governance and local autonomy. In contrast to most other development programs that allow for trial and error, the free commune program includes legal procedures for being granted exemption from laws that regulate local services.

In Norway, the Storting decided as early as 1986 to carry out a free commune experiment where selected municipalities were able to test new ways of organising their services. By January 1991, a total of 63 trials had been approved. The scope of the trials varied and some municipalities conducted more than one trial.

In 1989, Grimstad was granted free commune status to test coordinated management of social welfare measures, national insurance and labour market measures for young people. Løten and 12 other municipalities later tested similar schemes. These were also referred to as the coordination trials. Lessons learned from these municipalities were used when designing the Norwegian Labour and Welfare Service (NAV) Reform.

Experiences from the free commune trials were mixed. They triggered engagement and stimulated development and renewal work in general. However, it was difficult to transfer the experiences to reform measures and it was not easy to identify unequivocal causal relationships based on the experience of individual municipalities. The Pilot Schemes in Public Administration Act was in part based on lessons learned from the free commune trials. This became a general, permanent trial law that replaced the Free Commune Act and that also applied to the central government and state-initiated trials in the local government sector.

Denmark has conducted almost continuous free commune trials since 2012. The current trials apply during the period 2016–2020. The trials in Denmark are based on an agreement between the Government and KL.2 Unlike previous free commune experiments, municipalities under the current trial are organised into thematic networks. Eight networks have been selected, each with one overall theme, and 44 municipalities are participating. The idea is that more unified trials comprising more municipalities will make it easier to obtain and systematise knowledge and to follow up and spread the results to all municipalities. It will also be easier to monitor the development within each specific theme.

Lab activities

Many countries have established innovation labs for the public sector in the last decade. A lab is a space, physical or methodological, where testing, experimentation and measurement can take place under controlled conditions. The purpose of the lab activities is to find new ways of developing services, solutions and policies. The labs have used new methods such as design thinking and service design (Chapter 8), trials and big data analysis. Activities in the labs are often based on user needs within a framework that allows freedom to explore or test the solutions before they are potentially taken into use. There are thereby no clear distinctions between trials and lab activities.

In Sweden, the Government has tasked the innovation authority Vinnova with introducing experimental work methods through the policy labs initiative. They define a policy lab as a random or established group of actors with different expertise, who apply and develop innovative methods with a view to changing the regulatory framework. The policy labs use user-centred skills and methods to test, experiment and learn aspects of policy development.3

Policy Lab in the UK is introducing new policy development techniques to agencies across the public sector, helping to design services around people’s experience, and using data analytics and new digital tools. Regulatory change is therefore not at the core of its activities.4 A shared goal in Sweden and the UK is a more transparent, user-centred and agile public sector.

Helsinki Design Lab in Finland promotes what they call strategic design, which helps decision-makers to see the big picture and create whole-system solutions. This is similar to the role of StimuLab in Norway.

Examples of lab activities in Norway include Norwegian Battle Lab and Experimentation in the Armed Forces (Box 9.3), United Future Lab Norway in Ålesund (Box 9.4), StimuLab (Chapter 8), and TRD 3.0 – a collaboration between Trondheim municipality and the Norwegian University of Science and Technology (NTNU) (Box 12.2).

9.2 Assessment of the situation

Many things must be taken into account when planning a trial, in part because the public sector exercises authority, and citizens’ due process protection and equal treatment must be adequately addressed. The Pilot Schemes in Public Administration Act sets out some limitations, for example that trials must not entail a restriction of rights or extension of duties for any individual pursuant to applicable legislation. This is fundamental to maintaining trust in the public authorities. Trust in the public sector will be a key part of experimentation and trials in general. On the one hand, limited trials or experiments are a good way of testing new ideas before they introduced on a larger scale. On the other, too many trials with solutions that turn out to be inexpedient could also weaken trust in the public sector.

When the results of a trial are good, it should be possible to implement the new solutions and share the lessons learned. In cases where a trial does not have the intended effect, the lessons learned should be applied in further work and shared with other actors considering similar trials. To achieve this, experiments and trials must undergo thorough evaluation. Furthermore, potential implementation and diffusion should be facilitated from the offset. There are several examples of successful innovations in the public sector that are not scaled up and diffused. When many trials with positive outcomes are conducted without being implemented and without leading to permanent change, the result can be fatigue in the organisation.

It is important to develop a good system for trials, and a trial must sometimes be allowed to work for a certain period, often several years, in order to be able to conclude on its effect. When a trial has ended, it must also be evaluated. It can sometimes be expedient to attempt to gain experience more quickly than through a formal trial. It can be challenging to identify unequivocal causal connections between models and results. It is therefore not necessarily possible to draw general conclusions based on trials in one or a few municipalities or agencies that are applicable to all similar municipalities or agencies.

In some cases, trials can be so controversial or complicated that it can be difficult to reach agreement between the affected bodies as regards the project, its evaluation and any resulting reforms. It can also be challenging to identify which, if any, laws and regulations must be amended, or whether exemption from legislation is necessary at all. This is exemplified by the experience from the ongoing Danish free commune scheme, where it was concluded that a number of applications for trials would not be possible to follow up.5

9.3 The way forward

The Pilot Schemes in Public Administration Act has been in force for nearly 30 years. However, the Act has been used less frequently in recent years. Some municipalities have applied, but some of the applications have been for trials that do not require exemption from statutory provisions. The Government will consider whether it is necessary to make amendments to the Pilot Schemes in Public Administration Act that could, among other things, improve the possibility of innovation and new ways of thinking in the public sector. In spring 2020, the Ministry of Local Government and Modernisation put out to tender an official study to evaluate the Pilot Schemes in Public Administration Act and assess the potential need for amendments. One of the aims of the amendments must be to contribute to more trials. The study should also assess what freedom of action the Pilot Schemes in Public Administration Act provides and whether potential amendments are relevant for testing new solutions based on artificial intelligence.

A government-initiated measure for more testing of new solutions can call attention to and foster the use of experimentation and trials both with and without legal exemptions. It can help to generate new ideas and mobilisation for the testing of new solutions. At the same time, experience and research show that trials must have a clear and rigorous evaluation system in place in order to be able to transfer potential solutions to others, and that plans for how the lessons learned can be diffused should be in place from the outset. The Government will look at how the public sector can use trials more systematically to test new solutions to major societal challenges.

The Government will establish a regulatory sandbox for data protection under the Data Protection Agency’s area of authority, as described in the strategy for artificial intelligence.6 The Government takes a positive view of establishing regulatory sandboxes in more areas if there is demand for this.

9.4 The Government’s aims

Trials and regulatory sandboxes are examples of ways of testing new solutions in practice.

The Government will:

• look at the need to amend the Pilot Schemes in Public Administration Act in ways that could improve the possibility of innovation and new ways of thinking in the public sector

• assess how the public sector can use trials more systematically to test new solutions to major societal challenges

• establish a regulatory sandbox for data protection under the Data Protection Agency’s area of authority, as described in the strategy for artificial intelligence.