NOU 2023: 19

Learning: Lost in the Shuffle?

Use of pupil and student data to enhance learning

Dokumentet på regjeringen.no

To the Norwegian Ministry of Education and Research

On 10 September 2021, the Norwegian Ministry of Education and Research appointed an Expert Group to assess digital learning analytics in primary and secondary education and training, tertiary vocational education and higher education. The Norwegian Ministry asked the Expert Group to submit a two-part report. The group submitted Part I of its report on 1 June 2022. The Expert Group hereby submits its final report.

Oslo, 6 june 2023

Marte Blikstad-Balas (Chair)

Monica Andreassen

Einar Duenger Bøhn

Ann-Tove Eriksen

Michail Giannakos

Hedda Birgitte Huse

Malcolm Langford

Eirin Oda Lauvset

Per Henning Uppstad

Barbara Wasson

Hilde Hultin (Head of Secretariat)

Øystein Flø Baste

Øyvind Bjerkestrand

Helle Kristin Jensen

Jon Lanestedt

1 Introduction and summary of recommendations

1.1 Introduction

To live is to learn. Both the infant taking in their first impressions, and the 90-year-old signing into online banking are learning, whether they want to or not. In our culture, the education system is our most formal learning arena, and it can – at its best – be one of the most important.

Primary and secondary education and training currently comprises almost 900,000 pupils and apprentices.[1] In addition, we have close to 30,000 students enrolled in vocational colleges and more than 300,000 students taking higher education. In all such education programmes, the goal is for pupils and students to have the best opportunities in society. Norway also has a long tradition of placing education in high regard: Everyone has the same right to quality education, regardless of, e.g., social or ethnic background, gender or where in the country they reside. Schools, training establishments, vocational colleges, universities and university colleges are tasked with forming and educating pupils, apprentices and students so that they develop relevant competence for the present and future. As stated in the objects clause of the Education Act, education shall “open doors to the world and the future”. In order to succeed with the high ambitions for Norwegian education, we must always seek to improve the facilitation of learning. And here the question is how learning analytics may be helpful.

But what is learning analytics? Learning analytics is the analysis of learning for learning. Teachers and instructors have always been concerned with how they can assess the learning process of their pupils and students and in turn use this assessment to adjust and change their instruction. Since teachers have always been making such adjustments, it may feel a bit foreign to discuss this topic using a new term – learning analytics. The term learning analytics is a rapidly emerging phenomenon in education and society. The latter word analytics has a broader meaning than the conventional understanding of analysis. Analytics involves using digital technology to sort, analyse and interpret data to identify new knowledge and gain new insight. Thus, what is novel in learning analytics is that both data and analysis are digital. What is also new that those who collect and process the learning data of pupils and students have other items on the agenda than the teachers and instructors. How we should approach this development in education is one of the great challenges of our time.

There is no longer any question of whether schools, vocational colleges and higher educational institutions should adopt digital technology. They already do – every single day. Nor is there any question of whether we should collect digital data on pupils and students, as this is also done every single day. The big questions we need to ask are therefore not about digitalisation per se, but about what kinds of roles technology can play in learning analytics and which aspects of the tasks of teachers, instructors, school administrators and programme administrators we want technology to support, change or challenge.

We know that there is currently considerable ambiguity surrounding many aspects of digitalisation in general, and particularly with respect to pupil and student data. On the one hand, there is a clear potential in the opportunity to analyse what pupils and students are doing digitally. In a situation where the digital learning platform is a more frequent meeting place than the physical classroom, instructors are curious about how learning analytics can promote student learning. Teachers explore what information they need in order to gain a better understanding of the pupil’s learning process, and what best informs them how to develop their instruction and better adapt it to the pupils. In a digital school, parents expect that they will be able to gain better insight into their children’s academic development and thereby also support their children’s learning to a greater extent. School administrators and study programme administrators are monitoring data to see what can be collected and analysed, and what can provide a basis for better learning and progress and completion.

On the other hand, there are also unresolved questions and concerns. In course evaluations, students question whether educational institutions actually have a legal basis for allowing instructors to have access to data at the time they are signed into the learning platform. In the staff room, teachers discuss what the purchased teaching aid actually measures when the results display “42”. Pupils say they need to know when the computers are tracking what they are working on, and they want the chance to work without being monitored, as well as the opportunity for trial and error without all their mistakes being stored and made visible to others. Parents are concerned about what data commercial actors can receive about their children during the school day. Politicians and school administrators fear that the actual interest of the suppliers is not to earn revenue from licenses, but to harvest valuable and abundant data on natural persons.

In other words, learning analytics offers opportunities, but also pose considerable challenges. The desired digitalisation of schools and society is primarily what has initiated these processes, and the result to date has been the relatively free rein of market-driven technology development, thus allowing the principle of the right of the strongest to prevail. Society and education have therefore been virtually unprepared for the forms and paths that the functionalities of learning analytics have taken. The situation to date is a muddled picture of all aspects of learning analytics. In the public debate, digitalisation is often referred to as an ecosystem and thus compared to processes in nature that interact in balance. However, one may ask whether the rapid pace of digitalisation has more in common with an impact event than an ecosystem in balance.

In this context, we must emphasise that this perspective on digitalisation in education is not a romantic utopia of wanting to return to nature and get away from technology – on the contrary. In his essay Lyckad skövling i ny natur[2] [Successful Destruction in New Nature], entomologist and ecologist Fredrik Sjöberg describes the diversity of insects found in an old shooting range – destroyed nature – and reflects on how biodiversity in many urban environments surpasses that found in nature. This is also how we can look back on the 40-year long digitalisation process. Our ambition is for this report to be a first step towards a balanced ecosystem for the aspect of digitalisation concerning learning analytics, where learning and data protection are the keywords for cultivating a system that benefits human beings.

In the Norwegian education system, we have many digital tools with functionalities for learning analytics, but little systematic knowledge about the extent to which learning analyses actually takes place. In other words: We do not know whether the analysis of learning actually has consequences for continued learning. It is only when this cycle leads to a change in the direction of improving learning that it can be considered learning analyses. When the cycle of learning is not completed, learning becomes lost in the shuffle. In keeping with the ecology metaphor: this report will discuss the need to establish conditions for the learning habitat in a digital age.

Using pupil and student data in an attempt to enhance learning will always be associated with risk and uncertainty. When we digitise the data used to enhance learning, this brings about new and different types of risk. New forms of artificial intelligence are entering the learning analytics cycle. New types of data will emerge, yet the quality of each functionality included in the learning resources is nevertheless determined by human judgement.

An important point for the Expert Group is that we cannot view learning analytics solely as a technological phenomenon. There is not, nor can there ever be, an obvious, static path from the collected data to fulfilling the ambition of enhanced learning. This report identifies a number of areas where new challenges arise when learning analytics are performed. In many ways, the discussions in the report emphasise that the human aspects of using technology are always the most decisive. At the same time, there is a considerable risk associated with failing to consider the opportunities that learning analytics offer in terms of promoting learning for current and future pupils and students. The Expert Group has therefore placed considerable focus on the possibilities of learning analytics in its investigation work.

Is there a need for a new concept – learning analytics – in Norwegian education? The Expert Group has asked itself this question in its work on the report. The question is timely as many concepts have been introduced since the turn of the millennium with great fanfare by various actors seeking to change the way education is implemented, but which have quickly been disposed of at the education sector’s waste depot. Time and further academic debate will eventually show what role the concept of learning analytics can play.

However, an important finding for the Expert Group is that few people in the education sector have a clear understanding of what learning analytics entails. It is not just about the concept itself being unclear, but also about understanding how the collection, analysis and representation of data is part of – or can be included – in promoting learning. If so, this is an eye-opener, given that it is such perspectives that the education sector should be best equipped to address. An uncomfortable, inaccessible concept like learning analytics is perhaps what we need to shed light on how complex and demanding the process towards supporting learning is.

1.2 Summary of the Expert Group’s recommendations

The Expert Group has given weight to developing recommendations and proposals that contribute to a learning analytics that is secure and sound, with a clear pedagogical purpose. One of the main objectives of the recommendations is to take clear steps to strengthen trust in the safeguarding of privacy throughout the educational pathway. This would reduce the risk involved in using new technology in education. At the same time, there is also a risk of missing out on new opportunities by not addressing the question of how technology can be used to enhance learning. The Expert Group will therefore facilitate the exploration and development of good pedagogical practices where learning analytics is included, within secure frameworks. Such practices must be based on discussions among education professionals about the pedagogical purpose of learning analytics should be, and how learning analytics can affect learning processes, teaching situations and roles in education. The Expert Group’s recommendations aim to set the direction for such discussions and how they can contribute to further development of practice.

The Expert Group presents four main recommendations to support good and justifiable learning analytics. The recommendations address different levels of the education sector but should nevertheless be viewed in context. This is first and foremost because navigating an educational pathway should entail a certain degree of coherence and predictability, but also because the recommendations partly interact and complement one another.

The Expert Group recommends clarifying the legal basis for learning analytics in primary and secondary education and training, higher education and tertiary vocational education. The purpose of this recommendation is to clarify when the processing of personal data in learning analytics is lawful and to ensure better predictability.

The Expert Group recommends developing a data protection code of conduct in primary and secondary education and training. The purpose of this recommendation is to strengthen pupils’ and students’ data protection and facilitate good data protection practices, increased awareness and enhanced competence regarding data protection.

The Expert Group recommends establishing frameworks for good learning analytics in primary and secondary education and training. The purpose of this recommendation is to strengthen the free choice of pupils and teachers and to provide a better basis for pedagogical decisions regarding learning analytics to enhance learning.

The Expert Group recommends developing broad guidelines for good and justifiable learning analytics in higher education and tertiary vocational education. The purpose of this recommendation is to facilitate good data protection practices and justifiable learning analytics that promote student learning and increase the quality of education.

1.2.1 Legal basis for the processing of personal data in learning analytics

Primary and secondary education and training

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in primary and secondary education and training. The proposal is based on the general scheme in the proposal for a new Education Act (Prop. 57 L (2022–2023)). The provision will be added to the Education Act and in the corresponding provision of the Independent Schools Act:

The Expert Group proposes including a new paragraph in section 25-1 of the Education Act on the processing of personal data in learning analytics and the tasks in the Act where such processing will be necessary. Proposed new paragraph:

“Municipalities, county authorities and training establishments may process personal data about pupils and apprentices by means of machine analysis and alignment where this is ethically and pedagogically sound and necessary to perform tasks and duties in the Act and regulations pursuant to the Act. Examples of such tasks and duties may be to adapt the instruction, the work on quality development in section 17-12 and formative assessment in section 3-10 of the Regulations pursuant to the Education Act. The degree of personal identification shall not be greater than necessary for the purpose in question.”

Higher education

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in higher education. The provisions shall be inserted in the Universities and University Colleges Act and the Regulations pursuant to the Act:

The Expert Group proposes inserting a new paragraph in section 4-15 of the Universities and University Colleges Act on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:

“The educational institution may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties include quality assurance work and the responsibility to ensure that instruction is provided in accordance with recognised ethical and pedagogical principles, cf. section 1-5.”

The Expert Group proposes specifying the provisions on quality assurance work in section 4-1 of the Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:

“The institutions may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”

Tertiary vocational education

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in tertiary vocational education. The provisions shall be inserted in the Vocational Education Act and the Regulations pursuant to the Act:

The Expert Group proposes inserting a new paragraph in section 4 of the Vocational Education Regulations on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:

“The vocational colleges may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties may be quality assurance work and having learning and instruction methods that are suitable for the students to achieve the learning outcomes, cf. section 2-1 of the Vocational Education Academic Supervision Regulations.”

The Expert Group proposes specifying the provisions on quality assurance work in section 4-1, third paragraph of the Vocational Education Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:

“The vocational colleges may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”

1.2.2 Data protection code of conduct in primary and secondary education and training (School Code of Conduct)

The Expert Group recommends that, in cooperation with the sector, a code of conduct should be drawn up to safeguard data protection in schools. At a minimum, the School Code of Conduct should include the following:

the development and administration of specific data protection requirements in resources that have functionality for learning analytics

the preparation and administration of guidance materials for school owners, school administrators, teachers, pupils, parents, developers and suppliers

the preparation and administration of national data protection impact assessments for resources that have functionality for learning analytics

the facilitation of competence development on and exchange of experiences from data protection work in schools

The Expert Group recommends that, as part of the School Code of Conduct, concrete, verifiable data protection requirements should be drawn up for resources that have functionality for learning analytics. The requirements in the School Code of Conduct must be identical for both licensed and free resources. At a minimum, the requirements should be aimed at reducing the risks associated with the following four data protection principles:

fairness

transparency

data minimisation

accuracy

The Expert Group recommends that a national actor, as part of the School Code of Conduct, prepare and administer overall risk analyses, data protection impact assessments (DPIAs) and data processor agreements for resources that have functionality for learning analytics. The Expert Group emphasises that the responsibility for processing lies with the school owners. As the data protection situation in schools is precarious, we recommend as a first step to make arrangements for school owners to share their analyses and assessments with one another.

The Expert Group recommends that, as part of the School Code of Conduct, arrangements be made for developing competence on and exchanging experiences related to data protection efforts. It would be advantageous if an already existing relevant network can carry out this task.

The Expert Group recommends that the administration model for the School Code of Conduct include a steering group with representatives from key actors and user groups.

The Expert Group recommends that the School Code of Conduct be based on relevant measures and guidelines that are already firmly rooted in the school sector, but it emphasises that the code of conduct must take a comprehensive approach to data protection in schools.

The Expert Group recommends linking the School Code of Conduct with a national service catalogue for digital learning resources. This link must be in line with the procurement legislation.

The Expert Group recommends that the continued work on the School Code of Conduct:

be developed with a realistic level of ambition and include thorough investigations and evaluations along the way

be aligned with existing learning technology standardisation efforts and privacy by design

includes all processing of personal data in schools, including processing that does not have learning analytics as a purpose

involves pupils and parents, where relevant

1.2.3 Frameworks for good learning analytics in primary and secondary education and training

The Expert Group recommends that national authorities facilitate usage-based pricing models for digital teaching aids, and that a study be initiated on how trials involving usage-based pricing models can be scaled up.

The Expert Group recommends that the national service catalogue for digital learning resources supports good learning analytics in schools.

The Expert Group recommends that centrally defined quality criteria be developed for resources that have functionality for learning analytics. It is teachers, school administrators, school owners and developers who will be using these quality criteria. The criteria can be based on existing guidelines for quality assessment of teaching aids.

The Expert Group recommends that suppliers and developers cooperate on the use and further development of the quality criteria so that they provide guidance for product development.

The Expert Group recommends that suppliers be required to make available user-oriented information that justifies and explains how the resources work. Suppliers must also be able to document that the technical specifications in the resources correspond to the user-oriented information.

The Expert Group recommends a grant scheme for purchasing and developing digital teaching aids that have functionality for learning analytics. The grant scheme should stimulate innovative learning analytics functionality and artificial intelligence (AI), and must set requirements for data protection and responsible use of AI. Resources must also be required to comply with centrally defined quality criteria.

The Expert Group recommends that funding be announced for innovation, research and development pertaining to digital learning resources that have functionality for learning analytics and adaptivity, as well as funding for research on the use of such resources in authentic learning situations.

The Expert Group recommends measures aimed at student teachers, teachers, school administrators and school owners, so that they can develop competence in learning analytics. Competence in learning analytics and knowledge of artificial intelligence should be included in both basic education and supplementary and continuing education programmes.

The Expert Group recommends that school owners ensure that pupils receive adapted and comprehensible information, so that they can consider issues relating to learning analytics. Furthermore, it is recommended that school owners regularly evaluate whether pupils feel that the school is safeguarding their right to participation.

1.2.4 Guidelines for good and justifiable learning analytics in higher education and tertiary vocational education

The Expert Group recommends the development of broad national guidelines for good and justifiable learning analytics in cooperation with the relevant sectors. The national guidelines must be adaptable to local conditions. At a minimum, the guidelines should include the following action points:

data protection

participation

openness

free choice

procurements

The Expert Group recommends that a government agency develop and administer the broad guidelines for good and justifiable learning analytics in close cooperation with sectoral actors such as Universities Norway and the National Council for Tertiary Vocational Education. The Expert Group emphasises that the responsibility for good and justifiable learning analytics lies with the institutions.

The Expert Group recommends that the broad guidelines be revised regularly in light of rapid technological developments and at least every five years.

The Expert Group recommends that the guidelines include common solutions, local resources and resources that are freely available online.

The Expert Group recommends that a government agency develop a support system to aid educational institutions in preparing risk analyses (DPIAs) and data processor agreements. The government agency shall also assist educational institutions in connection with procurement processes and system development projects.

The Expert Group recommends that the guidelines explain what constitutes good learning analytics that promote student learning.

The Expert Group recommends that competence in learning analytics be included in training programmes for basic pedagogical competence in higher education and tertiary vocational education. In addition, the Expert Group recommends that learning analytics be included in various courses aimed at instructors, administrators and support staff who assist instructors and who participate in quality assurance work.

The Expert Group recommends that teacher training ensures that newly qualified teachers have the requisite competence in learning analytics and knowledge of artificial intelligence. The institutions must consider how they can ensure such competence in instruction and in learning outcome descriptions.

The Expert Group recommends that funding be announced for innovation, research and development pertaining to digital learning resources that have functionality for learning analytics and adaptivity, as well as funding for research on the use of such resources in authentic learning situations.

The Expert Group recommends that the institutions ensure that students receive adapted and comprehensible information so that they can consider issues relating to learning analytics. Furthermore, it is recommended that the institutions regularly evaluate whether students feel that the school is safeguarding their right to participation.

Part 1
Background

2 The composition, mandate and work of the Expert Group

Figure 2.1

2.1 Composition of the Expert Group

The Expert Group for learning analytics consists of experts in fields including education, ethics, technology and law.

Textbox 2.1 Members of the Expert Group

Chair:

Marte Blikstad-Balas, Oslo, Professor at the Department of Teacher Education and School Research at the University of Oslo

Members:

Monica Andreassen, Tromsø, teacher and advisor at Langnes School in Tromsø Municipality

Einar Duenger Bøhn, Lillesand and Oslo, Professor at the Department of Religion, Philosophy and History at the University of Agder

Ann-Tove Eriksen, Tromsø, Head of Department of Innovation in Education at the Norwegian Directorate for Higher Education and Skills

Michail Giannakos, Trondheim, Professor at the Department of Computer Science at the Norwegian University of Science and Technology (NTNU)

Hedda Birgitte Huse, Nittedal, Director General of the Division for Learning and Assessment at the Norwegian Directorate for Education and Training

Malcolm Langford, Moss, Professor at the Department of Public and International Law at the University of Oslo and Head of the Centre on Experiential Legal Learning (CELL).

Eirin Oda Lauvset, Asker, lawyer and data protection officer in Asker Municipality

Per Henning Uppstad, Randaberg, Professor at the Norwegian Reading Centre, the National Centre for Reading Education and Research at the University of Stavanger

Barbara Wasson, Bergen, Professor and Director of the Centre for the Science of Learning & Technology (SLATE) at the University of Bergen

2.2 Mandate of the Expert Group

2.2.1 Excerpts from the Expert Group’s mandate

The Expert Group will provide the Norwegian Ministry of Education and Research with a better basis for decisions regarding learning analytics and adaptive teaching aids, as well as exams and tests in primary and secondary education and training, higher education and tertiary vocational education. It will also advise on the need for regulation and input to policy development and measures by the Norwegian Ministry of Education and Research and subordinate agencies.

The tasks of the Expert Group

The Expert Group shall assess pedagogical and ethical issues in the use of learning analytics, as well as legal issues and data protection considerations. The group shall advise national authorities on the need for the development of legislation for the aforementioned levels of education. The work must include assessments of future opportunities relating to the tools and how the market for adaptive teaching aids will develop in the future.

Furthermore, the Expert Group will provide input to the education sector on how good practices can be developed for the use of learning analytics, in line with ethical and pedagogical standards and applicable legislation.

Key questions

In its work, the Expert Group will base its work on the following key questions:

How does learning analytics affect learning?

The Expert Group will assess whether and how learning analytics affects the professional roles of teachers and instructors, the relationship between teachers and teaching aids, views on teaching and on the pupil/student role. For primary and secondary education and training, it is essential to consider learning analytics in light of both the formative and pedagogical missions of the education, and to consider whether learning analytics affect the breadth of what the pupils are to learn and the differences between various subjects.

What are the challenges and potential of learning analytics?

The Expert Group shall assess ethical issues that are closely linked to the pedagogical assessments. Among other things, the Expert Group shall assess how learning analytics can contribute to the inclusion or exclusion of pupils/students or groups thereof from the instruction, e.g., due to special needs, including those requiring universal design or those with linguistic minority backgrounds, and the effects on possible differences in learning outcomes. Assessment of data protection issues and monitoring of the use of the generated data will be key, as well as whether there are various ethical considerations associated with different types of data/data sources. For primary and secondary education and training, there are also ethical considerations, particularly with respect to the age of the pupils, as well as the balance between the need for good supporting data and the desire for data minimisation, and between requirements for the protection of children and the interest in early intervention. Furthermore, social science issues, such as the relationship between the use of learning analytics and public interests, and democratic values such as openness, transparency and privacy, may be highlighted. The Expert Group must assess whether the quality of the knowledge base has consequences for ethical choices, proposed measures and other recommendations for learning analytics.

How can legislation provide appropriate support to the sector?

A key question is whether there is a need for additional regulation or guidelines for the use of learning analytics in sectoral legislation or other legislation. Assessments of pupils’/students’ data protection are an important aspect of the ethical issues, especially when processing data on children and vulnerable groups of pupils and students. Consideration must be given to whether there is a need for clarification as to what types of processing of personal data are permitted to safeguard the rights of pupils/students and whether all use of learning analytics will constitute a form of profiling, cf. Article 22 of the General Data Protection Regulation (GDPR).

What competence do the education sectors require to make good assessments of learning analytics?

The Expert Group shall assess what competence the education sectors require if learning analytics are to be used in training and education, including the legal, financial and digital competence and competence to assess risks related to data protection, ethics and education in the exercise of various roles.

Structure

The work of the Expert Group shall result in two or more interim reports submitted to the Norwegian Ministry of Education and Research. Basic ethical and pedagogical assessments of the opportunities, benefits and risks relating to learning analytics shall be included in the first interim report. Where there are issues common to the different levels of education, these can be addressed jointly. Where there are significant differences between the levels of education, primary and secondary education and training shall be given priority in the first interim report.

Supplement

In December 2022, the Norwegian Ministry decided that the Expert Group’s main report should be published as an Official Norwegian Report (NOU).

2.2.2 The Expert Group’s interpretation of the mandate

The main task of the Expert Group is to contribute to a better basis for future decisions on learning analytics and adaptive teaching aids, exams and tests in the Norwegian education system. We are also to advise on necessary statutory regulations and input to policy development and measures. We have emphasised providing an overview of a complex field and considering pedagogical, ethical and legal issues in context, rather than separately.

As the title of the report suggests, we are concerned with pupils’ and students’ learning and the ways in which learning analytics affect their learning process. This is also a key issue in the mandate. In order to shed light on the question of learning, it has been necessary to identify what characterises learning analytics in today’s schools, vocational colleges, universities and university colleges. We have considered it crucial to gain an overview of what kinds of experiences various actors in the Norwegian education system actually have had with learning analytics. Through our work, it has become clear to us that this area is understudied and that there are very limited sources to inform us of this subject. Simply put, we know little about learning analytics in practice in primary and primary and secondary education and training, tertiary vocational education and higher education.

Discussions about learning analytics are characterised by a considerable gap between the technological potential envisaged for learning analytics and the pedagogical reality in which instruction and learning take place. Therefore, we have given priority to assessing experiences with and research on learning analytics in Norwegian pedagogical practice. It is this knowledge, including a number of responses and ongoing dialogue with the education sector and other stakeholders, that forms the basis for our recommendations.

A holistic view of learning analytics

The Expert Group understands learning analytics to be a process in which data generated by pupils or students are used systematically to enhance learning and improve instruction. Although our mandate requires thorough legal discussions, we have found great value in considering the issues of learning analytics from various academic perspectives. In accordance with the recommendations made in NOU 2019: 23 Ny opplæringslov [New Education Act] we find it essential that technological, pedagogical, normative and ethical aspects are continuously assessed in all learning analytics. The Expert Group has therefore chosen to discuss these aspects in context, rather than keeping them separate.

We have found it valuable that our mandate is broad in scope and does not reduce learning analytics solely to legal issues. Our intention has always been to assess not only what is legal, but also what are pedagogically prudent and ethically justifiable choices related to learning analytics. Our ambition has also been to place the objective of good learning and education at the centre of the assessments to the extent possible.

A balanced view of opportunities and challenges

Most issues regarding digital technology can quickly end up in a polarised debate involving fixed positions either for or against digitalisation in general. This tendency also applies to issues regarding the role of digital technology in learning situations. The discourse surrounding pupils’ and students’ use of technology is characterised by bold claims and strong emotions. However, both the potential and the adverse aspects of using technology in instruction are well documented, and we have strived to remain objective in the general discussions about the overall effect of technology on schools and educational institutions. We have chosen to give a balanced presentation of the opportunities and challenges that learning analytics create or reinforce in education. Therefore, we will not take a position on whether there should be more or less learning analytics at different levels of Norwegian education. However, we will provide advice on potential value and risks.

Artificial intelligence

Our mandate does not expressly mention artificial intelligence, but as we will explain in section 2.3, this is directly relevant to learning analytics. Artificial intelligence has also become increasingly important in pedagogical issues in the past year, largely due to major innovations in the field that were quickly adopted by pupils, teachers, students and instructors – and which raise a number of new pedagogical, ethical and legal questions. The Expert Group has considered it important to include issues related to artificial intelligence and we have done so where appropriate based on our mandate.

Unclear understanding of learning analytics has impacted work of the Expert Group

We will briefly comment on how the fact that the very concept of learning analytics is so foreign has been a challenge in our work. The Expert Group finds that the definitions of learning analytics found in scientific literature are usually incongruous with how learning technology suppliers, instructors, students, teachers, pupils, parents and administrators in the education sector define the term. Today, many people use pupil and student data to improve instruction but without referring to it as “learning analytics.” We have also experienced the opposite, i.e., that the term learning analytics is used rather uncritically and in an excessively broad manner. The ambiguous use of terminology makes it more difficult to understand and obtain knowledge about when and how learning analytics occur.

The task of the Expert Group is twofold: 1) to assess how learning analytics affects learning, and 2) to advise on good practice for learning analytics in today’s schools, training establishments, vocational colleges, universities and university colleges. To carry out the first task, we need to turn to the research and theoretical definitions of learning analytics. For the second task, we must address how the education sector itself uses the term and discusses the analysis and interpretation of pupil and student data. A number of comments to the Expert Group emphasise the need for more systematic insight into learning analytics in practice. It is also important to have relevant, comprehensive and concrete examples that show how analyses of pupil and student data can support learning, in order to be able to assess the value of learning analytics in primary and secondary education and training, higher education and tertiary vocational education in the future. The Expert Group emphasises that it has been challenging to find good examples of practice in Norwegian education, even though many of the digital learning resources in use in the education programmes have functionality for learning analytics.

2.3 The terms learning analytics, adaptivity and artificial intelligence

Learning analytics and adaptivity are two key concepts in the mandate of the Expert Group. Artificial intelligence is not mentioned in the mandate but is nevertheless part of the Expert Group’s work. The reason is that artificial intelligence is an important component of many forms of learning analytics in general and adaptive systems in particular.

There are many different perceptions of these three concepts. In this chapter, we briefly describe the understanding we have applied in this report and how we perceive the terms to be used in practice.

2.3.1 How the Expert Group understands the terms

The Expert Group assumes that learning analytics is the systematic use of data to enhance learning and improve instruction. Learning analytics can most simply be described as a cycle, and it is a process with several necessary steps, as illustrated in Figure 2.2.

Figure 2.2 The steps in learning analytics. Adapted from the visualisation of learning analytics processes defined in ISO/IEC TR 20748-1.

The starting point of the cycle is a learning situation. Data is collected from the learning situation, which is then analysed by a computer. Sometimes, this analysis combines data from the learning situation with data from other sources. The results of the analysis are presented in a manner that allows the recipient to use the information to make a decision on learning or the instruction. An example of how to present results (referred to as visualisation in Figure 2.1) may be a report that provides an overview of a learning assignment or a recommendation for new learning activities. In many digital learning resources, we find elements from learning analytics, such as data collection and visualisation or recommendations for activities. However, learning analytics can only be said to have been performed when a change has occurred based on the data from the analysis.

The Expert Group assumes that adaptivity means the automated, individual adaptation of a learning resource using artificial intelligence. An example of such adaptation is that a pupil is automatically assigned assignments in a test based on the pupil’s answers to previous assignments in the test. Other forms of adaptivity may be that the content or display of a teaching aid automatically adapts to the pupil’s preferences, based on information about how the pupil has previously used the resource. Adaptivity is relevant for learning analytics because learning analytics will increasingly be based on data from adaptive systems – especially in schools.

The Expert Group assumes that the form of artificial intelligence that is currently most relevant for the education sector is increasingly based on machine learning, i.e., computer programmes with the ability to experience and act – learn – based on large volumes of data. Artificial intelligence is a prerequisite for stating that a learning resource is adaptive but is not necessarily part of learning analytics. However, some resources that have functionality for learning analytics will have built-in artificial intelligence.

2.3.2 How does the education sector use and understand these concepts?

Through our work, it has become clear that the terms learninganalytics and adaptivity are used and understood in many different ways. The term learning analytics is scarcely used in the education sector. The term adaptivity, on the other hand, is almost overused as a general term for all types of adaptations of digital resources. When data is used systematically to enhance learning and improve instruction, it is rarely referred to as learning analytics. At the same time, we see that when the term is used, it is often to describe parts of the steps included in the learning analytics process. Adaptivity – which assumes an automated adaptation for the user with the aid of artificial intelligence – is used to also describe the adaptations the user personally makes to personalise a digital resource.

Artificial intelligence is developing at a rapid pace. In the last year alone, artificial intelligence in education has gained a significantly more prominent position – both in the field of practice and in the public debate. However, not all use of artificial intelligence in education constitutes learning analytics or adaptivity but it can be difficult to define the parameters. Nor is there necessarily a goal to establish fixed boundaries between learning analytics with and without artificial intelligence or to determine when the use of artificial intelligence qualifies as learning analytics. It is more important to focus attention on when pedagogical decisions are in practice made by human beings and when they are made by machines. It is crucial to have a conscious approach to what kinds of decisions should and shall be made by humans and which ones can we leave to the machines.

2.4 The work of the Expert Group

The Expert Group has held a total of nine meetings. Four meetings were held before the group’s interim report was submitted on 1 June 2022 and a further five meetings were held until the submission of the main recommendation on 6 June 2023.

2.4.1 Input and knowledge gathering

During the course of its work, the Expert Group has involved many stakeholders and specialist environments.

Many of the meetings included external opening speakers, who presented and highlighted key topics and initiated discussion. See the list of external speakers in Table 2.1.

Table 2.1 Opening speakers at the meetings of the Expert Group

Name of speaker

Topic

Finn Myrstad, Director General of the Norwegian Consumer Council and member of the Norwegian Privacy Commission

The work of the Norwegian Privacy Commission and relevant issues

Crina Damşa, Associate Professor at the University of Oslo

Challenges and opportunities in higher education related to learning analytics and collaborative learning

Cathrine E. Tømte, Professor at the University of Agder

Opportunities, challenges and dilemmas related to learning analytics in schools

Leonora Onarheim Bergsjø, Associate Professor at Østfold University College and the University of Agder

Digital ethics and learning analytics in the education sector

Lene Karin Wiberg, Special Adviser at the Norwegian Association of Local and Regional Authorities (KS)Brian Jørgensen, Specialised Consultant at the City of Oslo

The AVT project (Activity data for assessment and adaptation), an R&D project on learning analytics

Vidar Luth-Hanssen, Assistant Professor at OsloMet

Hans Gunnar Hansen, Head of Department at Nordland Vocational College

A model and tool for learning analytics in online electrical engineering programmes at vocational colleges

Kine Marisdatter, Associate Professor at UiT The Arctic University of Norway

Øystein Lund, Director of Academic Affairs at UiT The Arctic University of Norway

Learning analytics, instruction and learning in higher education

Vegard Moen, Product Area Manager at Sikt

Natasha Harkness, Project Manager at Sikt

Ole Martin Nodenes, Product Area Manager at Sikt

Geir Magne Vangen, Technical Director at Sikt

Services and platforms, the needs of data in the sector: Potential of and obstacles to learning analytics in higher education

Kristian Bergem, Head of Department at the Norwegian Directorate for Education and Training Øystein Nilsen, Head of Department at the Norwegian Directorate for Education and Training

Digital ecosystem and the market for teaching aids in primary and secondary education and training

Clas Lenz, Project Manager at Rambøll

Peder Laumb Stampe, Consultant at Rambøll

Preliminary findings from assessment of learning analytics in primary and secondary education and higher education

Maren Hegna, Senior Policy Adviser at the Norwegian Ministry of Education and Research

Status of work on a new digitalisation strategy for primary and secondary education and training 2023–2030

Mona Naomi Lintvedt, PhD candidate at the Centre for Computers and Law, University of Oslo

Secure frameworks for learning analytics

Annette Grande Furset, Senior Adviser at the Norwegian Directorate for Higher Education and Skills

Kristin Selvaag, Head of Department at the Norwegian Directorate for Higher Education and Skills

Action plan for digital transformation in higher education and research

In November 2022, the Expert Group organised an industry forum in collaboration with ICT Norway, where Canvas, Cappelen Damm, Conexus, Gyldendal, Hypatia Learning, Inspera, Neddy and Visma InSchool participated. At this forum, the suppliers demonstrated various resources with functionalities for learning analytics and provided comments to the Expert Group. The industry forum included speakers from Microsoft, Google and Apple, who also answered prepared questions.

Rambøll conducted an assessment of learning analytics in primary and secondary education and training and higher education on behalf of the Expert Group. See the results of this assessment in section 3.4.

Open input meetings

In February 2023, two open digital input meetings were held on learning analytics in primary and secondary education and training, higher education and tertiary vocational education.

Invited speakers were Rambøll, Bogstad School in Oslo, the School Student Union of Norway, Union of Education Norway, Gyldendal, BI Norwegian Business School, the National Union of Students in Norway, the Organisation of Norwegian Vocational Students and Universities Norway. Several participants made brief statements. Around 150 participants participated in the two input meetings.

Textbox 2.2 Questions discussed in the open input meetings

What types of learning analytics do we need in primary and secondary education and training?

Can adaptivity contribute to better differentiated instruction?

What are the barriers to good learning analytics in primary and secondary education and training?

What types of learning analytics do we need in higher education and tertiary vocational education?

Can learning analytics contribute to closer follow-up of students?

Can learning analytics contribute to better development of study programmes?

What are the barriers to good learning analytics in higher education and tertiary vocational education?

Input meetings

During 2022 and 2023, the Expert Group conducted input meetings with the following actors:

Norwegian Data Protection Authority

Norwegian Directorate of eHealth

Industry: BS Undervisning, Cappelen Damm, Cyberbook, Conexus, Disputas, Fagbokforlaget V&B, Gyldendal, Hypatia, ICT-Norway, Kikora, LearnLab

School Student Union of Norway

Fylkesledernettverk for fylkeskommunale ungdomsråd [Network of County Youth Council Chairs]

Parents’ Committee for Primary and Secondary Education (FUG)

Lawyers: Emily Weitzenboeck (OsloMet), Jon Christian Fløysvik Nordrum (University of Oslo), Kirsten Kolstad Kvalø, Malgorzata Cyndecka (UiB), Mona Naomi Lintvedt (University of Oslo) and Sebastian Schwemer (University of Copenhagen)

Norwegian Association of Local and Regional Authorities (KS)

Municipalities/county authorities: Asker, Lillestrøm, Lørenskog, Oslo, Surnadal (the ICT-Orkidé cooperation), Voss, Møre and Romsdal, Vestfold and Telemark and Vestland

Medietilsynets ungdomsnettverk for digital oppvekst [the Norwegian Media Authority’s youth network for digital upbringing]

Nettverket for medvirkning i opplæringen (NEMIO) [Network for Participation in Education]: School Student Union of Norway, Norwegian Ombudsperson for Children, UNICEF, UngOrg, Voksne for Barn [Adults for Children], Norwegian Association of Youth with Disabilities, Save the Children Norway

Norwegian Association of Graduate Teachers

Norwegian National Union of Students

Organisation of Norwegian Vocational Students (ONF)

Secretariat of the Norwegian Privacy Commission

Save the Children Norway

Umbrella Youth Council of Oslo (SUR)

Sikt – the Norwegian Agency for Shared Services in Education and Research

Norwegian School Heads Association

Universities: Norwegian University of Life Sciences (NMBU), Norwegian University of Science and Technology (NTNU), University of Oslo, University of Stavanger and UiT The Arctic University of Norway

Organisation of Norwegian Vocational Students and Universities Norway.

Norwegian Directorate for Education and Training

Union of Education Norway

Vestland ungdomsutval [Youth Council of Vestland County]

Viken ungdomsråd [Viken Youth Council]

Written comments

The Expert Group has received written comments from the actors in Table 2.2. The written comments are published on the group’s website[3] .

Table 2.2 Actors from whom the Expert Group has received written comments

Anja Salzmann

Umbrella Youth Council of Oslo (SUR)

Christer V. Aas

Save the Children Norway

Cyberbook

Sikt

Parents’ Committee for Primary and Secondary Education (FUG)

Norwegian School Heads Association

Research, Innovation and Competence Development in School (FIKS), University of Oslo represented by Director of Academic Affairs, Øystein Gilje

Norwegian Union of School Employees

BI Norwegian Business School

Norwegian National Support System for Special Needs Education (Statped)

Hypatia Learning

Ungdomspanelet i Møre og Romsdal Youth Panel of Møre og Romsdal]

ICT Norway

University of Oslo

Magne Aarset, Department of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology

Universities Norway (UHR)

Møre og Romsdal County Authority

City of Oslo Agency of Education

Neddy

Union of Education Norway

Nordland Vocational College

Vestfold and Telemark County Authority

Norwegian Association of Graduate Teachers

Vestland ungdomsutval [Youth Council of Vestland County]

Organisation of Norwegian Vocational Students

Viken ungdomsråd [Viken Youth Council]

Call for reduced screen time in primary schools

2.4.2 Involvement of children and youth

The Expert Group has held several meetings with pupils and youth councils in addition to meetings with organisations representing these groups. See Boxes 2.3 and 2.4 for issues discussed at the meetings. Learning analytics applies to and has consequences for children and youth. Therefore, it was important for us to listen to this group’s perspectives on the issues in question. Furthermore, children have the fundamental right in both Article 104 of the Constitution of Norway and Article 12 of the UN Convention on the Rights of the Child, to freely express their views and be heard in all matters that affect them.

Comments from children and youth have broadened our understanding, and we refer to these comments in the report, where relevant.

Conversations with pupils

The Expert Group has had facilitated meetings with pupils in the upper primary level (grades 5-7) and the lower secondary level (grades 8-10) in different parts of the country. During the school visits, we have had semi-structured conversations with pupils in smaller groups. The purpose has not been to obtain a representative overview of pupils’ perceptions, but rather to enhance our understanding of what pupils find to be important.

Textbox 2.3 Questions posed to the pupils

What digital tools do you use in school? How are these tools used?

How do you feel about the tools ‘remembering’ what you have done and adapting accordingly?

How does it feel to receive feedback from a machine compared to getting feedback from your teacher?

What information about you should not be collected?

Who has and should have access to information about you and the data you leave behind in digital tools?

What should parents have access to and in what manner?

How do you feel about developers’ access to data?

Comments from youth councils and networks

The Expert Group has participated in meetings with several of the county youth councils. In the meetings, we have explained what learning analytics are and presented questions that the councils have discussed. We have subsequently received several written comments from the youth councils.

Textbox 2.4 Questions posed to youth councils and networks

What benefits do you see in using digital tools that collect data for use in instruction and learning?

What challenges do you envision when digital tools are used to collect data and such data are used by teachers?

Who should have access to the information that is collected?

How shall pupils and students be informed about what data are collected and how the data are used?

How should youth be involved in deciding what data should be collected and how the data should be used?

Frameworks for the involvement of children in the investigation work

We have greatly appreciated all the comments submitted by children and young people. It is nevertheless challenging to implement processes for the involvement of children. The challenges have involved limited resources and lack of expertise related to such involvement processes. The Norwegian Directorate for Children, Youth and Family Affairs guide Principles and Advice: Child and youth involvement at the system level (Norwegian Directorate for Children, Youth and Family Affairs, 2021) and the Norwegian Ombudsperson for Children’s Participation Handbook (Norwegian Ombudsperson for Children, 2021) have been useful in this work, but we do not believe that these written guides are sufficient.

One shortcoming is that citizens are inadequately involved at the investigation level with respect to digitalisation reforms in the public administration (Broomfield and Reutter, 2022). In recent years, however, there has been a greater focus on involving children and youth in public investigation work, and there are several examples of thorough involvement processes. Several of these investigation efforts have indicated the need for strengthening the frameworks for implementing involvement processes with children.

The Education Act Committee found itself in uncharted waters when it involved children and youth in its investigation process (NOU 2019: 23). Therefore, it called for more comprehensive efforts to contribute to improved processes and routines for involving children. The Children Act Committee proposes that the Norwegian Government consider establishing a body whose main task is to contribute to justifiable involvement processes with children (NOU 2020: 14). Among other things, the Children Act Committee proposes that such a body contribute by designing plans, creating questions for children, recruiting, interpreting results and, if relevant, implementing or facilitating such processes with its own employees.

The Expert Group supports the proposal by the Children Act Committee for the Norwegian Government to consider establishing a body whose main task is to facilitate involvement processes with children in investigation work. With respect to learning analytics, including the use of artificial intelligence, it is essential to safeguard the perspectives of youth, not only in the investigation process, but also in how learning analytics resources are used.

2.5 The Expert Group’s publications

The Expert Group has submitted the following reports: Læringsanalyse – noen sentrale dilemmaer [Learning Analytics – Some key dilemmas] 1 June 2022 and Læring, hvor ble det av deg i alt mylderet? Bruk av elev- og studentdata for å fremme læring [Learning: Lost in the shuffle?] 6 June 2023.

The first interim report comprises two parts. The first part comprises a primer on learning analytics, discusses different types of learning analytics and provides insight into knowledge development, ethics and activities in the field. The second part delves deeper into four dilemmas related to learning analytics. In the first interim report, we emphasise primary and secondary education and training.

In this last report, we delve more thoroughly into learning analytics in Norwegian pedagogical practice, discuss data types and data quality in learning analytics, and examine the legislation relevant to learning analytics for primary and secondary education, higher education and tertiary vocational education. The second part contains the Expert Group’s assessments of how learning analytics can enhance learning and improve instruction, as well as assessments of the pedagogical and ethical challenges and the need to regulate learning analytics. The third part of the report addresses the Expert Group’s proposals and recommendations.

2.6 Other investigation processes relevant to the work of the Expert Group

New Education Act for primary and secondary education and training

A proposal for a new Education Act was circulated for consultation in the autumn of 2021. Prop. 57 (2022–2023) Lov om grunnskoleopplæringa og den vidaregåande opplæringa (opplæringslova) [Act relating to Primary and Lower Secondary Education and Training (the Education Act)] was submitted to the Storting in March 2023. The bill refers to the Expert Group’s work on “assessing pedagogical, ethical, legal and data protection issues in the use of learning analytics and advising on the need for legislative development and comments on good practice”, which “shall provide the Norwegian Ministry with a better basis for decisions concerning learning analytics in the knowledge sector” (p. 473). Proposals for new regulations to elaborate on the provisions of the Act will be circulated for consultation in autumn 2023. The proposal for a new Education Act will be discussed at the Storting and is scheduled to enter into force from the start of the 2024 school year. The Norwegian Ministry will adopt new regulations in the spring of 2024.

Committee for Quality Development in Schools

The mandate of the Committee is to identify and review the needs of teachers, school administrators, school owners and national authorities for tools, tests and data sources for quality development.[4] The Committee shall propose changes to current tests, tools and data sources with the aim of facilitating improved quality development. The interim report Kvalitetsvurdering og kvalitetsutvikling i skolen: Et kunnskapsgrunnlag [Quality assessment and quality development in schools: A platform of knowledge] was presented in January 2023 (NOU 2023: 1). The final recommendation by the Committee will be presented in autumn 2023.

Report to the Storting on professional education programmes

The Norwegian Ministry of Education and Research has announced that it will present a report to the Storting in spring 2024 on professional education programmes in higher education. The report to the Storting will emphasise the education programmes governed by framework plans, including teacher training.

Report to the Storting on tertiary vocational education

The Norwegian Ministry of Education and Research has announced that it will commence work on a new report to the Storting on tertiary vocational education. This report aims to provide additional knowledge on how tertiary vocational education can best utilise its potential and how this sector should be further developed. According to the Norwegian Ministry of Education and Research, the report to the Storting will be presented no later than spring 2025.

3 Learning analytics in Norway today

There has long been considerable interest in questions concerning the role of technology in schools and education. The debate rages on about screen time in schools, the use of artificial intelligence in assessments, children and young people’s privacy and a number of other issues concerning digitalised learning processes. Because the interest is so great, it is surprising that there is hardly any systematic research either on what is collected in terms of pupil and student data, what teachers and instructors believe they need learning analytics for, and what actually constitutes the common use of digital footprints in today’s education system.

As the Expert Group explained in the first interim report, we still have little systematic knowledge about learning analytics in practice at Norwegian schools and educational institutions. Advice on how to develop good and justifiable practices for learning analytics must be based on what we know about current practice and what needs the actors in the education sector believe that learning analytics can address. In this chapter, we will present new findings from an assessment of learning analytics in Norwegian primary, secondary and higher education and training and provide a brief overview of the research. We will also account for the type of needs the various actors in the education say they have for learning analytics and what barriers stand in the way of good learning analytics.

3.1 Brief overview of learning analytics research

In our first interim report, we outlined interdisciplinary research and research and development (R&D) on learning analytics. We pointed out that much of the work done in the field is small-scale practical testing. There is considerable activity, but research is still at a point where it is difficult to envision what will be possible to achieve in practice, where the breakthroughs will occur and what the legal constraints will be (Kluge, 2021; Selwyn, 2022). As the research project GrunnDig[5] on digitalisation in primary and secondary education and training shows in its final report, there is a lack of systematic research that examines the effect of different forms of technology use in academic contexts (Munthe et al., 2022). The GrunnDig report also shows that much of the existing research concerns either science or language subjects. The knowledge summary by Misiejuk and Wasson (2017) on learning analytics also emphasises the lack of knowledge about what they refer to as “everyday analytics”, i.e., knowledge about how the data collected is actually used.

There is little systematic research conducted on learning analytics in real pedagogical practice at all levels of the education system. Another major challenge is that the new knowledge actually being developed about learning analytics both in Norway and internationally tends to remain within the research environment. This means, among other things, that the results often only have a limited impact on established pedagogical practice, for the products and for the market.

However, several research projects related to the use of artificial intelligence in education have been planned and initiated. The projects Artificial Intelligence for Assessment for Learning to Enhance Learning and Teaching in the 21st Century (AI4AfL)[6] and Gameplay[7] , which will integrate machine learning with a gaming platform to detect reading and writing difficulties, are already underway. The innovation project Learning in the age of algorithms[8] and the research projects Ethical risks assessment of artificial intelligence in practice (ENACT)[9] and Artificial intelligence in education:Layers of trust (EduTrust AI)[10] have recently received funding – to name a few.

3.2 Different practices at the different levels of education

Although there is little knowledge about learning analytics in practice in primary, secondary, higher vocational and higher education and training, we know that there are some significant differences between the different levels of education. The differences are evident both when we assess which tools are available and in use, and in conversations with suppliers and users.

Practical training in primary and secondary education and training

The general situation in Norway is that much of the commercial development of digital learning resources is primarily aimed at primary and lower secondary schools. This is also where we find the most use of various resources that support learning analytics, and it is here that we find the most learning analytics aimed at individual follow-up, as well as the most adaptive feedback to each pupil. In upper secondary education we find somewhat less use than in primary and lower secondary schools. Although there is more use in primary and lower secondary schools than elsewhere, we can describe the use as limited here as well, and it is often limited to certain subjects. Comments we have received from Research, Innovation and Competence Development in School (FIKS) at the University of Oslo specifically underline this issue. Based on their activities in a number of municipalities, they write that learning analytics is “unfamiliar and is actively used only by a minority of teachers, especially in the lower grades” (FIKS, 2023, p. 1).

Practice in higher education and tertiary vocational education

While it is primarily in primary and secondary education and training that we find most practical examples of learning analytics, most of the research both in Norway and internationally has been conducted on higher education, especially within specific programmes of study with customised tools for planning instruction and courses or related to counteracting dropout. We should note that the use of digital resources in higher education and tertiary vocational education in Norway is primarily limited to various administrative tools such as digital learning platforms, reading list systems and administrative examination systems. This means that the existing use of technology that supports learning analytics is also often intended to address more administrative tasks. For example, the knowledge sector’s service provider Sikt writes in its input that it primarily provides “tools or systems intended to support the instructor or administrative staff in planning and carrying out instruction and assessment, i.e., technical tools that support the educational process” (Sikt, 2022, p. 1).

An important common feature of the administrative systems in higher education is that they are not primarily purchased to facilitate learning analytics, even though they may have functionalities that allow for this. We also note that whereas learning analytics in primary and secondary education and training often involve individual resources developed for specific subjects or areas in subjects, it tends to be more generic and linked to managing entire subjects in higher education. We are aware that some educational institutions have tested specific tools that facilitate learning analytics in individual courses within, e.g., science subjects. Apart from learning management systems and other administrative systems, we do not find the widespread use of tools with functionalities for learning analytics on a larger scale.

One of the reasons we do not see more learning analytics in pedagogical practice in higher education is that the sector finds the legal basis for learning analytics to be ambiguous. This has been expressed in several comments and it is also highlighted in a report on learning analytics prepared by a working group at the University of Oslo (Langford et al., 2022). There is also less interest in learning analytics in higher education than in primary and secondary education and training.

As Universities Norway (2023) emphasises in its comments to the Expert Group; the fact that learning analytics is less prevalent in higher education than in primary and secondary education and training “may be due both to a lack of knowledge and access to tools and opportunities but also to traditions and culture” (p. 1). This difference in culture is also an issue student unions have raised with the Expert Group: They emphasise that a student is not – and should not be – a pupil. Whereas pupils in primary and secondary education and training have a number of rights related to formative assessment, differentiated instruction and close follow-up, students have traditionally had a far freer and more independent role, where they receive less continuous individual assessment. Although students in higher education and tertiary vocational education are also entitled to adaptations, participation and follow-up, this is usually ensured in other ways than in primary and secondary education and training.

3.3 What types of learning analytics do different groups need?

Both the performance of – and the need for – learning analytics appear to vary between primary and secondary education and training and training, tertiary vocational education and higher education. Below, we explain what needs learning analytics can cover for different groups from these educational levels. We also describe the purposes for which the actors themselves believe they need learning analytics.

3.3.1 Primary and secondary education and training

Pupils

According to section 2-3 of the Education Act, pupils shall be actively involved in the education. The general part of the National Curriculum emphasises that the pupils shall both contribute to and take joint responsibility in the learning community they create with their teachers every day. Pupils shall also participate in the assessment of their own work and reflect on their own learning and academic development pursuant to section 3-10[11] of the Regulations to the Education Act.

In order to achieve such active participation and pupil contribution, pupils require some knowledge of their own learning and academic progress. Learning analytics can help pupils gain more insight into their own learning processes and thereby better equip them to assess their own learning and take a position on issues that concern them in everyday school life. For this to be possible, pupils must understand what the representations of the learning process tell us.

In conversations with pupils, automated feedback is highlighted favourably if it supplements the feedback provided by teachers and is used with due care:

It’s good that the programs can give us feedback more often. After all, teachers don’t always have time for that. We only get feedback from big assignments, not small ones. I need feedback for small assignments just as much. It might be okay not to get it all the time, as it may be demotivating if I’m just making mistakes (pupil, grade 9).

The fact that the feedback is provided immediately is also highlighted favourably: “It’s often good to get the answers right away. Then you know whether you’ve done it right or wrong” (pupil, grade 7). Pupils have also given examples of the type of feedback that might be helpful to them: “It would be good if the programme could give you feedback and suggestions for further work. For example, it would be good if tips on regular mistakes popped up. For example, if I write æ instead of jeg” (pupil, grade 9). However, they have expressed concerns about whether such automated feedback distributed from a programme directly to the pupil could reduce human communication between pupils and teachers. Many also clearly express that automated feedback is of less value to them: “It’s much more true if the teacher says it. I wouldn’t feel happy if the machine just writes ‘good job’, I wouldn’t care. This is not an assessment. It is what is always there” (pupil, grade 7). The pupils also describe automatic adaptations as beneficial: “It is good that apps adapt to how good you are. There are some people who have received math for lower secondary school” (pupil, grade 7).

Parents

According to the Education Act, municipalities and county authorities have a duty to ensure cooperation with parents of pupils in primary, lower and upper secondary education. Parents’ attitudes towards school are of great significance for the pupils’ involvement and effort in school (Drugli and Nordahl, 2016). If parents receive information from learning analytics, it can help them support their child’s development and learning.

In its comments to the Expert Group, the Parents’ Committee for Primary and Secondary Education (2022a) states that learning analytics “may mean that pupils receive more frequent and accurate feedback on the basis of the data available to the technology and education services that are better adapted than what they are currently receiving” (p. 3). The committee also emphasises that the data on the pupil must be accurate, and that the technology is able to use the data correctly. Parents have also expressed to the Expert Group that, it is important to exercise a high degree of caution when using digital devices in education, especially with respect to the youngest children at school (Aas, 2023).

Teachers

Teachers in primary and secondary education and training shall regularly monitor pupils’ learning and adapt instruction as needed. Learning analytics can provide important insight into both academic development and how pupils are working on subject matter. For example, learning analytics can help adapt instruction, provide varied instruction and also document the instruction. Good learning analytics can also support teachers in their integrated assessment work that must be done continuously in the subject, e.g., by informing them of how each pupil has solved various assignments over time, or by providing an overview of how an entire group has solved the same assignment. There is also streamlining potential in learning analytics by analysing each pupil’s assignment solution automatically and simultaneously. This is significantly more efficient than having the teacher assess each of the same assignments individually. In a survey among the members of the Norwegian Association of Graduate Teachers (2023), immediate feedback to pupils was highlighted as the greatest advantage of learning analytics.

In its comments to the Expert Group, Union of Education Norway (2022) maintains that it is the teacher who is the most important factor for pupils’ and that it is therefore important to distinguish between methods that are wholly or partly aimed at replacing the teacher, and methods that can give the teacher a better basis for the pedagogical assessments. The Norwegian Association of Teachers (2022) believes that it is an open question as to what types of data will be capable of providing information with added value to teachers who “on a daily basis have access to richer data sources through communication with pupils in the classrooms” (p. 1). However, in the survey it conducted among its members, it was highlighted that learning analytics can help to more quickly identify pupils in need of additional support: “I find it to be a good tool to gain an overview of the pupil group, and I find that I can implement measures to assist pupils more quickly because I detect them faster” (Norwegian Association of Graduate Teachers, 2023, p. 1).

School administrators

School administrators have the overall responsibility for the quality of education at schools. Therefore, they have a considerable need for information about instruction and learning. Learning analytics can provide useful insight into a class or a grade’s academic progress in various subjects. Such data can contribute to the school’s quality development. For instance, a school administrator can monitor how a class or group of pupils develops over time, the current results in a class compared to other classes or previous cohorts, or the academic level in different classes in the same grade: “Teachers/school administrators will be able to have a more complete/unambiguous overview of pupils’ skill levels and can continuously measure/monitor pupil development. The frequency/rate of data collection will increase the accuracy of the measurement of the pupil’s development and skill level” (Norwegian School Heads Association, 2022, pp. 1–2).

Such results can also support school administrators in their work on quality development, e.g., where additional follow-up is needed, or if resources need to be allocated differently. The Norwegian School Heads Association also states in its comments that experiences with learning analytics among teachers and school administrators vary, and that there is a need for competence in good learning analytics with pedagogical value. Because school administrators have a responsibility for the pedagogical quality of their schools, school administrators also need competence in how learning analytics can be incorporated into school development.

School owners

School owners are responsible for ensuring that all schools have good pedagogical and administrative services for pupils, staff members and parents. Learning analytics can contribute to more evidence-supported quality development in schools and across schools, and it can offer school owners useful insight into schools’ practices and pupils’ learning. In a comment to the Quality Development Committee, school owners stated that they need to be able to compare data across the municipalities and county authorities (NOU 2023: 1). They also stated they need a timeline that follows cohorts of pupils throughout their education. To achieve this, school owners say that they require analytical tools that can guide them and support them in alignment, interpreting and analysing quantitative data and other information.

School owners need data that provide them with information at an overarching level. This applies to schools in the municipality and in the county authority (Skedsmo, 2022). The Norwegian Association of Local and Regional Authorities (KS) network Aggregated Management Data for Large Collaborating Municipalities has expressed that one problem with this is that it is difficult to access data at the municipal level (ASSS, 2022). In order to provide schools with sufficient support to analyse information and develop quality, municipalities want more access to the schools’ results than they currently have (NOU 2023: 1).

3.3.2 Higher education and tertiary vocational education

Students

Having an overview of one’s own academic development can support students in higher education and tertiary vocational education with respect to progress in their study programme. The Organisation of Norwegian Vocational Students (2022) states in its comments to the Expert Group that few students are currently familiar with the concept of learning analytics. Vocational college students therefore have little experience of how learning analytics can affect learning, even though there are student groups at vocational colleges who are more aware of this in their everyday studies, particularly online students. Despite the somewhat low interest among students to date, ONF believes that by collecting data from teaching situations, educational institutions can identify what challenges individual students and the class as a whole have with regard to learning.

In individual learning, it may be beneficial to be made aware of one’s own routines. This can be done by gaining access to your own data, e.g., from task solving, time use and the like. This can enhance your reflection on your own learning process. Botnevik et al. (2020) found that students are willing to share personal data if this benefits them in the form of better marks, improved instruction or a better learning experience. The student unions, for their part, have expressed concern that the analysis of the data may be so instructive that there is no room for self-reflection and self-assessment, or that it invades privacy. The Organisation of Norwegian Vocational Students (2022) states that the use of the analyses must not be at the expense of the formative aspect and the independence that comes with undertaking a longer education.

Instructors

Those who teach at universities, university colleges and vocational colleges have a clear and unambiguous responsibility to structure their instruction in a pedagogically justifiable manner. There is currently a shift towards increasing the use of coursework requirements and student evaluations in various courses, which underlines the desire for good student follow-up during the course of study. In this context, learning analytics can play a role: It can provide insight into how students relate to the subject matter and the status of their academic progress. Universities Norway (2023) notes that learning analytics offers many opportunities to gain additional knowledge about what produces good learning and how to adapt the education to each student: “It can be used by instructors who want to develop their own teaching practice and to support students’ learning processes in real time” (p. 1). In higher education, some of the instruction takes place in large groups in courses with limited opportunities for the instructor and individual students to interact. In this context, learning analytics can contribute to improved insight into and an overview of students’ academic development.

Currently, however, the data on learning that instructors in higher education and tertiary vocational education can obtain via digital resources remain very limited for the vast majority. Digital resources that teachers generally have available are what are referred to as Learning Management Systems (LMS). For the most part, such systems are used to provide an overview of which students are in which groups, facilitate the collection of assignments, share academic resources and facilitate communication and interaction. The need that digital learning platforms mostly appear to cover at present is the documentation of the instruction that has been provided. Learning management systems often contain resources such as slides from the instructor’s presentations, an overview of all the assignments students have received, student responses, assessments of these, and data that shows which students have visited certain resources at different times.

Administrators

Learning analytics can provide important documentation of how different courses are organised with far more detailed insight than a course description would typically provide. Learning analytics can also provide details about how students across courses relate to different resources, assignments and discussions shared on the learning platform (Misiejuk et al., 2023). Sikt (2022) notes that learning analytics can contribute to a “more holistic and coherent design of courses and study programmes” (p. 2). Universities Norway (2023) emphasises that institutions can use data at an aggregated level to make decisions on e.g. the purchase of digital solutions, procedures and legislation.

For programme coordinators in higher education and tertiary vocational education, data from learning platforms can be useful as part of the quality assurance work for individual courses, cohorts or entire courses of study and study programmes. It is precisely in the area of administrative and programme administration across courses that the use of data from learning platforms appears to be greatest at universities, university colleges and vocational colleges today. The extent to which such data are actually included in learning analytics is nevertheless difficult to ascertain. Sikt (2022) also proposes other sources of data that may be included in learning analytics in the future, such as curriculum systems, video services, plagiarism controls and the Common Student System (FS). Sikt further claims that, overall, such data can contribute to quality development in study programmes, among other things by providing new insight into patterns in the manner in which students relate to the various digital resources beyond the learning platforms.

3.4 Assessment of learning analytics in pedagogical practice

In order to obtain systematic knowledge about learning analytics in pedagogical practices in Norway, Rambøll has, on behalf of the Expert Group, assessed the scope, pedagogical practice, attitudes and challenges related to learning analytics. For capacity reasons, this assessment is limited to primary and lower secondary school, programmes for general studies in upper secondary education and higher education.

Rambøll’s research design consists of two different phases of data collection. The first stage was qualitative. During this phase, Rambøll conducted exploratory interviews with experts (3) and focus groups (14) on questions related to practice and attitudes and how the various groups actually speak about learning analytics. Rambøll interviewed teachers, administrators and instructors. In the next phase, quantitative methods were used to investigate the extent of, attitudes towards and variations and distribution in practice. In the quantitative section, the respondents were administrators and ICT managers in primary and lower secondary schools, university colleges and universities.

The quantitative survey was conducted between December 2022 and January 2023. Through a recruitment survey (primary and secondary education and training) and manual registration via contact information from the institutions’ websites (higher education), contact information was gathered from 878 schools and educational institutions. Of these, 625 were primary and lower secondary schools, 215 were upper secondary schools and 38 were university colleges or universities. Based on contact information from the 878 undertakings, Rambøll drew up a list of respondents where everyone was sent the final questionnaire. The survey was distributed electronically to 1560 recipients. There were 673 respondents to the entire survey, while 143 respondents answered some of the questions without completing the entire survey. In total, 43 per cent of the respondents completed the entire survey. Rambøll’s entire survey and a complete account of the methods used can be read in the report Digital læringsanalyse i norsk utdanning: omfang, pedagogisk praksis og holdninger [Learning analytics in Norwegian education: scope, pedagogical practice and attitudes] (Rambøll, 2023).

In the forthcoming review of knowledge regarding learning analytics in primary and secondary education and training and higher education, we combine the data from Rambøll with comments we have received, in addition to previous research from a Norwegian context. We have found that there are systematic differences in how primary and secondary education and training and training and higher education relate to learning analytics. In the forthcoming review, we have therefore chosen to distinguish between what characterises learning analytics in primary and secondary education and training and training and what characterises learning analytics in higher education. Where relevant, we also comment on learning analytics in tertiary vocational education. With regard to barriers to learning analytics in pedagogical practice, we have chosen to discuss this collectively for all levels of education as we see clear common features.

3.4.1 Challenges in assessing the use of learning analytics

Comments to the Expert Group show that the concept of learning analytics is not well established in Norwegian education (Parents’ Committee for Primary and Secondary Education, 2022a; Norwegian Association of Graduate Teachers, 2022; Organisation of Norwegian Vocational Students, 2022; Sikt, 2022; Norwegian School Heads Association, 2022; Universities Norway, 2023). The fact that the term is so unknown in the education sector has meant that it has not been easy to assess learning analytics in practice. We have found that the definitions of learning analytics that derive from the field usually do not correspond with the manner in which instructors, students, teachers, pupils, parents and administrators in the education sector understand the term.

Rambøll’s (2023) study also has a methodological problem as the respondents often seem to have a significantly broader understanding of the terms learning analytics and adaptivity than the definitions would suggest. The high proportion of don’t know responses to some questions indicates that many of the informants in the survey are uncertain about the terms learning analytics and adaptivity, or what the terms actually mean in practice. As Rambøll emphasises, there is a “discrepancy between the conceptual (in research) and contextual understanding of digital concepts in the field of practice” (p. 12), which creates validity challenges. Rambøll has attempted to resolve this both in the focus groups, by repeating and specifying what parts of digital practice were important to assess, and in the survey, by providing definitions of learning analytics and adaptivity in all questions that specifically asked about these terms.

With regard to the term adaptivity, the assessment indicates that respondents in the sector have a far broader understanding of adaptivity than the way it is defined by the field of research and the Expert Group. We note that the respondents in the survey state that they also use adaptive tools in subjects where scarcely any adaptive tools exist on the market today. There is reason to believe that the respondents interpret adaptivity as an adaptation that can be done apart from the tools, while the technical definitions of adaptivity emphasise that adaptive systems refer to automated, individual adaptations to the pupil’s situation using artificial intelligence. In reality, a number of the programmes that respondents in Rambøll’s survey refer to as adaptive do not use artificial intelligence to adapt their content with the aim of supporting pupils’ or students’ learning. We believe this is an important finding in itself: The education sector is not well acquainted with terms such as learning analytics and adaptivity – a point the sector itself also conveys in its comments.

We are not surprised that the respondents are uncertain about the terminology and we emphasise that we fully understand how demanding it can be to navigate all the terms related to various digital solutions. Although we have not systematically evaluated all products on the Norwegian market today, we note that the rhetoric used to market much of today’s learning technology is characterised by an imprecise use of terminology, excessive optimism and promises that are difficult to keep (Egelandsdal et al., 2019) – which is a well-documented and long-lasting trend internationally (Cuban, 2001; Selwyn, 2022).

We also emphasise that the complexity of learning situations with digital resources makes it difficult to compile an exhaustive overview of resources that contain functionalities for learning analytics. As can be seen from the comments we have received from FIKS (2023), the use of licensed resources constitutes a limited part of the overall teaching directed at pupils. This applies in particular to the higher grades of primary school, lower secondary school and upper secondary school. Office support tools such as word processors and presentation tools play a major role across subjects and grades. In such programmes, digital footprints are also left behind, although there is little evidence that such data are used systematically today to track and make pupils aware of their progress in writing and presentation work (FIKS, 2023). Despite the fact that it is not common to include such programmes in discussions on learning analytics in a Norwegian context, we see that this is where the largest volumes of data on pupils’ work is actually found in most schools and in most subjects. We should note that it was precisely such resources that many of the teachers in Rambøll’s survey associated with their digital practice.

3.4.2 What characterises learning analytics in primary and secondary education and training?

Below, we review what we know about learning analytics in primary and secondary education and training. As mentioned, one challenge for the Expert Group is that there is very little prior research to build upon in this knowledge review. Furthermore, it is challenging for both the sector and the participants in Rambøll’s assessment to distinguish learning analytics questions from general questions about digitalisation. Therefore, we have chosen to review the practice highlighted by the informants in the assessment and the extent to which it allows for learning analytics.

Tools and resources

Rambøll’s assessment provides information about the specific resources used in learning analytics at the respondents’ schools. At the primary and lower secondary level, it is the larger publishing companies that are behind the most used resources. Many schools report that they use Salaby (Gyldendal) (68 per cent), Skolen (Cappelen Damm) (54 per cent) and Skolestudio (Gyldendal) (48 per cent), which are fully digital teaching aids in subjects. Well over half of the schools (67 per cent) report that they use Conexus Engage, which is a tool that can provide an overview of data from various examinations and tests such as assessment tests, standardised assessments and digital learning resources. In addition, there are a good number of individual apps in use to varying degrees, such as Multi Smart Practice (44 percent) and Dragonbox (22 percent) in mathematics, the reading tool Aski Raski (41 percent), the vocabulary game Captain Morf (4 percent) and other similar apps.

The resources most upper secondary school respondents report using are subject-specific resources such as Campus Increment (49 per cent) and Kikora (35 per cent) in mathematics, and Duolingo (24 per cent) for language learning, or resources where the teacher can enter content, such as Kahoot! (81 per cent), Quizlet (46 per cent) or WeVideo (2 per cent). Rambøll’s study does not cover various vocational programmes and here there is also minimal prior research. GrunnDig’s final report explicitly calls for more research on digitalisation in vocational education (Munthe et al., 2022).

To sum up, we can divide the resources that primary and secondary education and training uses today into these five main categories. We emphasise that this is not intended as an analytical framework, but rather as a practical overview of the types of tools and resources that are prominent in primary and secondary education and training practice today, beyond office support tools.

Teaching aids: These resources are customised to different subjects and grades and have been developed in line with the Norwegian curricula. The teaching aids often have ready-made assignments related to specific subject areas in the form of e.g. multiple-choice questions to which the pupils can receive feedback. Digital teaching aids can cover many subjects, such as Skolen, Skolestudio and Aunivers, or individual subjects, such as Multi Smart Øving (mathematics).

Learning apps: These apps tend to focus on specific skills or parts of a subject area – often in language and mathematics. Such resources include Duolingo, Dragonbox, and Kaptein Morf.

Administrative tools and learning platforms: These are tools and platforms for digital interaction and for registering submissions and absence, as well as structuring courses. Examples include MS Teams, Itslearning and Visma InSchool.

Analysis tools: These are tools that collect data from other sources and align them for analysis and visualisation of both individual learning and of aggregated data. Examples include Conexus Engage and Conexus Insight.

Various question tools: These are tools where the teacher can create questions, content or assignments. Examples include Kahoot! and Quizlet.

Subjects and grades

Rambøll’s assessment shows that adaptive resources are most frequently used at the primary school level and in mathematics subjects. The reason for this, according to the informants, is that parts of the mathematics subject are well suited for creating assignments with predefined answers. It is also noted that there is a need for volume training in this context: “There is very little use of learning analytics in the higher age levels and in more reflective, creative and physical subjects […]” (p. 15).

Pedagogical and administrative practice

A significant finding in Rambøll’s assessment is that it appears that the use of digital resources in primary and secondary education and training is primarily driven by a desire to create an appetite for learning and a sense of proficiency among the pupils. The informants who were interviewed stated that the challenge for them is to motivate the pupils, not that they lack insight into the pupils’ academic level. We believe this is a finding worth noting, because it indicates that the perceived need for learning analytics in schools is not necessarily substantial. The assessment also indicates that the adaptive resources and information from learning analytics are to a limited extent used to adapt education and adjust instruction.

The report reveals that the most widespread form of learning analytics in primary and secondary education and training is the one that is both easiest and most accessible, i.e., the one that has an administrative purpose. We are referring to analyses of absence and marks (for grades where marks are given), either at the aggregate level for classes, grades, or entire schools. Furthermore, Rambøll finds that teachers tend to look at summaries of a more practical nature, such as how many individuals have performed a particular task, or how the results in a class or grade are distributed. They are significantly less concerned with the fact that learning analytics can provide more detailed insight into the pupils’ academic progress. This is also true for schools that often use adaptive resources.

Assessment of pupils – formatively and summatively

In Rambøll’s assessment, the informants were asked a number of questions to reveal whether information from learning analytics and adaptivity is used to assess pupils, both formatively and summatively. Several factors emerged that we consider significant. First, it is clear that the respondents have a broader understanding of the term adaptivity than is common in the field of research, as both the focus group interviews and the survey reveal. Second, many administrators in primary, lower and upper secondary school respond that they do not know whether or how often teachers use information from adaptive resources in feedback to pupils. Although few adaptive resources are used as a basis for formative assessment, we can characterise the extent as limited in upper secondary school and somewhat broader in primary school and lower secondary school. The extent to which adaptive resources are used also varies between schools. Several of the schools, in primary, lower and upper secondary schools state that they never use adaptive resources.

With regard to summative assessment and marking, the percentage responding don’t know is also high. When asked to what extent information from learning analytics is used to inform marking in various subjects, the percentage of don’t know responses is between 17 and 25 per cent for subjects in primary and lower secondary school and 39 and 45 per cent for upper secondary school. We find it concerning in itself that the informants (in this case school administrators) respond that they do not know whether information from learning analytics is included in the summative assessment of the pupils or when setting marks. However, it is conceivable that the responses would have been somewhat more precise had it been the teachers themselves who had responded to this question. In any case, the findings paint a picture of little common practice and considerable uncertainty about the actual uses of pupil data.

3.4.3 What are the characteristics of learning analytics in higher education?

Below, we review what we know about learning analytics in higher education. The sector is very diverse and its use therefore varies between institutions and subject areas.

Tools and resources

In Rambøll’s assessment, Canvas (78 per cent) is the tool most respondents in the university and university college sector state that they use, but tools such as Kahoot! (59 per cent), Mentimeter (56 per cent) or Inspera (53 per cent) are frequently reported. Again, we must remark that it appears that the respondents understand questions about learning analytics much more broadly than the academic definition, because they also mention certain digital tools that do not currently have a functionality for learning analytics. For higher education, it was also clear in the focus groups that specific questions about learning analytics were quickly met with general answers about learning platforms and assessment tools.

Both the assessment and comments from the sector emphasise that it is primarily learning platforms such as Canvas and Blackboard that are used. These are systems that can allow for some learning analytics, but in Canvas, for instance, these are additional modules that have to be enabled. It does not appear that many people do so. Several representatives of the sector have told the Expert Group that the procurement of learning management systems occurs without emphasising functionality for learning analytics. In practice, this means in places where learning analytics are currently utilised in university and university college sector with the aid of the learning platform, the platform was actually acquired for purposes other than learning analytics.

Administrative use of learning management systems

Learning analytics in higher education is primarily related to the administrative use of learning platforms, as Rambøll’s assessment clearly shows. The reasons for this are not easy to determine with certainty. It may be that they do not know what learning analytics is and what added value it can provide in instruction. It may be that the legal framework is perceived as ambiguous and that this prevents a number of functionalities from reaching the instructors or places of study. It may also be that they do not find that they need access to the students’ academic progress during the course of study.

Meetings we have had with representatives of higher education indicate that there is considerable uncertainty about the legal basis for collecting personal data and that the centralised schemes for approving tools that can be used by all staff members at a given university would limit access to learning analytics. We are aware of individual examples of more systematic learning analytics in higher education (BI Norwegian Business School, 2023), but there are so few of them that they are not included in this review of what typically characterises learning analytics in higher education in general.

When so much of the use of digital resources for which we receive comments from higher education takes place on learning platforms, it is timely to ask what kind of information about learning and instruction can be gathered there. Part of the reason why learning analytics is limited when exclusively retrieving data from learning platforms is that such platforms are not intended to develop or measure students’ academic benefits or progress. In fact, most learning platforms are in practice a collection of academic resources, practical notifications and submissions of various kinds (Lester et al., 2018). A study of how various academic communities used Canvas before, during and after the COVID-19 pandemic shows that the use of functionalities such as discussions and quizzes has increased. However, the study also found considerable variation between different subjects (Misiejuk et al., 2023).

Assessment of Canvas data

In 2020, a working group explored the use of Canvas data in higher education (Unit, 2020). The ambitions were to assess what kinds of data exist in Canvas (variables, interfaces, format, storage time, etc.) and to initiate a discussion about what we want to achieve via Canvas data collection and how such data can be viewed in connection with other data initiatives (Sikt, 2022). The Expert Group notes that the working group reports that it has “found it challenging to identify the needs/wishes for learning analytics/data analysis among staff members, as they do not understand or know what the possibilities are” (Unit, 2020, p. 1). The working group writes that the academic staff are unfamiliar with learning analytics and that this makes it difficult for them to perceive the opportunities and describe the needs. At the same time, technical-administrative staff lack the pedagogical and contextual understanding to be able to “communicate the opportunity space” (p. 1).

3.4.4 Barriers to learning analytics

Rambøll (2023) reveals that the data collected on pupils’ and students’ learning activities is currently primarily used for administrative purposes – not to follow up on individual students or improve instruction. In other words, most of the use is not encompassed by the definition of learning analytics. An important matter is then to identify the barriers to learning analytics in pedagogical practice to enhance learning and improve instruction. In Rambøll’s study, the question of barriers was connected to the use of adaptive resources.

Figure 3.1 shows several factors that are also highlighted in various comments received by the Expert Group on general challenges with learning analytics. Below, we review the barriers we believe are most significant for primary and secondary education and training and higher education.

Figure 3.1 Barriers to increased use of adaptive resources. The figure shows the average of responses.

Source: Rambøll (2023)

Time for familiarisation with relevant resources

As figure 3.1 shows, many respondents in primary, secondary and higher education find one barrier to learning analytics is not having the time to learn how to use new tools. In the assessment, a clear finding is that the respondents find that the pedagogical use of adaptive learning resources and learning analytics largely depends on a certain commitment. Those who wish to make use of such resources have to spend their spare time familiarising themselves with and adopting the solutions as there is usually no time set aside to test them during a workday. If a teacher, school administrator, instructor or programme coordinator does not see any direct added value in learning analytics, they are naturally not willing to invest time in familiarising themselves with specific resources.

The final report of the GrunnDig project emphasises that a small number of the teachers in the study programme on their own initiative seek knowledge about the use of digital resources they can use in their instruction (Munthe et al., 2022). Many teachers who participated in GrunnDig’s survey also report that they prefer to see other teachers try out new technology before using it themselves. This underlines the importance of the professional environment and the role of management.

Employees’ digital competence

Digital technology is the area where most Norwegian teachers say that they need continuing education and training (Throndsen et al., 2019). In GrunnDig’s final report, it is emphasised that teachers in primary and secondary education and training are largely supportive of digitalisation (Munthe et al., 2022). However, it also appears that they are dependent on good support and guidance in terms of local development work and sharing of experiences before they adopt new technology. Rambøll’s survey also confirms that employees perceive insufficient digital competence as an obstacle to performing learning analytics in schools.

With regard to higher education, the 2020 report on the state of higher education in Norway shows that nearly half of the faculty staff at universities and university colleges state that they have not been offered training in the pedagogical use of digital technology, while slightly more than half believe that they need more training (Berg et al., 2020). Although many of the educational institutions offer instruction and learning using technology, such offers are not mandatory – and not everyone has emphasised learning in technological environments in their compulsory offer of basic competence (Ørnes et al., 2021).

The Norwegian Agency for Quality Assurance in Education’s (NOKUT) (2017) inspection of vocational colleges revealed that many vocational colleges fail to clearly describe the digital competence of the academic community, and that, where it is described, they mention competence in the use of tools rather than a more pedagogical and didactically oriented digital competence.

The Expert Group also notes that teacher training programmes have been largely unsuccessful in integrating digital competence into their study programmes and that there are major differences between educational institutions in terms of the type of training the students receive (Gudmundsdottir and Hatlevik, 2018; Instefjord and Munthe, 2016). This is something we also discussed in the first interim report. It is highly concerning that half of newly qualified teachers in one of the relatively few study programmes in this field state that they have a low level of digital competence (Gudmundsdottir and Hatlevik, 2018). This constitutes a significant barrier to good learning analytics in Norwegian schools.

Inadequate interaction between digital resources

Rambøll (2023) identifies it as an obstacle that various digital resources are inflexible and cannot necessarily be combined with other digital resources. Several stakeholders have expressed to the Expert Group that this is a challenge. Among other things, Vestfold and Telemark County Authority (2022) write that it is the market and system suppliers in primary and secondary education and training that largely define the technical instruments and how they can be used in practice. It becomes difficult when the various resources require separate logins, operate with different ways of assessing pupils and collect little information about pupils’ learning. The fact that the resources appear to such a large extent as closed systems makes it more challenging for schools to utilise the access they may have to various resources. Research on different digital platforms in higher education also shows that there are limited opportunities to combine data across different digital resources (Samuelsen et al., 2019) and that various data are stored in different formats that do not adhere to common standards – and which are therefore difficult to align (Samuelsen et al., 2021).

Lack of relevant learning resources

The Expert Group’s first interim report, previous research, Rambøll’s survey and comments received by the Expert Group all note that the current Norwegian market for primary and secondary education and training lacks good learning resources that facilitate learning analytics.

We know that most such learning resources for primary and secondary education and training are found in the subjects of mathematics and languages, while barely any resources exist for a number of other subjects. The Norwegian Association of Graduate Teachers (2022) explicitly states that a lack of resources in upper secondary education is a barrier to good learning analytics. They point out that fewer digital teaching aids are being developed for upper secondary education because the market is not viewed as large enough as long as extensive funding is tied to the Norwegian Digital Learning Arena (NDLA)[12] .

We have also found that there are currently very few digital learning resources for both Norwegian language forms. The right of Sámi pupils to receive instruction in Sámi is also something that is challenged by the market-driven supply of digital teaching aids in schools. Save the Children Norway (2023) expresses concern that there is insufficient development of digital learning resources that ensure universal design:

Universal design in schools shall promote inclusion, equality and equal opportunity for all pupils to participate in the instruction, in social activities and everything that happens at school […] Save the Children Norway is concerned that digital learning resources are increasingly being developed that do not meet the requirement of universal design, and this is therefore contrary to Norwegian law. (p. 6)

When schools report a lack of resources, it may be that the resources only cover parts of the subject but it may also be that there is a lack of resources with universal design, resources in Nynorsk or resources in Sámi.

In higher education and tertiary vocational education, there are even fewer opportunities for learning analytics in individual subjects. One explanation for this is that it is difficult for the market to offer customised solutions for individual courses. The way specific courses are structured varies depending on the institution and a number of courses are unique to their educational institution and in terms of course description. Furthermore, they do not have as clearly defined learning objectives as in primary and secondary education and training. This makes it challenging for developers to design learning analytics solutions for individual courses in higher education and tertiary vocational education. Therefore, there are few genuine opportunities for learning analytics within individual courses at these levels, even though there is general functionality for learning analytics.

The Expert Group finds that a lack of learning resources is a significant barrier to good pedagogical practice involving learning analytics. This will continue to be the case in the future if the market-driven tendency is allowed to dominate.

Lack of guidance from management

It comes as no surprise that a lack of guidance from management is a barrier to learning analytics in primary and secondary education and training and higher education. Previous research has also noted that the decision to use (or not use) digital tools and learning resources is too often at the discretion of the individual teacher (Gudmundsdottir and Hatlevik, 2018). Rambøll also finds a connection between a lack of guidance from management and a lack of systematic learning analytics, particularly in upper secondary education.

We believe that the lack of guidance from management with respect to learning analytics should be viewed in light of the strong autonomy enjoyed by Norwegian teachers and instructors. There is no tradition in Norwegian schools for management to control how teachers conduct their instruction and follow up their pupils (Mausethagen and Mølstad, 2015; Mølstad and Karseth, 2016). At the same time, this barrier indicates that if the school administration or programme administration wish to have more learning analytics, they must actively facilitate this and support their teachers and instructors in including learning analytics in their pedagogical practice.

The teachers and school administrators in the GrunnDig project agree that good support structures are crucial for teachers to develop in a digital school (Munthe et al., 2022). The fact that so many school administrators in Rambøll’s survey respond that they do not know whether learning analytics is included in various pedagogical practices, indicates that school administrations are often also unclear as to whether schools should have more learning analytics. The Expert Group also notes that this, i.e., the lack of guidance from management, is perceived as greatest barrier in higher education. This may indicate that it would be wise to prioritise the formulation of local guidelines for learning analytics at institutions that can inform educational institutions of their learning analytics needs.

Lack of connection to the National Curriculum

One important finding in Rambøll’s survey is the fact that various digital resources with learning analytics are not explicitly linked to specific competence aims, which is a barrier for respondents in primary and secondary education and training, particularly in lower secondary and upper secondary education. This means that teachers themselves must assess in which parts of the National Curriculum learning analytics can help shed light on the pupil’s competence and academic progress. This barrier can be exacerbated by the above-mentioned problem that teachers do not have enough time to familiarise themselves with new tools.

Possible reasons why the proportion of teachers at the primary level are less concerned with links to the National Curriculum are both that they have significantly more tools to choose from at the primary level and that teachers are not required to mark their pupils’ academic performance.

Uncertainty regarding data protection

It is not a new phenomenon that teachers receive a lot of information about their pupils during their schooling, but what is new in terms of using digital resources is the extent of information and that it is stored and becomes part of the pupil’s digital footprint. The use of digital resources is increasing, as is the proportion of digital data that is continuously collected about each individual. This has led to a growing interest and concern about whether schools are protecting the privacy of pupils and about the increased legalisation of the field. The Norwegian Privacy Commission highlights this tendency in its work (NOU 2022: 11). A number of comments we have received confirm that uncertainty about data protection is a major barrier to learning analytics in education.

ICT Norway (2023) emphasises that there is uncertainty among school owners regarding legislation – which means that different school owners arrive at contrasting conclusions on data protection impact assessments – and that the requirements for addressing data protection in procurements are unclear. We have repeatedly been made aware that school owners find that they spend a disproportionate amount of time assessing the data protection implications of using digital learning resources. There is also a considerable fear of making mistakes and using resources that violate pupils’ right to privacy.

The uncertainty surrounding data protection is perceived as even greater in higher education than in primary and secondary education and training. Many cite this as an obvious explanation for why learning analytics in higher education more or less exclusively involves administrative tools and analyses. In its comments to the Expert Group, Sikt (2022) has stated that an unclear legal basis is a barrier to collecting information and conducting learning analytics.

Stable internet access

Although the vast majority of schools and educational institutions should have good access to the internet and good digital infrastructure, Rambøll’s survey shows that grades 1–10 in particular cite lack of access to stable internet as a barrier. There are fewer respondents in upper secondary education – and even fewer in higher education – that report this issue but it is of course a problem for the few institutions that do report a lack of stable internet access. Other national surveys also show that although Norway is generally a highly digitalised society, there are differences in the extent to which schools experience having a sufficient digital infrastructure (Vika et al., 2021). In its comments to the Expert Group, the Norwegian Association of Graduate Teachers (2023) explicitly mentions that lack of internet access constitutes a barrier to learning analytics in today’s schools:

At some schools, teachers have to plan two lessons for each period, one that involves internet access and another that does not, as the school’s network is often down. In such settings, a guideline or ambition to use learning analytics will only be met with a shrug. (p. 4)

Costs

In order for schools and educational institutions to be able to use license-based tools that facilitate learning analytics, it is essential that they have the means to acquire such licenses and that the market for development is sustainable.

As noted by ICT Norway (2023), there are currently major variations in the amount of money municipalities allocate to purchase digital teaching aids. The variation is both national and local as there may also be differences between schools within a municipality in terms of the budget for purchasing relevant resources. ICT Norway emphasises that schools need predictability in order to make good use of learning analytics: “School administrators must also have good and stable financial framework conditions, with a high degree of predictability, to ensure a wide range of digital teaching aids so that all pupils can receive the adapted instruction and follow-up they need” (p. 5).

3.5 Summary

One important point with respect to all digitalisation of education is that realising the potential of technology is never a given (Lund, 2021; Solomon, 2016; Selwyn, 2022). As GrunnDig’s final report also emphasises in its review of research on digital classrooms, we do not always know whether the potential of digitalisation “is actually a potential or just an imaginary potential” (Munthe et al., 2022, p. 10). The reason we emphasise this is a crucial premise for understanding current practice related to learning analytics: There is not necessarily any correlation between the amount of data on learning collected and the systematic use of such data in learning analytics. Although we have never before had so many digital footprints of pupils’ and students’ academic activities as we have now, there is little to suggest that such data are systematically included in learning analytics. A clear finding in the report from Rambøll, which is also confirmed by comments received by the Expert Group, is that widespread use of digital resources does not necessarily mean that the analytical potential of the data collected is being realised. Despite the widespread use of digital resources, few actors in the sector are interested in the analysis opportunities.

Rambøll (2023) summarises the findings from the qualitative focus groups by stating that “learning analytics is something one wants but does not feel that one needs” (p. 27). Those with more enthusiasm for its use are often interested in the possibilities for adaptation in the subject of mathematics. However, it is noted that this interest is as much about future opportunities as it is about the opportunities offered by today’s solutions. This is something we have also experienced in input meetings – the enthusiasm shown for learning analytics is not about today’s learning resources, but about the opportunities that lie ahead.

It is in primary and secondary education and training that learning analytics influences the practice the most, but here too the scope is limited and often also limited to certain subjects. In higher education, the use of digital resources is limited to the use of various administrative tools. Furthermore, the legal basis for learning analytics is perceived as unclear and there is a fear of making mistakes. Barriers to learning analytics largely concern time, competence and lack of guidance from management. Respondents to the Rambøll survey also reported barriers related to a lack of good learning resources, ambiguous connections to the curriculum, inadequate interactions between the various resources and uncertainty related to data protection.

4 Data types and data quality in learning analytics

One prerequisite for all learning analytics is the access to relevant data that has the potential to provide us with insight into learning and instruction. The quality of the data is always crucial to the quality of the insight the data can provide. In this chapter, we will take a closer look at what types of data are relevant for learning analytics and what is meant by good data quality in connection with learning analytics.

4.1 What is data?

There are many different definitions of data depending on one’s perspective. Data is often perceived as “a way of storing, transmitting and processing information in the form of a specific data format”[13] .

In this context, we are primarily concerned with data that can be included in learning analytics. During a typical day, many pupils and students use learning platforms, apps and programmes. Such interactions with digital devices create digital data. Virtually everything we do on digital devices leaves traces and generates data. For example, digital data are created every time a pupil taps the screen in a language app, or every time a student watches an instructional video. In addition to such traces, digital data can be based on analogue signals from, e.g., sensors, which are then digitised. When such data are included in the analysis, it is referred to as multimodal learning analytics (Giannakos et al., 2022).

To find the best possible starting point for interpreting the data, we rely on information about the data itself and the way it was collected. Metadata is often described as “data about data” and provides descriptive information about the data we have. An example of metadata would be the date a digital photo was taken or when a particular document was created or was last modified. The context in which the data was collected is also important for interpreting the data. For learning analytics, the pedagogical context will be relevant, such as whether an assignment pupils have written was a collaborative task, or what kind of instruction students received just before taking a particular multiple-choice test.

4.1.1 Data viewed from different perspectives

From a technical perspective, different types of data must be stored in a database or file system in order to be analysed by statistical software or algorithms. The data are then stored in a data format that is readable and understandable to the software in a computer. Data formats include:

Numeric (integer or decimal)

text (e.g., plain text, html, xml)

audio (e.g., WAV, AIFF)

visual (e.g., images such as JPEG, PNG, TIFF, or video as MPv4)

instrument-specific (e.g., biosensor, gaze tracker, motion sensor)

From a technical perspective, the following technical terms about data are also important:

Metadata is data about data, or data that defines or describes other data (e.g., the time the data was recorded, the type of camera used to take a picture, or the textbook from which the data in question originated).

Multimodal data are combinations of different modalities (e.g. text, image, sensor data, gaze tracking data).

Dataset is a structured collection of data (e.g., consisting of student number, mark and time spent) or an organised collection of unstructured data.

From an analytical perspective we can categorise data as follows:

raw data (unprocessed data recorded and collected but not acted upon) or processed data (data that has been manipulated, e.g., by turning it into a format that allows for visualisation, and alignment and comparison with other data)

real-time data (data that is presented to the user as soon as it is recorded) or historical data (data recorded at an earlier point in time)

structured data (data organised and defined according to specific rules, which is necessary for exchange and interaction), unstructured data (unorganised data) or semi-structured data (a mixture of structured and unstructured data)

From a practical perspective we can refer to the following:

raw data, e.g.,

content data (deliberately created by humans, e.g., when providing personal information to create an account to use an app or upload a video on a platform) or

sensor data (data recorded by a sensor, such as when your movements are recorded by a smartwatch)

analytical data (processed data created by machines following human-machine interaction)

functional data (data created by a machine to enable communication between machines)

From a learning analytics perspective, we often use the term activity data (Kay and Harmelen, 2014). Such data are defined as traces of human action in the electronic or physical world that can be detected by a computer or digital device. The term activity data encompasses visible raw and analytical data and invisible functional data. The different types of data are also reflected in the definition from the report Å lykkes med åpenhet [Succeeding with Openness] where activity data from adaptive teaching aids is described as “the information that is created when a pupil performs tasks in a learning tool. This may be the pupil’s answer to an assignment, information about what assignment the pupil has done, how long the pupil spent on the task and whether the pupil answered the assignment correctly or incorrectly” (Norwegian Data Protection Authority, 2022c, p. 3). Metadata are also generated about the situation where the data are collected, such as what kind of digital device is used or which Feide [centralised identity management solution for the education sector] ID[14] is logged in.

From a data protection perspective, data are referred to as personal data when they can be used to identify a person, either directly or indirectly. This includes data such as name, address, date of birth, telephone number, email address, national identity number, passport number or other identifiers that are unique individually or in conjunction with other data. In learning analytics, personal data may be collected from pupils, students or others to analyse and enhance learning and improve instruction. However, it is important to ensure that personal data are processed in a lawful, responsible and ethical manner and that appropriate technical, administrative or rights-enhancing measures are in place to comply with the requirements of the GDPR in order to protect the rights and interests of individuals.

Far from all forms of learning analytics must be able to identify an individual. Data is often aggregated at the group or organisational level. Aggregated data is data at a higher level that is obtained by combining data from an individual level. The management level, administrators and researchers use aggregated data for a variety of purposes. For instance, data can be used to assess the consequences of measures, recognise trends and patterns in processes and gain relevant insight into make strategic decisions. When data are aggregated to a group or organisational level, the identity of the individual is only used when data are aligned into larger datasets for analysis. The identity is not available in retrospective analyses.

4.2 Data used in learning analytics

The data from the education sector is generated from a wide range of sources. A systematic review of ten studies on data use in learning analytics in higher education in different countries shows that the most commonly used data are activity data, followed by data from the course management systems (e.g., students’ background information) and data from assessment (Samuelsen et al., 2019).

Being able to process data from multiple sources without losing the integrity of the data when aligning different data sets is important for scaling the learning analytics. In order to succeed in this effort, data and data sets must be available in standardised formats. This also ensures what is referred to as interoperability between the different applications that will work together, so that they can exchange data seamlessly.

We find some examples from recent years where learning analytics uses multimodal data (Di Mitri et al., 2017; Giannakos et al., 2019; Worsley et al., 2021). Multimodal data are typically collected from data sources that contain sensors, such as physiological signal bracelets and gaze tracking, but also audio and video. The use of sensor data in learning analytics is still at an early stage and faces technical challenges such as synchronisation and data integration (Samuelsen et al., 2019). In addition, there are a number of unexplored ethical challenges associated with using multimodal data, especially when there are many data sources involved (Worsley et al., 2020), or when data include health data (Martinez-Maldonado et al., 2020).

Data management

Before anyone can analyse the data, it must be organised and stored in a structured format that allows an application or algorithm to manage it. The most common data storage technologies used for learning analytics data are relational databases, files, spreadsheets, or what are referred to as “learning record stores,” such as the open-source solution Learning Locker[15] . In some forms of learning analytics, it is desirable to align data from different data sources. Such data can be stored in different formats and have different levels of structure, or it can be real-time data that is included in the analysis without being stored. In order to make data with different formats available for further management, international standards for data structures have been developed. xAPI[16] and IMS Caliper Analytics[17] are examples of such standards. If someone wishes to structure data from different data sources into an analysis, a unique identifier that can link the data from the different sources is needed. For example, it is possible to use a student’s Feide ID to connect data from different sources.

4.3 Data analysis

Data analysis can involve a variety of techniques and methods. Examples include statistical analysis, machine learning, data mining, and data visualisation. Data analysis can be used to identify patterns, trends and relationships in the data and to test hypotheses. The aim is to uncover and present useful information and support decisions. Developers can also use algorithms to automate such data analysis processes, reducing manual intervention and speeding up analysis.

Selection bias

A source of error that is particularly relevant for machine learning and artificial intelligence, and which is also relevant for learning analytics, is what is referred to as selection bias (Norwegian Ministry of Local Government and Modernisation, 2020). Selection bias may occur if the data sets used in the training of the algorithms only contain information about a part of the relevant data. This may lead to the results referring to associations where they do not exist, or not referring to associations where they actually do exist (Larsen, 2020). Thus, the algorithms may be less effective, or they may contribute to maintaining or reinforcing social biases based on, e.g., gender, background or socioeconomic status.

For learning analytics, this selection bias entails a risk that the learning analytics algorithms will contribute to maintaining and reinforcing existing inequalities and discrimination in education (Lester et al., 2019; Selwyn, 2022). To reduce bias in the learning analytics algorithms, it is important to carefully consider the data used to train the algorithms. This is also part of ensuring that the algorithms are regularly revised and tested for biases. Incorporating ethical principles for how learning analytics should be designed and how learning analytics should be implemented helps to promote fairness and equality in education.

4.4 Data quality

What an analysis can actually tell us about learning and instruction is always inextricably linked to the quality of the data we have available. Data quality is about whether or how well the data correspond to the situation or activity they represent. In other words, data quality is about ensuring correct, complete and current data. It is also necessary to ensure that data are not altered or manipulated, intentionally or unintentionally, in ways that affect the end result. The data must be complete, consistent, accurate, timely, valid and unique in order for it to be described as good data quality (Pipino et al., 2002). Of these six principles, it is often easiest to assess whether the data are complete and valid, and then whether it is timely and unique. The most difficult aspect to determine is whether the data are accurate and consistent.

Complete data

An important principle of data quality is complete data, meaning that no data are missing. In other words, all the data that one expected to collect is actually present. There are various reasons for receiving incomplete data, such as missing values or errors when the data are entered. Missing data or incomplete records can lead to skewed analyses, erroneous conclusions or inaccurate predictions. The steps to ensure the completeness of the data are having clear data collection procedures, validating the accuracy and consistency of the data, and cleaning the data in a manner that addresses missing values and incomplete records. In learning analytics, providing complete data is critical to gaining meaningful insight into pupil and student learning.

Consistent data

Consistent data involves collecting the expected versions of the data and ensuring that they do not contain contradictions or systematic irregularities. A simple example to illustrate this is if an instructor wishes to use information about when students performed a learning activity. If the dates when students participated are recorded in different date formats, the inconsistent entry leads to useless data. It becomes difficult to understand what the data really means, or to align it in meaningful ways.

Accurate data

Accurate data refers to the extent to which the data represents the real phenomenon or information they are intended to represent (construct validity) and how close the data values are to the true values of the underlying phenomenon (validity). Thus, accurate data is data that are correct, precise and represents what it is intended to represent. The procedure to improve accuracy involves quality assurance of data sources, verifying integrity and consistency and methods of cleansing and validating in order to detect and correct errors. In learning analytics, accurate data are critical to providing reliable and valid insight into pupil and student learning.

Timely data

Timely data means that the data must be collected and available at a time that allows for the appropriate use of the information. In pedagogical contexts, this often relates to the proximity of the data to the learning situation. If the data from a given learning situation is not available at the right time, the teacher, instructor or others will not be able to use the information to improve the pupils’ or students’ education.

Valid data

Valid data means that the data provides information in accordance with its intended purpose. If the goal of a learning app, such as a multiple-choice quiz, is to provide information about what pupils know about a given academic topic, and the pupils realise that the longest answer is always correct and therefore always choose this option regardless of the content, the data will not be valid. The data will then not be a valid measure of a pupil’s academic insight into the topic.

Unique data

Unique data simply means that the data must not be recorded more times than it should, i.e., that duplicates are avoided.

5 Legislation relevant to learning analytics

Parts of the legislation that are relevant to learning analytics are basic standards at a general level, such as the Constitution of Norway and conventions. Other parts of the relevant legislation are intended for specific areas. An example of this is the data protection legislation that apply to the processing of personal data. Otherwise, the sector-specific legislation in the field of education plays a key role in regulating learning analytics.

In this chapter, we describe the parts of the legislation that applies to learning analytics. This includes relevant provisions in the Constitution of Norway, the UN Convention on the Rights of the Child and the European Convention on Human Rights (ECHR). Next, we provide an account of general legislation relevant to learning analytics, such as data protection legislation and sectoral legislation. At the end of the chapter, we describe the ongoing work in the EU and in the Council of Europe on general regulation of artificial intelligence.

5.1 The Constitution of Norway and human rights conventions

The provisions of the Constitution of Norway and the human rights conventions may have several functions. With regard to the legal function of the Constitution of Norway, the provisions can firstly establish the frameworks for what is lawful. For example, the legislature cannot ignore the needs of children when drafting new laws. Second, the Constitution of Norway can act as an interpretation factor when interpreting other legislation. Third, the Constitution can serve as a guide in connection with legislative and other policy development.

5.1.1 The right to education

Constitution of Norway

The right to education was incorporated into the Constitution of Norway in connection with the 2014 constitutional revision. At the time, the Norwegian Human Rights Commission found that the provision would not alter the state of the law because the Commission assumed that the Education Act that was force at the time and the Act relating to universities and university colleges were in accordance with the international human rights conventions (Document 16 (2011–2012), section 37.5.1). Article 109 of the Constitution reads as follows:

Everyone has the right to education. Children have the right to receive basic education. The education shall safeguard the individual’s abilities and needs, and promote respect for democracy, the rule of law and human rights.

The authorities of the state shall ensure access to upper secondary education and equal opportunities for higher education on the basis of qualifications.

The right to education is a right in itself, but also a prerequisite for the realisation of other human rights. The wording “safeguard the individual’s abilities and needs” emphasises that the education should not only take place on society’s terms (Document 16 (2011–2012), section 37.5.2.2).

The Constitution of Norway also stipulates that everyone should have access to upper secondary education and that this right applies regardless of qualifications. On this point, the Constitution goes further than what is enshrined in the international conventions. The constitutional provision also contains a duty on the part of the State to facilitate higher education where abilities and qualifications are the determining criteria for access.

The right to education in international conventions

According to Article 2 of Protocol 1 to the ECHR, no person shall be denied the right to education. The ECHR has the force of Norwegian law with the adoption of Human Rights Act of 1999, section 2, first paragraph. If there is conflict between the Convention and Norwegian law, the Convention shall take precedence pursuant to section 3. Norwegian legislation and regulations must therefore comply with the frameworks established by the Convention obligations.

Article 13 of the International Covenant on Economic, Social and Cultural Rights (ICESCR) stipulates that states shall recognise the right of everyone to education. The provision contains broad objectives that “education shall be directed to the full development of the human personality and the sense of its dignity”. Article 13 also states that primary education shall be compulsory, accessible and free to all. Furthermore, higher education shall also be equally accessible to all, on the basis of capacity. Article 28 (education) and Article 29 (objectives of education) of the UN Convention on the Rights of the Child contain similar wordings to those of the ECHR and ICESCR. Both the ICESCR and the Convention on the Rights of the Child have the force of Norwegian law pursuant to Section 2, second and fourth paragraph. A main feature of the conventions is that they grant children a right and a duty to education. The needs of individuals shall be safeguarded, in addition to the authorities facilitating higher education.

5.1.2 The right to privacy

Constitution of Norway

In 2014, the right to privacy was incorporated into Article 102 of the Norwegian Constitution, the provision reads as follows:

Everyone has the right to the respect of their privacy and family life, their home and their communication. Search of private homes shall not be made except in criminal cases.

The authorities of the state shall ensure the protection of personal integrity.

The establishment of the right to privacy in the Constitution of Norway did not constitute a change in the state of the law but was intended to reflect the essence of the international human rights provisions and contribute to highlighting the right to privacy through a principled provision in the Constitution (Document 16 (2011–2012), section 30.6.5). The provision does not mention whether interferences in the right to privacy may be permissible, nor anything about the conditions under which a possible interference may occur. The provision in the Constitution of Norway is related to the principle of legality in Article 113 of the Constitution, which expresses the key principle that “[i]nfringement of the authorities against the individual must be founded on the law.”.

When the Storting’s Standing Committee on Scrutiny and Constitutional Affair considered the proposal, the Committee stated that “the proposal shall be read as meaning that systematic collection, storage and use of information about the personal affairs of others may only take place in accordance with law, be used in accordance with the law or informed consent and erased when the purpose no longer applies” (Recommendation to the Storting No. 186 (2013–2014), section 2.1.9). In addition to the fact that the interference must be founded in law, the Supreme Court of Norway has stated that a law that interferes with privacy or personal integrity must safeguard a legitimate purpose and be proportionate in order to comply with Article 102 of the Constitution of Norway (Supreme Court Reports (Rt.) 2014 p. 1105, paragraph 28; Rt. 2015 p. 93, paragraph 60).

There is a close connection between Article 102 of the Constitution of Norway and Article 8 of the ECHR on the right to privacy. The Supreme Court of Norway has stated that Article 102 of the Constitution of Norway must be interpreted in the light of Article 8 ECHR, but it has stressed that the Supreme Court has an independent responsibility to interpret and develop the Constitution (Rt. 2015 p. 93, paragraph 57).

European Convention on Human Rights (ECHR)

Article 8 of the ECHR establishes the right to privacy and reads as follows:

1. Everyone has the right to respect for his private and family life, his home and his correspondence

2. There shall be no interference by a public authority with the exercise of this right except such as is in accordance with the law and is necessary in a democratic society in the interests of national security, public safety or the economic well-being of the country, for the prevention of disorder or crime, for the protection of health or morals, or for the protection of the rights and freedoms of others.

The most important source for determining the content of state authorities’ obligations and individuals’ rights is the European Court of Human Rights (ECHR). The Court has found that privacy is wide-ranging and has noted that the protection of personal data is of fundamental importance to safeguarding the right to respect for private life. If public authorities store or process someone’s personal data, it will directly affect their privacy, regardless of whether or not the data are used (Marper v United Kingdom No. 30562/04 and 30566/04, paragraph 121). Collecting and processing pupils’ and students’ personal data in learning analytics will constitute an interference with the right to privacy pursuant to Article 8 ECHR. The ECHR contains a framework for how the authorities are to safeguard the fundamental right to privacy in the event of interference and this includes legislative measures.

The central purpose of Article 8 is to prevent authorities from arbitrarily interfering with privacy and this obligation to safeguard the right to privacy is therefore directed at the authorities. Nevertheless, the authorities cannot waive responsibility by delegating duties to private actors and the requirements of Article 8 also apply in such cases (Vukota-Bojić v. Switzerland No. 61838/10, paragraph 47). When it is a private actor that interferes with privacy, the authorities may have a positive duty to safeguard the right to privacy. For example, the authorities may need to take appropriate measures to effectively ensure that the right to privacy is protected (Craxi. 2) against Italy No. 25337/94).

For an interference of privacy to be in line with the Convention, the interference must pass a three-part test. The interference must:

occur accordance with the law

further a legitimate aim

be proportionate to the legitimate aim pursued

The intervention must occur in accordance with the law

The requirement that the interference must occur in accordance with the law means that there must exist a legal basis in national legislation. In addition, the legal basis must be sufficiently foreseeable for the person to whom the interference applies (Satakunnan Markkinapörssi Oy and Satamedia Oy v Finland [Grand Chamber] No. 931/13, paragraphs 150 and 151). It must also contain adequate safeguards against arbitrariness (L.H. v. Latvia No. 52019/07, 2014). What safeguards are necessary must be viewed in the context of the type of interference and the scope thereof (P.G. and J.H. v. United Kingdom No. 44787/98, 2001). The requirement that the interference must be in accordance with the law is closely related to the requirement that the interference is necessary in a democratic society (Marper v United Kingdom No. 30562/04 and 30566/04, paragraph 99).

The intervention must have a legitimate purpose and be proportionate

Legitimate aim means that the interference must be necessary in a democratic society, it must respond to a pressing social need and be proportionate to the need. In its assessment, the European Court of Human Rights (ECtHR) has generally considered whether the interference complies with the fundamental principles of the Article 5 of the Council of Europe’s Convention of 28 January 1981 No. 108 for the Protection of Individuals with regard to Automatic Processing of Personal Data (European Court of Human Rights, 2022, paragraph 105). These fundamental principles concern the minimisation of collected data, whether the data are accurate, adequate and relevant, and whether the data are excessive in relation to the purposes for which they are stored. In addition to this, there are requirements for storage limitations and that the use of the data must be limited to the purpose for which they are collected.

The right to privacy in other international conventions

The right to privacy is also enshrined in other international conventions, including as the Council of Europe’s 1981 Convention for the Protection of Individuals with regard to Automatic Processing of Personal Data – the only legally binding international agreement on data protection. In addition, there is Article 17 of the International Covenant on Civil and Political Rights (ICCPR), which has the force of Norwegian law pursuant to section 2, third paragraph of the Human Rights Act.

5.1.3 Children enjoy special rights protection.

Human rights also apply to children. Children enjoy special rights protection in the Constitution of Norway and in other human rights obligations. In 2004, the UN Convention on the Rights of the Child from 1989 was incorporated into Norwegian law via the Human Rights Act. In the 2014 constitutional revision, article 104 was adopted, which reads as follows:

Children have the right to respect for their human dignity. They have the right to be heard in matters that concern them, and due weight shall be attached to their views in accordance with their age and development.

For actions and decisions that affect children, the best interests of the child shall be a fundamental consideration.

Children have the right to protection of their personal integrity. The authorities of the state shall create conditions that facilitate the child’s development, including ensuring that the child is provided with the necessary economic, social and health security, preferably within their own family.

The constitutional provision on children’s rights aims in particular to highlight those needs that are not covered by the other human rights provisions (Document 16 (2011–2012), section 32.5.1). The constitutionalisation of children’s rights has legal significance, both as an interpretation factor when interpreting legislation and by setting limits for what the legislature can adopt. The provision also has policy and symbolic significance. The policy significance is that decision-makers are to include consideration for children as a goal of policy design. The symbolic significance is that “children are made visible in the Constitution of Norway”.

The first paragraph of Article 104 of the Constitution of Norway stipulates that the right to co-determination in matters concerning the child and the child’s views shall be given due weight in accordance with their age and development.

The fundamental consideration of the best interests of the child is set out in Article 3 (1) of the Convention on the Rights of the Child. The best interests of the child as a fundamental consideration entails that this consideration should not be assessed at the same level as other considerations. Children’s particular situation relates to their dependency, maturity, legal status and, often, voicelessness.

This, in turn, means that children have less of an opportunity than adults to make a strong case for their interests (UN Committee on the Rights of the Child, 2013, section 37). The general comments of the Committee on the Rights of the Child are intended to elaborate on how states parties are to implement the UN Convention on the Rights of the Child and initiate measures that are suitable for fulfilling the Convention obligations and promoting children’s rights. The best interests of the child may conflict with other interests or rights, e.g. of other children, the public, parents, etc. The best interests of the child shall be weighed against other considerations and larger weight must be attached to what serves the child best (UN Committee on the Rights of the Child, 2013, pt. 39). In the proposal for a new Education Act, the Norwegian Ministry proposes to codify the principle of the best interests of the child in a separate and general section (Prop. 57 (Bill) (2022–2023), section 10.5.1). The proposal also entails including pupils over the age of 18 in the scope of the provision.

Under the third paragraph of Article 104 of the Constitution of Norway, children have the right to protection of their personal integrity, which includes protection of privacy. Article 16 (1) of the Convention on the Rights of the Child also protects the child’s right to privacy and family life and reads as follows:

1. No child shall be subjected to arbitrary or unlawful interference with his or her privacy, family, home or correspondence, nor to unlawful attacks on his or her honour and reputation.

2. The child has the right to the protection of the law against such interference or attacks.

The UN Committee on the Rights of the Child has prepared a separate general comment on children’s rights in relation to the digital environment which contains several statements of relevance to learning analytics. The general comment underlines that the processing of children’s personal data that takes place in schools and the authorities’ collection and processing of data, may pose a threat to children’s privacy (UN Committee on the Rights of the Child, 2021, pt. 67).

5.2 Data protection legislation

Learning analytics will in most cases involve the processing of personal data. Section 1 of the Personal Data Act implements the EU General Data Protection Regulation (GDPR) in Norwegian law. The broad objective of the GDPR is to ensure the protection of natural persons and their rights when personal data about them is processed. The GDPR sets requirements for how the processing of personal data can and should take place.

The GDPR also contains a number of provisions on the establishment of a supervisory authority and its role. In Norway, this role is held by the Norwegian Data Protection Authority. The review of data protection legislation below is based on provisions of particular relevance to learning analytics.

Processing of personal data

The scope of the data protection legislation is broad. Article 2 (1) of the GDPR states that the Regulation applies to “the processing of personal data wholly or partly by automated means”. The meaning of the term processing is not intuitive, but is further defined in Article 4(2):

Any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction.

The provision includes a non-exhaustive list of various operations involving personal data that can be defined as processing. The concept of treatment may consist of one or several operations that relate to multiple stages of the processing. The provision is technology-neutral in the sense that it is not limited to specific techniques. There is no requirement for the operation to be automatic.

Learning analytics will often consist of several operations. For example, collection, storage, use, alignment and erasure are typical operations for learning analytics.

Personal data

If the information cannot identify a person in accordance with the requirements of the Regulation, we consider the data to be anonymous. If anonymous data are processed, the GDPR does not apply. It is important to note that the legal understanding of which data are anonymous differs from the common and everyday use of the terms anonymous or anonymised data. Personal data is defined in Article 4(1) of the Regulation.

The provision contains four elements that follow directly from the wording: (1) “any information”, (2) “relating to”, (3) “identified or identifiable”, (4) “natural person”. In the case of learning analytics, the elements “any information” and “natural person” will not present interpretive challenges. In the case law of the European Court of Justice (ECJ), it has been clarified that “personal data” is to be understood broadly (C-434/16 (Nowak), 2017, paragraph 33).

Regarding the element “relating to”, the ECJ has found that information provided as an answer by a candidate during an examination constitutes personal data. In its decision, the ECJ also ruled that the examiner’s comments on the candidate’s answer are part of the candidate’s personal data (C-434/16 (Nowak), 2017, paragraph 42).

For learning analytics, a relevant question is when information would be sufficiently decoupled from an individual to fall outside the scope of the definition of personal data in Article 4(1). In many cases, it will be difficult to determine where to draw the line between personal data and anonymous data (Norwegian Data Protection Authority, 2015). As the Expert Group explains in the first interim report, there is an unresolved question of what criteria should be applied as a basis for assessing whether a natural person is identifiable. Two interpretations have been put forth. On the one hand there is the risk-based approach. Here, the decisive factor is whether there is a reasonable probability that the data controller or others can identify a natural person with the aid of advanced technology. On the other hand, anonymisation is regarded as the result of a process that irreversibly prevents identification, rendering it impossible to identify the natural person.

The Expert Group emphasises that since the boundary between personal data and anonymous data is so blurred, it may be difficult to clarify the scope of data protection legislation in learning analytics.

Who is responsible for the processing of personal data in learning analytics?

The GDPR is based on the principle of responsibility set out in Article 5(2). The principle of responsibility means that the controller is responsible for ensuring that the processing is lawful and in accordance with the requirements otherwise stipulated in the Regulation. Data subjects have rights in relation to the data controller and the data controller has an obligation to fulfil the rights of the data subjects.

As defined in Article 4(7), a data controller may be “natural or legal person, public authority, agency or other body”.

What determines whether the data controller is responsible for the processing is whether it, alone or jointly with others, decides the purposes and means of the processing. Municipalities and county authorities are data controllers in relation to primary and secondary education and training. In the case of private schools, the school board is the school owner, and the school board is then responsible for the processing. It is the school owner who is responsible for ensuring that the processing of personal data occurs in accordance with the rules in the Personal Data Act and the GDPR. In higher education and tertiary vocational education, the educational institution is the data controller when personal data are processed in the undertaking.

Data controllers may enter into agreements with other parties or undertakings so that they process personal data on the controller’s behalf. Such an actor is referred to as a processor. In learning analytics, data processors may include suppliers of resources with functionality for learning analytics. The relationship between controller and processor is regulated in a data processor agreement. Such an agreement limits how the data processor may process personal data on behalf of the data controller. The data controller may only use data processors who provide sufficient guarantees that the processing of personal data complies with the requirements of the law in practice and safeguards the rights of data subjects. The Norwegian Privacy Commission notes that it is beneficial that the data processor agreement stipulates that the processor shall use certain built-in solutions that are suitable to safeguard privacy (NOU 2022: 11).

The data processor is not permitted to process the data in any other manner than what is stipulated in the data processor agreement. A key point is that processors who breach the data processor agreement or decide the purpose and means of processing themselves are to be regarded as controllers pursuant to Article 28(10).

If two or more controllers jointly determine the purposes and means of processing, these actors are to be regarded as joint controllers pursuant to Article 26(1). In such cases, the actors shall decide how responsibility for fulfilling the processing obligations are to be distributed, unless this is regulated in the legislation.

Data protection principles

School owners and higher education and tertiary vocational educational institutions are responsible for ensuring compliance with the principles for processing personal data set out in Article 5 of the GDPR. (See Box 5.1 for a description of the data protection principles.) The Norwegian Privacy Commission summarises as follows:

Most provisions of the GDPR contain ordinary legal rules. In addition, six data protection principles have been established, cf. Article 5(1). The principles can be regarded as basic norms for the processing of personal data and provide broad guidelines for what to emphasise in order to safeguard privacy. The principles have been developed over a period of more than 40 years and have long formed the basis for various European data protection legislation. They are always relevant and always mandatory to take into account. (NOU 2022: 11, p. 40)

Textbox 5.1 Data protection principles

The processing of personal data shall be lawful, fair and transparent. This assumes that the processing occurs in accordance with the GDPR, human rights enshrined in international conventions and EU law. Fairness means that the controller must consider the interests of data subjects and the expectations they have of the processing of their personal data. This means that conflicting interests are weighed against each other in a manner that ensures proportionality (Bygrave, 2014). Transparency regarding how the processing occurs is a prerequisite for fairness, where data subjects are able to assess how their interests are safeguarded and supervisory authorities are able to inspect that personal data are processed in accordance with the legislation.

Personal data shall be collected for specific purposes (purpose limitation) and shall in principle only be processed in accordance with the original purposes.

Data minimisation means that personal data must be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed.

Closely connected to data minimisation is the principle of storage limitation, which means that personal data shall be stored no longer than is necessary for the purposes for which the personal data are processed.

The personal data shall be accurate with regard to the purpose for which they are processed (accuracy).

The principle of integrity and confidentiality entails that personal data are processed in a manner that ensures appropriate security of the personal data. This involves protection against unauthorised access, unlawful processing, accidental loss, destruction or damage. These conditions are generally referred to as information security.

5.2.1 Requirements for legal basis in the GDPR

In order for the processing of personal data to be lawful, there must be a legal basis for the processing in question. Article 6 of the GDPR contains six possible legal bases for the processing of personal data.

In its first interim report, the Expert Group noted that it is mainly two of the legal bases in Article 6(1) that are relevant for the processing of personal data in learning analytics:

(c) processing is necessary for compliance with a legal obligation to which the controller is subject […].

(e) processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller […].

Common to the processing of personal data in connection with a “legal obligation” or that it is a “task carried out in the public interest” is that Article 6(3) requires the establishment of a basis in EU (regulations and directives) or national law. In other words, it is not sufficient to use Article 6(1)(c) or (e) as the sole basis for the processing. A basis must also be found in national legislation.

Recital 41 of GDPR states that when the Regulation refers to a legal basis or legislative measure, “this does not necessarily require a legislative act adopted by a parliament”. In the preparatory works to the Personal Data Act, the Norwegian Ministry of Justice and Public Security finds that “statutory and regulatory provisions may constitute a supplementary legal basis” Prop. 56 LS (2017–2018), section 6.3.2). Thereby, both acts and regulations can be used as legal basis. The Norwegian Ministry also stated that the GDPR’s rules regarding legal basis in national legislation must be interpreted and applied in the light of the requirements in Article 102 of the Constitution of Norway and Article 8 of the ECHR.

Requirements relating to the design of legal basis pursuant to Article 6(1) (c) and (e) in national law

A number of factors apply when assessing whether a provision in national legislation belongs to the category of “legal obligation” or “task carried out in the public interest” pursuant to Article 6(1).

One practical and important consequence is that the processing that occurs on the basis of ‘public interest’ triggers a right for data subjects to object to the processing pursuant to Article 21 (see further details in section 5.2.3). This means that if the basis for processing falls under the category “legal obligation”, it will restrict the rights of the data subject. This implies greater caution in preparing supplementary legal bases in the legislation based on a “legal obligation”.

For a “task carried in the public interest”, it may be sufficient that the supplementary legal basis presumes or orders a public institution to perform a task that requires the institution to process personal data in order to perform the task in question.

When will a provision fall under the category legal obligation?

Where the legal basis is a legal obligation, the aim of the processing of personal data shall be laid down in national legislation pursuant to Article 6(3) of the GDPR. Nevertheless, a legal basis that is a legal obligation need not expressly regulate the processing referred to in the obligation Prop. 56 LS (2017–2018), section 6.3.2).

In parts of the legal literature one unresolved question has been whether Article 6(1)(c) may constitute a legal basis when the public administration is the controller (Kotschy, 2020). In the preparatory works to the Personal Data Act, it is stated that private actors’ processing of personal data may be necessary for compliance with a legal obligation and for the performance of a task carried out in the public interest Prop. 56 LS (2017–2018), section 6.5). The Norwegian Data Protection Authority has found that a legal obligation may constitute a legal basis for a public authority acting as data controller (Norwegian Data Protection Authority, 2022a). The Norwegian Privacy Commission presumes that Article 6(1)(c) applies to a public administrative body that is acting as controller (NOU 2022: 11). The same view is also found elsewhere in the legal literature, where it is noted that it should be more clearly stated in the GDPR or its recitals whether the basis legal obligation should be reserved for private actors (Udsen, 2022).

If a provision on processing personal data only authorises or allows someone to do something, the provision will not be covered by legal obligation (Kotschy, 2020). Where there is legislation entailing that public authorities can take action that requires the processing of personal data, the provision will be covered by Article 6(1)(e) “task carried out in the public interest”. The Norwegian Data Protection Authority states that a legal obligation as a basis for processing indicates that there are no real alternative ways to achieve the aim of the processing set out in the obligation, without processing the data (Norwegian Data Protection Authority, 2022a).

The requirement of necessity and proportionality

For the processing to be lawful, it must be necessary. The requirement of necessity applies both when the processing concerns “compliance with a legal obligation” and the “performance of a task carried out in the public interest”. The GDPR does not define the term necessary.

The requirement of necessity relates to both the data being processed and the actual processing operation(s). Data that are not relevant for the aim of the processing will also not be necessary to process. The requirement of necessity of processing must be viewed in the context of the area being regulated (Kotschy, 2020). In the preparatory works to the Immigration Act, the Norwegian Ministry of Justice and Public Security comments on the criterion of necessity:

The data shall be objectively related to the purpose(s) sought to be achieved through the processing. It is not sufficient that the data may be useful. The data must either on its own, or in conjunction with other data, be significant to the work or to exercise authority. (Prop. 59 (Bill) (2017–2018), section 4.1.3.2)

The Norwegian Ministry of Justice and Public Security provides in an interpretive statement how it interprets the criterion of necessity:

[…] We understand that there is no absolute requirement that the specific processing is strictly necessary, especially that it is not strictly necessary that the processing occurs in a particular manner. (Norwegian Ministry of Justice and Public Security, 2022, section 3.2)

The ECJ has, inter alia, stated that the requirement of necessity may be met in cases where the processing: “contributes to the more effective application” of the legislation in question (C-524/06 (Huber), 2008, paragraph 62).

Whether the criterion of necessity is met will depend on a specific assessment of the relevant legal obligation or task carried out in the public interest and the relevant processing of personal data. We will more closely examine whether this applies to the processing of personal data in learning analytics in section 10.2.

Article 6(3) stipulates that the national supplementary legal basis based on Article 6(1)(c) and (e) shall be proportionate to the legitimate aim. Proportionality concerns the selected means to realise the aim. In this context the means are the type of data (quality), the volume of data (quantity) and the manner in which the data are processed. Recital 39 of the GDPR states that personal data “should be processed only if the purpose of the processing could not reasonably be fulfilled by other means”. The ECJ enunciates the principle of proportionality as follows: “Under the principle of proportionality, limitations may be made only if they are necessary and genuinely meet objectives of general interest recognised by the European Union or the need to protect the rights and freedoms of others” (C-439/19 Latvijas Republikas Saeima [Grand Chamber], 2021, paragraph 105). This entails that the interference with privacy must be justified in relation to the obligation or purpose of the task carried out in the public interest that the processing of personal data is intended to fulfil. In order to meet the requirement for proportionality, limitations or measures related to the processing that reduce the disadvantages will be relevant.

The requirement for a clear and precise legal basis in national legislation

The provisions of the GDPR do not explicitly state that the supplementary legal basis must be clearly and precisely worded. In case of an interference with the right to privacy under Article 104 of the Constitution of Norway or Article 8 of the ECHR, it may be necessary to have a more specific supplementary legal basis in national law than what is indicated in the provisions of the Regulation Prop. 56 LS (2017–2018) section 6.3.2). Recital 41 states that the legal basis of the legislation “should be clear and precise and its application should be foreseeable to persons subject to it, in accordance with the case-law of the Court of Justice of the European Union (the Court of Justice) and the European Court of Human Rights”. The ECJ stresses that interventions must be necessary and proportionate and that the legislation allowing interference “must lay down clear and precise rules governing the scope and application of the measure in question” (C-439/19 Latvijas Republikas Saeima [Grand Chamber], 2021, paragraph 105).

National margin of discretion in the formulation of a legal basis in legislation

Article 6 (3) specifies what the supplementary legal basis established pursuant to Article 6(1)(c) and (e) may contain in terms of specific provisions to adapt the application of the rules in the GDPR. The specific provisions may, inter alia, involve the general conditions on the lawfulness of the processing, the types of data being processed, the data subjects in question, the entities to which the data may be disclosed, the purposes thereof, purpose limitation, storage period and processing operations and procedures.

Recital 10 of the GDPR states that when processing personal data for compliance with a legal obligation or for the performance of a task carried out in the public interest, “Member States should be allowed to maintain or introduce national provisions to further specify the application of the rules of this Regulation”. In this context, the Norwegian Ministry of Justice and Public Security understands the Regulation to mean that, in principle, it is permissible to issue rules that clarify the principles set out in Article 5(1) in particular for the principles of purpose limitation, data minimisation, accuracy, storage limitation and integrity and confidentiality (Prop. 56 (Bill and Resolution) (2017–2018), section 6.5). The Norwegian Ministry states that it is uncertain whether Article 6(2) and (3) allows for tightening the requirements for processing beyond what follows from the general rules in the Regulation. At the same time, the Norwegian Ministry notes that the principles in the Regulation are so discretionary that the distinction between clarifying and tightening rules is fluid.

A key point regarding learning analytics is that Article 6(2) and (3) does not permit the establishment of less stringent requirements than would result from an interpretation of the general rules of the Regulation (Prop. 56 Prop. 56 LS (2017–2018), section 6.5). In this area, the Regulation sets out minimum requirements that national authorities can make more stringent.

Legal basis for processing special categories of personal data

Article 9(1) of the GDPR stipulates that the processing of special categories of personal data is prohibited. The special categories of personal data in Article 9 concern data revealing:

racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person’s sex life or sexual orientation.

In order to process this type of personal data, one of the bases set out in Article 9(2) must be present. Among the possible bases for processing health data is letter (g): “[P]rocessing is necessary for reasons of substantial public interest”. For research, letter (j) may be used as a basis for the processing, i.e., if it is “necessary for […] scientific […] research or for statistical purposes in accordance with Article 89(1)”.

Common to the bases in (g) and (j) is, firstly, that they require a legal basis for their use in national law. Secondly, Article 9(2)(g) stipulates that the processing shall be proportionate to the aim pursued, respect the essence of the right to data protection and provide for suitable and specific measures to safeguard the fundamental rights and the interests of the data subject. When assessing the necessity and proportionality of the processing, the nature of the data will be key in relation to the type of interference and scope thereof. The requirements to ensure suitable and sufficient measures do not provide a clear answer to the Regulation Prop. 56 LS (2017–2018), section 7.1.3). However, the Norwegian Ministry of Justice and Public Security believes that the primary purpose of the guarantees will be to safeguard fundamental data protection principles when personal data are processed. At the same time, the Norwegian Ministry notes that the content of the guarantees will vary considerably and that one possible form of measures may be rules that specify the processing itself.

5.2.2 Requirements for conducting Data Protection Impact Assessments (DPIAs)

In some cases, the controller has a duty to consider the data protection implications of the planned processing of personal data. The duty to consider data protection implications will, pursuant to Article 35(1), be triggered if the planned processing is likely to result in a high risk to the rights and freedoms of the natural persons in question.

Pursuant to Article 35(4), the supervisory authority (the Norwegian Data Protection Authority) shall prepare a list of the kind of processing operations which are subject to the requirement for a data protection impact assessment. The Norwegian Data Protection Authority’s overview includes, among other things, “processing of personal data to evaluate learning, coping and well-being in schools or kindergartens. This includes all levels of education: Primary and lower secondary schools, upper secondary schools and higher education” (Norwegian Data Protection Authority, 2019, section 2). This means that learning analytics that require the processing of personal data are high risk and there is a requirement to consider the data protection implications thereof.

Article 35(7) lists four elements to the content of DPIAs:

a systematic description of the envisaged processing operations and the purposes of the processing

an assessment of the necessity and proportionality of the processing operations in relation to the purposes

an assessment of the risks to the rights and freedoms of data subjects

the measures envisaged to address the risks, including safeguards, security measures and mechanisms to ensure the protection of personal data and to demonstrate compliance with the Regulation.

Article 35 (9) stipulates that, where appropriate, the controller shall seek the opinions of data subjects or their representatives on the intended processing. In other words, this means that the school owner should obtain the opinions of pupils and parents/guardians or their representatives on the processing. Similarly, institutions in higher education and tertiary vocational education should obtain students’ or representatives’ opinions on the processing.

The Article 29 Working Party (2017) notes that obtaining opinions can take place in different ways depending on the context in question, e.g., with the aid of surveys. If the opinions of the data subjects conflict with the assessments made by the data controller, the controller shall document how the data are followed up. If the controller chooses not to obtain the opinions of the data subjects, this should also be documented.

The Norwegian Data Protection Authority describes such assessments of data protection implications as a continuous process, especially in cases where the processing of personal data changes (Norwegian Data Protection Authority, 2019). Changes to the processing of personal data can often occur when using artificial intelligence.

5.2.3 Rights of data subjects

Chapter 3 of the GDPR contains provisions on the rights of data subjects. These rights enable pupils and students to protect their personal data and rights. Some of the provisions are aimed at the data controller (the disclosure duty in articles 12–14) and the controller is in any case required to facilitate the exercising of data subjects’ rights.

Children’s rights in the data protection legislation

The starting point of the GDPR is that everyone has the same rights in the processing of personal data. This means that children and adults have the same rights. The Regulation does not define children. It was originally proposed to define children as persons under the age of 18 but this definition was not included in the adopted text.

However, recital 38 of the Regulation emphasises that children merit specific protection with regard to their personal data. This is justified on the grounds that children may be “less aware of the risks, consequences and safeguards concerned and their rights in relation to the processing of personal data”.

Regarding information on rights and the communication thereof in relation to children, the controller shall, pursuant to Article 12(1) present the information in a “concise, transparent, intelligible and easily accessible form, using clear and plain language”.

Parents’ and guardians’ exercise of rights on behalf of the child

The data protection legislation does not contain rules that explicitly regulate the right of parents and guardians to assert their child’s rights.

In the European Data Protection Board’s guidelines on the right of access to personal data pursuant to Article 15, the Board emphasises that children have a right to access their personal data and that the right of access belongs to the child. At the same time, the Board notes that depending on the maturity and capacity of the child, the child may need the holder of parental responsibility to act on the child’s behalf (European Data Protection Board (EDPB), 2022).

The Norwegian Data Protection Authority assumes that there is no general age of majority in the field of education and notes that there is no age of majority under the data protection legislation (Norwegian Data Protection Authority, 2023). At the same time, the Norwegian Data Protection Authority assumes that parents and guardians have parental responsibility until the child is 18 years of age and that parents can, in principle, request access to data stored about the child on learning platforms. However, the Norwegian Data Protection Authority also states that this will have to be assessed on a discretionary basis in each specific case, where, among other things, the age of the child, maturity and the type of personal data will form part of the assessment of whether parents and guardians can request access on behalf of the child.

The Expert Group notes that the legal right of parents and guardians to assert rights on behalf of the child is highly discretionary. It can be challenging to make this assessment without clear guidelines, while flexibility makes it possible to adapt assessments to the individual pupil and the circumstances in general.

With regard to parents’ and guardians’ independent right to access information about children, section 47 of the Children Act stipulates that, as a general rule, parents with parental responsibility have the right to information about the child upon request. Any rejections can be appealed to the county governor.

The right to information

Regarding information collected from the data subject, the person concerned shall, pursuant to Article 13, be provided with, among other things, information on the purposes of the intended processing and the legal basis for the processing. In addition, the data subject shall be provided with information about the storage period and the right to exercise the other rights in the Regulation.

In learning analytics, personal data are not always collected directly from pupils and students. When personal data has not been collected from the data subject, the enhanced disclosure duty in Article 14 is triggered. In addition to the requirements pursuant to Article 13, Article 14 entails, among other things, that the data subject shall be provided with information on the categories of personal data concerned and from which source the collected personal data originates.

Right of access

Article 15 stipulates that data subjects have the right to access personal data concerning themselves. In addition, the provision contains an overview of what kind of information data subjects have the right to access. Of particular relevance to pupils and students is the right to know which personal data are being processed, the purposes for which they are processed, the storage period for the personal data and the criteria determining the duration thereof.

Right to rectification and erasure

The right to rectification in Article 16 grants data subjects a right to obtain from the controller the rectification of inaccurate personal data concerning him or her. The right to rectification must be viewed in the context of the purpose of the processing. If, e.g., the purpose is to evaluate or measure the competence of a pupil or student, it is the degree of precision and error in the answers that forms the basis for achieving the purpose of the processing. Such errors will not constitute grounds for rectification under the data protection legislation (C-434/16 (Nowak), 2017, paragraph 53). Nevertheless, situations may arise where an examination answer and the comments made by the examiner thereto may prove to be incorrect within the meaning of the Regulation. One example is if the answer has been exchanged for another, or if parts of the answer have been lost, meaning that the answer is incomplete.

In Article 17, the right to erasure grants data subjects the right to have personal data erased by the controller. This right is often referred to as the “right to be forgotten”. Certain conditions must be met for the right to erasure to apply. Among other things, the data subject has the right to erasure of data that are no longer necessary in relation to the purposes for which they were collected or where the personal data have been unlawfully processed. The right to erasure will also apply if the data subject has objected to the processing pursuant to Article 21 and there are no overriding legitimate grounds for the processing.

This provision does not apply to processing that is necessary for compliance with a legal obligation and the performance of a task carried out in the public interest, cf. Article 17 (3).

Right to data portability

According to Article 20 of the GDPR, the right to data portability, i.e., the opportunity to move data (content) between different services and systems, entails that data subjects, in principle, have “the right to receive the personal data concerning him or her, which he or she has provided to a controller, in a structured, commonly used and machine-readable format and have the right to transmit those data to another controller […]”.

Right to object

The right to object to processing in Article 21 of the GDPR entails that data subjects may, upon request, halt an otherwise lawful processing of personal data. If the conditions for the right to object are met, the data subject may also demand that the processed personal data be erased. The right to object applies if the legal basis for the processing is a “task carried out in the public interest” pursuant to Article 6(1). This means that the right to object does not apply if the legal basis for the processing is a legal obligation pursuant to Article 6(1)(c).

There is one key exception to the right to object in Article 21(1) of the Regulation. If the data controller can demonstrate “compelling legitimate grounds for the processing which override the interests […] of the data subject”, the processing of the personal data may continue. According to the wording, this assessment will be based on “grounds relating to his or her particular situation”.

It has not been clarified how the specific content of the right to object shall be determined in a pedagogical context. The Norwegian Privacy Board has considered several cases concerning the right to object and erasure and has concluded that there were “compelling legitimate grounds for the processing”.[18] In the opinion of the Expert Group, none of these cases have direct relevance for learning analytics, as they have mainly concerned erasure of internet search engine results and archiving obligations weighed against the interests of data subjects.

There has been uncertainty as to how the right to object should be managed in practical terms. For instance, what if this right triggers numerous requests? And then there is the issue of how the controller should manage requests to object to the processing (Prop. 56 (Bill and Resolution) (2017–2018), section 10.5.4).

The right not to be subject to automated decision-making

Decisions that are fully automated are regulated by Article 22 of the GDPR. Article 22(1) stipulates that the data subject has the right not to be subject to a “decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her”. Three conditions must be met for the data subject to have the right not to be subject to an automated decision: (1) “decision” (2) “based solely on automated processing, including profiling” (3) “which produces legal effects concerning him or her or similarly significantly affects him or her”.

Firstly, it must involve a decision, i.e., something to indicate that a decision has been made or assessments have been performed that could form the basis for further action.

Second, it is a condition that the decision is “based solely on automated processing, including profiling.” Pursuant to Article 4(4) “profiling means any form of automated processing of personal data consisting of the use of personal data to evaluate certain personal aspects relating to a natural person […]”.

The fact that the decision is “based solely on automated processing” presupposes that a person is not able to actually influence the decision. Situations where a person is involved in the decision-making process but does not actively take a position on the automated assessment before the person concerned formally makes the decision will fall under Article 22 (Bygrave, 2020). Article 22 does not apply in situations where decision support is actually considered by the person making the decision.

Third, the decision must have “legal effects concerning him or her or similarly significantly affects him or her [the data subject].” This will typically include administrative decisions, which have legal effect in the sense that the decision determines rights and duties. What might similarly affect the person concerned may be difficult to determine specifically. However, such decisions must have consequences that could seriously affect the well-being of the person concerned. The Article 29 Working Party (2018) provides examples of what may fall under the category and includes the following example from the education sector: “decisions that affect someone’s access to education, for example university admissions” (p. 22). In addition, the Article 29 Working Party notes that the threshold for the decision to significantly affect the person concerned may have been reached in the case of decisions with a clear impact on circumstances, behaviour or choices, which may have significant long-term or permanent effects, and which could lead to discrimination or exclusion of individuals.

Under certain conditions, exceptions may be made to the right not to be subject to an automated decision. Pursuant to Article 22(2)(b), national authorities may lay down legislation permitting automated decision-making, provided that suitable measures have been established to safeguard the data subject’s rights and freedoms interests.

Pursuant to Article 22(4) automated decision-making shall not be based on special categories of personal data referred to in Article 9(1). Nevertheless, there may be exception to this principle if the processing is necessary pursuant to Article 9(2)(g) and suitable measures to safeguard the data subject’s rights and freedoms are in place.

5.3 Legislation in the education sector

There is a comprehensive legislative and regulatory framework that regulates the education sector. In this section, we refer to general provisions that stipulate the objectives of education. However, we discuss the general provisions on the processing of personal data and possible supplementary legal bases for the different levels of education in sections 10.3–10.5.

5.3.1 Legislation in primary and secondary education and training

It is mainly the Education Act and the Regulations to the Education Act that constitute the relevant legislation for learning analytics in primary and secondary education and training. The Independent Schools Act applies to primary and secondary education with the right to government subsidies. To avoid duplicate work, we will not discuss the provisions of the Independent Schools Act and its accompanying Regulations. We assume that assessments and proposals related to the Education Act are also relevant to the corresponding provisions in the Independent Schools Act.

During the final phase of our work, the Norwegian Ministry of Education and Research submitted a proposal for a new Education Act Prop. 57 L (2022–2023). However, the following description is based on the provisions of the current Act.

Objectives of education and training

Section 1-1 of the Education Act stipulates the objectives of education and training in seven paragraphs. Among other things, the provision expresses the values that are to form the basis for education and training and what education and training shall contribute towards and provide insight into. The statutory objective does not directly address pedagogical methods that are to form the basis for the education and training, but the fifth paragraph stipulates that “pupils and apprentices must develop knowledge, skills and attitudes so that they can master their lives and can take part in working life and society. They must have the opportunity to be creative, committed and inquisitive.” The sixth paragraph states that the pupils “shall have joint responsibility and the right to participate”.

Provisions of the Education Act with specific relevance to primary and lower secondary school

Pursuant to section 2-1, first paragraph of the Education Act, children and young people are obliged to attend primary and lower secondary education, and they have the right to “public primary and lower secondary education in accordance with this Act and regulations pursuant to the Act”. Section 13-1, first paragraph of the Education Act stipulates that municipalities must comply with the right of all residents in the municipality to primary and lower secondary education. With regard to the content and assessment of education and training, the third paragraph of section 2-3 of the Education Act stipulates that the Norwegian Ministry of Education and Research may, among other things, issue regulations on “content of the instruction in the subjects and the conduct of the instruction”.

Provisions of the Education Act with specific relevance to upper secondary education

The right to upper secondary education is laid down in section 3-1 of the Education Act, which stipulates that “[p]upils, apprentices, candidates for certificate of practice and training candidates have the right to education and training in accordance with this Act and regulations issued pursuant to the Act”. Pursuant to section 13-3, first paragraph of the Education Act, the county authority must comply with the right of all residents of the county to upper secondary education and training. Pursuant to section 3-4, first paragraph of the Education Act, the Norwegian Ministry of Education and Research may issue regulations, including on the scope and implementation of the education and training.

5.3.2 Legislation on higher education and tertiary vocational education

There are several acts and regulations governing higher education and tertiary vocational education. Universities and university colleges are regulated by the Universities and University Colleges Act. The Regulations on the quality of programmes of study[19] relate to quality assurance and quality assurance work in both higher education and tertiary vocational education. The Academic Supervision Regulations[20] only apply to higher education. Vocational colleges are regulated by the Vocational Education Act, the Vocational Education Regulations[21] and the Vocational Education Academic Supervision Regulations[22] .

Objective

The statutory objective in Section 1-1 of the Universities and University Colleges Act stipulates that one of the objectives of the institutions is to “provide higher education at a high international level”. Section 1-3 of the Universities and University Colleges Act prescribes the tasks of the institutions. It states that the institutions must, among other things, provide “higher education based on the foremost within research, academic and artistic development work, and experience-based knowledge”. Pursuant to Section 1-5, first paragraph of Universities and University Colleges Act, institutions are responsible for ensuring that instruction is “conducted in accordance with recognised scientific, artistic, pedagogical and ethical principles”.

Section 1 of the Vocational Education Act stipulates that the purpose of the Act “is to ensure the provision of high-quality vocational education and satisfactory conditions for students of vocational education”. Regarding requirements for vocational education, Section 4, third paragraph of the Vocational Education Act stipulates that the education “shall be based on knowledge and experience from one or more occupational fields and be in accordance with relevant pedagogical, ethical, artistic and scientific principles”.

5.4 Regulation of artificial intelligence (AI)

It is likely that the regulation of AI will be expanded in the near future with two new European regulations. The new regulations on AI go further than the general regulation found in the GDPR and the specific rules governing the use of personal data for, among other things, profiling in Article 22. The two new regulatory proposals at the European level aim to regulate, among other things, the development, marketing and use of AI. The first is the EU’s proposed AI Act (European Commission, 2021) and the second is the Council of Europe’s proposed AI Convention (Council of Europe, 2023).

These proposed regulations could affect how AI is used in learning analytics. These developments may lead to the codification of certain ethical principles, which may in turn lead to a more transnational development of learning analytics technologies within Europe. New actors may then emerge and other mechanisms may be established, which will be relevant for the education sector.

5.4.1 EU regulation of artificial intelligence

In April 2021, the European Commission proposed an AI Act: Artificial Intelligence Act. The broad objective of the proposal is twofold. On the one hand, the objective is to make it easier to utilise the potential of AI, e.g., by eliminating trade-related barriers. On the other hand, it is about protecting societies and individuals from harm, especially in terms of individual safety and human rights.

The Act will apply to systems with artificial intelligence, which are defined very broadly in Article 3(1):

An AI system is a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.

A risk-based approach has been used to both define the level of regulation of each AI system and to the application of the Act (Mahler, 2022). With regard to the level of regulation, the proposal distinguishes between four main categories of risks. There are AI systems that (1) contain unacceptable risks and which are therefore prohibited; (2) that are high-risk systems that must comply with specific requirements; (3) that represent a limited risk and entail fewer requirements, and (4) that involve a minimal risk and where no requirements apply (Veale and Borgesius, 2021).

AI in education is categorised as high-risk in specific areas in Annex III (3) of the Act:

(a) AI systems intended to be used to determine access or admission or to assign natural persons to pedagogical and vocational training institutions at all levels

(b) AI systems intended to be used to evaluate learning outcomes, including when those outcomes are used to steer the learning process of natural persons in educational and vocational training institutions at all levels.

Chapter 2 of the Act contains a number of requirements for high-risk systems, including a risk management system, good training models and good data governance, technical documentation, record-keeping of data processes, transparency and provision of information to users, human oversight, accuracy, robustness and cyber security (Articles 8-15 of Title III). Articles 16-51 describe in detail the obligations of suppliers and users of high-risk AI systems.

5.4.2 Council of Europe Convention on Artificial Intelligence

In autumn 2019, the Council of Europe appointed a committee to assess the opportunities and threats that artificial intelligence entails for human rights (Norwegian Ministry of Local Government and Modernisation, 2020). Following a preliminary report, the committee was formalised in 2022 as the Committee on Artificial Intelligence (CAI) (Council of Europe, 2023). In January 2023, the Committee submitted a draft convention: “Revised Zero Draft [Framework] Convention on Artificial Intelligence, Human Rights, Democracy and the Rule of Law”. As in the EU proposal, attention is geared toward systems and the definition of AI is broad. However, the draft convention places greater emphasis on functionalities:

artificial intelligence system means any algorithmic system or a combination of such systems that, as defined herein and in the domestic law of each Party, uses computational methods derived from statistics or other mathematical techniques to carry out functions that are commonly associated with, or would otherwise require, human intelligence and that either assists or replaces the judgment of human decision-makers in carrying out those functions. Such functions include, but are not limited to, prediction, planning, classification, pattern recognition, organisation, perception, speech/sound/image recognition, text/sound/image generation, language translation, communication, learning, representation, and problem-solving […]

The scope of the Convention may be broader than the EU proposal, as the Convention addresses the entire life cycle of AI systems, regardless of whether public or private actors are involved in their design, development or use (Article 4). Articles 5–11 contain a number of state obligations. This includes a duty to ensure that the use of AI in administrative decisions respects human rights, to minimise harm from using AI systems, and to assess potential risks. Education is explicitly mentioned in Article 8(a):

Each Party shall, within its respective jurisdiction, ensure that: […] the application of an artificial intelligence system in provision of goods, facilities and services in essential areas, such as but not restricted to, health, family care, housing, energy consumption, transport, food supply, education, employment, finance, environmental protection, digital information, media and communication is fully compatible with its domestic law and any applicable international law insofar as these require relevant public and private actors to respect human rights and fundamental freedoms.

The remainder of the Convention contains principles relating to the design, development and different types of use of AI systems (Articles 12–18), monitoring mechanisms (Articles 19–23), and risk assessment and training (Articles 24–26). Key principles include equal treatment and non-discrimination, respect for privacy and data protection, compliance with the law, accountability, transparency and security procedures. In addition, the principles involve preventing harmful innovation processes, facilitating public debate and contributing to increased digital literacy in the population. States are also to ensure that suppliers and users take into account and assess AI-related risks.

Part 2
The Expert Group’s assessments

6 Assessing the potential and challenges related to learning analytics

Figure 6.1

A key task of the mandate given to the Expert Group is to provide advice on a good and sound practice for learning analytics. This involves describing the potential of learning analytics, the challenges to be resolved, and how learning analytics affect learning for pupils and students. In this section, we will give explain our assessment of the value of learning analytics in pedagogical work, as well as a few of the special educational and ethical challenges that learning analytics entail. The purpose is to assess how learning analytics can contribute to the realisation of the fundamental values and principles on which the educations are based, and to assess the extent to which learning analytics may challenge these principles.

An assessment of the educational value of learning analytics and the ethical and pedagogical challenges they may present is intricately linked to the rights regarding participation and privacy. The educational value of the information is crucial with respect to, among other things, the issue of the legal aspects of collecting and processing personal data. There is a close connection between the educational value of this data, and the extent to which learning analytics meet the requirements of the data protection legislation stating that processing personal data must be necessary to fulfil its purpose. We will therefore also assess participation in learning analytics and the need to regulate learning analytics to a greater extent than we do today.

Difficult to separate learning analytics from the general digitalisation of education

It is difficult to separate learning analytics from the general digitalisation of education. Digitalisation often encompasses the same central elements that are part of learning analytics. Examples include the measurement, collection, analysis and reporting of data. We have found it extremely challenging and at times impossible to separate the issue of learning analytics from the broader issues of digitalisation and the use of digital learning resources in education.

Although we view learning analytics as part of a broader discussion on digitalisation, our aim in this section is to try to answer questions that directly involve the use of pupil and student data to enhance learning in various ways. We have focused our attention on when and how learning analytics can lead to better learning for pupils and students, and when the disadvantages of learning analytics outweigh the advantages. We are also concerned with situations where learning analytics reinforce existing opportunities and challenges related to digitalisation.

The assessment of learning analytics is closely linked to other forms of learning analyses

Teachers and instructors have always measured, collected, analysed and reported information about pupils and students with the objective of understanding and promoting learning. This means that the value of learning analytics largely coincides with the value of well-known didactic processes related to teaching and learning. It may therefore be difficult to identify how learning analytics provide added value.

Learning analytics introduces new methods, sources, data and systems for understanding and promoting learning. Teachers and instructors continue, as before, to collect and analyse information about the academic development, performance, behaviour and working methods of their pupils and students. The major difference is that in a digitalised education, all of these areas have been given parallel digital information sources, analytic methods and presentations. Our aim is to shed light on when these aspects entail new disadvantages and opportunities.

7 How can learning analytics enhance learning and improve teaching?

In this chapter, we will explain our assessment of the pedagogical value of learning analytics for primary and secondary education and training, higher education and tertiary vocational education. This is predicated upon the knowledge base and discussions in the Expert Group’s interim report, the status description in chapter 3, and on the input we have received during our work. We will compare the value of learning analytics in pedagogical work with the areas of education where we believe learning analytics would have the greatest added value.

We will begin by pointing out seven general requirements that must be present in order for this added value to be realised, and to ensure that learning analytics can appropriately enhance learning and improve teaching and instruction (the list is not exhaustive):

the data on pupil and student learning must be relevant and of good quality

there must be good management systems for privacy and data security that are adapted to the use of artificial intelligence

information on data processing must be clearly communicated to students, pupils and parents

information from the analytics must be presented in a comprehensible manner

information from learning analytics must be viewed in connection with other relevant information about learning and teaching

teachers and instructors must have sufficient and relevant competence

it must be possible to adapt the use of learning analytics to the unique nature of the subject, professional judgement and local conditions

sufficient time, resources and capacity must be provided in order to follow up data gained from the analyses

In addition to these broad requirements, we will in each subchapter explain certain relevant prerequisites in each of the areas.

7.1 The value of learning analytics in primary and secondary education and training

The value of learning analytics is intricately linked to how learning analytics can contribute to the realisation of the values, goals and principles of primary and secondary education and training. It is thus necessary to take a closer look at how learning analytics relate to the principles expressed in laws and regulations, white papers and other governing documents.

The broad part of the National Curriculum for primary and secondary education and training is a key source, as it elaborates on the fundamental values of the education and describes the basic perspectives that characterise pedagogical practices. The National Curriculum has status as a regulation and is therefore binding for schools. Fundamental values of teaching and instruction reflect the universal values of society based on human rights and emphasise that the best interests of the pupil must always be a vital consideration (Norwegian Ministry of Education and Research, 2017). Consideration of the best interests of the pupil must therefore also be a guiding principle when performing learning analytics.

7.1.1 Insight into own learning

Teaching and instruction must give pupils a strong foundation for understanding themselves and for making good decisions in life. The National Curriculum emphasises that schools have the important task of providing pupils with knowledge of and insight into their own learning processes: “Schools must help pupils to reflect on their own learning, understand their own learning processes and independently acquire knowledge” (Norwegian Ministry of Education and Research, 2017, Chapter 2.4).

Through learning analytics, it is possible to obtain more knowledge of pupils’ academic work during the learning process, and to convey this insight to the pupils as an integral part of the instruction. Learning analytics that provide pupils with greater insight based on relevant and qualitatively good data, where they also receive the necessary assistance to interpret and understand the data, can support schools in their formative task that involves having pupils reflect on their own learning and understand their own learning processes. Vestfold and Telemark County authority (2022) describes this as follows: “Learning data can be visualised and presented in ways that can potentially make pupils more capable of understanding what they need to work on to increase their learning outcomes and desire to learn – an increased understanding of their own learning” (p. 3).

Learning analytics can help pupils become more aware of their own positions and help them reflect on what could be good choices for further development. These are important elements for strengthening pupils’ self-regulation[23] . A better understanding of their own learning processes with the aid of learning analytics can better equip pupils to make decisions on issues related to their instruction. Learning analytics can thereby provide better conditions for pupil participation and involvement.

Preconditions for learning analytics to contribute to insight into personal learning

One of the necessary conditions for learning analytics to help enable pupils to achieve a better understanding of their own learning is for teachers to include pupils when interpreting and understanding the information from the analytics. Pupils must be given assistance to understand and make use of the information from the analytics and not be left to interpret analyses on their own. The type of guidance given to pupils must be based on their age group and context. For the youngest pupils, it will be essential for the school to cooperate with the parents to determine how pupils can understand and make use of information from learning analytics and other feedback.

7.1.2 Formative assessments in school subjects

Regulations to section 3-3 of the Education Act state that the purpose of assessments in a school subject is to continually enhance learning, contribute to the desire to learn, and to provide information on academic performance both during the school year and upon conclusion of instruction in the subject. Section 3-10 relating to formative assessments emphasises that the assessment must become an integral part of the instruction and that it should be used to enhance learning, adapt instruction and increase competence in school subjects. Pupils shall

participate in the assessment of their own work and reflect on their own learning and academic development

understand what they are to learn and what is expected of them

be informed of their proficiencies

receive advice on how to continue working to improve their competence

Systematic use of data from learning activities can provide information on pupils’ academic development throughout the year and over time. This information may be useful for giving pupils feedback on their academic progression and learning processes.

Learning analytics generally provide good support for teachers’ feedback to their pupils. However, relevant feedback from the learning resource and directly to the pupil may also be highly valuable. Learning analytics enable pupils to receive more immediate feedback on their own work than a teacher would normally have the capacity to manage in a classroom or with a large group of pupils.

Preconditions for learning analytics to support formative assessments

In order for learning analytics to support formative assessments, more complex information from the analyses would generally have to be conveyed to the pupils through feedback from their teacher. Concrete information about pupils’ assignment answers may be suitable for direct feedback from the learning resource to the pupil. Pupils must also be permitted to make mistakes during their learning processes without the need for storing this data and using it as part of the formative assessment. They should also be made aware that data is being collected for the purpose of formative assessments, or that the data will be incorporated in the final assessments.

7.1.3 Adapted and inclusive instruction

According to the National Curriculum, schools must facilitate equal opportunities for learning and development for all pupils, regardless of their abilities, through differentiated instruction (Norwegian Ministry of Education and Research, 2017). Schools must make plans to ensure that “instruction is experienced as both manageable and sufficiently challenging” (Norwegian Ministry of Education and Research, 2017). The National Curriculum emphasises that differentiated instruction should primarily involve variations in the materials and adaptations for diversity in the community. It is valuable for children from different backgrounds and with different abilities to learn together at school Meld. St. 8 (2022–2023). According to the National Curriculum, teachers must also reflect on how their pupils learn and how they themselves can best lead and support pupils’ learning development and formative development.

Differentiated and inclusive instruction requires knowledge of how pupils learn and what they are able to do. Learning analytics may be one source of such information. With the aid of digital tools, it may be easier to gain an overview of how pupils solve academic problems, how they work, what they are able to achieve, and what they are struggling with in various parts of a school subject. Teachers can gain a better understanding of pupils’ work and learning processes through access to various types of information. Data in the form of test scores can provide ongoing information about how pupils solve specific problems, and it can chart academic progression. Furthermore, activity data on pupil navigation through a learning resource may indicate the components, modules and tasks that work best for different pupils. In this way, systematic use of digital traces from learning processes could provide insight into pupils’ misconceptions and identify academic areas where they require more adapted instruction. The teacher can then use this information to adapt the instruction and to identify pupils who need additional help. Learning analytics can help uncover challenges at an early stage. The analytics can provide a basis for implementing measures quickly, which would then contribute to achieving the goal of early intervention with a greater probability of completing an upper secondary education.

Adaptive teaching aids and learning resources have often been highlighted as a means of differentiating instruction. One of the purposes of adaptivity is to offer pupils assignments that are tailored to their levels and preferences. It may thus be easier and quicker to identify pupils who need additional support and follow-up as the means to achieve this lie in the resource. Having all pupils work on individually adapted assignments in the same classroom can also help promote inclusion (Statped – National Support System for Special Needs Education, 2022). We will elaborate on the value of adaptivity in Chapter 7.3.

Preconditions for learning analytics to contribute to differentiated and inclusive instruction

In order for learning analytics to provide a basis for differentiated and inclusive instruction, teachers must have access to a variety of digital teaching aids that can encompass all pupils, and that teachers are given sufficient time and guidance to learn how to use them.

7.1.4 Quality development

According to section 13-3(e) of the Education Act, municipalities and county authorities must ensure that schools regularly determine the extent to which the organisation, adaptivity and implementation of the instruction contributes towards achieving the objectives set out in the National Curriculum.

Data from learning situations with digital resources is a relevant basis for assessing and evaluating a school’s practices and for supporting decisions. Resources that analyse and compile data from several sources can provide insight into teaching and instruction, as well as learning that takes place in the classroom, for a specific year group, at a school or across schools. The Norwegian Government’s digitalisation strategy for schools also states that data from pupils’ learning situations can be aggregated and used analytically at a broad level as support for decisions, e.g. on purchases or for knowledge development (Norwegian Ministry of Education and Research, 2023). In upper secondary education, it may be relevant to assess how learning analytics can contribute to knowledge of conditions that may increase the probability of completing studies, and for implementing measures to prevent dropout.

Learning analytics are thus well-suited for assisting the work on quality development in school, which is consistent with findings from the Committee for Quality Development in Schools regarding school owners’ need for support and information for quality development (NOU 2023: 1). If analytics are based on data generated in learning situations, the contribution will also be time-efficient compared to data generated through reports. Data from pupils’ learning can also provide an insight into their participation over time, and in this way play a role in schools’ long-term work to prevent dropout.

Learning analytics as a basis for quality development at an organisational level is discussed in more detail in Chapter 7.4. This form of learning analytics is also relevant for the ongoing work of the Committee for Quality Development in Schools.

Preconditions for learning analytics to contribute to quality development work

In order for learning analytics to provide a valuable and meaningful contribution to quality development work, the data must contain information that can be analysed at an organisational level. This is particularly important for compilations and comparisons across subjects, year groups, schools and municipalities. It is also essential to be able to link the various data sources

7.1.5 Professional practice

Information from learning analytics is primarily used to give teachers insight into pupils’ learning activities and learning environments, and as support for pedagogical decisions. It can therefore support professional practices as described in the National Curriculum.

Chapter 3.5 of the core curriculum notes that the teaching profession must regularly assess its pedagogical practices to best ensure the needs of the pupils: “Teachers must carefully consider what, how and why pupils learn, and how they best can lead and support the pupils’ education and all-round development” (Norwegian Ministry of Education and Research, 2017). At the same time, it is emphasised that complex pedagogical questions rarely have definite answers. Learning analytics provides information about learning based on data from a number of actions by pupils. This can reduce a complex phenomenon to a manageable selection of variables that can help form the basis for pedagogical decisions.

Teachers and the professional community at a school have a responsibility to assess their pedagogical practices in light of research and evidence-based knowledge and use relevant information about how the instruction is working. If a school has access to high-quality learning analytics, it would be natural for this to be incorporated as one of the tools for further developing the school.

Preconditions for learning analytics to contribute to insight into professional practice

Information from learning analytics must be viewed in connection with other information teachers have about their pupils. Practising professional judgement also involves making decisions on when and how learning analytics should be performed.

7.2 The value of learning analytics in higher education and tertiary vocational education

The Expert Group will assess the value of learning analytics in higher education and tertiary vocational education based on how they can help realise fundamental principles expressed in laws and regulations, white papers and other governing documents. Section 1-1(a) of the Universities and University Colleges Act describes the purpose of such educations as “offering higher education at a high international level”. As stipulated in section 1-5, first paragraph, institutions have a responsibility for ensuring that teaching is “conducted in accordance with recognised scientific, artistic, educational and ethical principles”. Apart from this, institutions are given substantial freedom and responsibilities to design their own academic and set of values within the legal framework (Section 1-5, second paragraph of the Universities and University Colleges Act). In tertiary vocational education, there is a broad set of values stated in section 4 of the Vocational Education Act: “Tertiary vocational education must be based on knowledge and experience from one or more occupational fields and be consistent with relevant pedagogical, ethical, artistic and scientific principles”.

7.2.1 Active student learning

A fundamental principle in higher education and tertiary vocational education is that students should engage in their studies as responsible participants in their own learning (Meld. St. 9 (2016–2017); Meld. St. 16 (2016–2017). Section 2-2, fifth paragraph of the Academic Supervision Regulations emphasises that higher education institutions must facilitate opportunities for students to take an active role in their learning processes. The Report to the Storting (white paper) on vocational college education states the following: “An attractive vocational college will have engaged students who are involved in its direction and can influence its development” (Meld. St. 9 (2016–2017), p. 7).

Students must be able to plan, implement and monitor their own learning and assess the extent to which they must change something to achieve their objective. This is key to what is known as self-regulation. In higher education and tertiary vocational education, learning analytics directed at the students could be valuable for enhancing active student learning and facilitating self-regulation. If students have access to data and analyses of their own learning, they may have a better understanding of their own learning processes, which in turn would provide a good basis for taking steps to make changes as needed. Learning analytics can also give students information on learning activities they have completed, how they have spent their time, and what results they have achieved. Certain tools also have a functionality that provides students with notifications and reminders to help them structure their work. Student organisations point out that students must be given control over these functionalities to prevent them from contributing to greater stress. The Expert Group has received input suggesting that students may experience more stress if they, for instance, receive notifications on learning platforms late at night, during weekends or on public holidays.

Preconditions for learning analytics to contribute to active student learning

In order for learning analytics to strengthen active student learning, students must be given sufficient instructions for understanding and interpreting information coming from the analytics. Higher education institutions must involve students to find out what type of information students need from the learning analytics to help them in their learning processes. They must also ensure that students have some control over functionalities that are directed at them in the form of notifications and reminders. It is important that learning analytics are not performed in ways that could blur the lines between student life and private life.

7.2.2 Student follow-up

Education should be based on knowledge of how students learn best Meld. St. 16 (2016–2017). Although research is lacking on what is needed to ensure that students achieve the best possible learning outcomes, certain factors appear to be more important than others. “The most important factors determining a student’s success are student engagement, the amount of time they spend on their studies, and how they use that time” Meld. St. 16 (2016–2017), p. 16)).

One essential question in higher education and tertiary vocational education is to what extent the use of data that directly identifies individual students has pedagogical value for student follow-up. The answer to this question is crucial for determining whether learning analytics will constitute a proportionate intrusion on privacy.

Good student follow-up requires information on their study activities. Here, learning analytics can contribute. In certain areas, the data base from student learning situations would be sufficient for providing valuable information, and for following up individual students or groups of students. This may, for instance, apply to studies where much of the instruction takes place on digital platforms, or in subject areas where there are quality digital resources that are suitable for learning analyses.

Information from learning analytics may be included as a basis for providing feedback to students and for adapting instruction. However, it is essential that information coming directly from learning analytic systems does not become the only form of student feedback, or that the information from learning analytics replaces feedback that has traditionally been the responsibility of the teacher or instructor. In dialogue with the Expert Group, the National Union of Students in Norway has expressed their concern that an increased collection of data from students may lead to a less authentic dialogue between students and teachers. Learning analytics must not undermine the existing dialogue between students and teachers, regardless of how good the analyses are. Universities Norway (2023) has also emphasised that learning analytics neither can nor should replace student participation and involvement.

Data collected on student learning often says something about how students manage resources, their use of time, ongoing assignments and learning outcomes measured by digital tools. To learn more about larger issues – such as how students experience their instruction, what type of alternative methods of teaching they may envision, or what they believe would be a positive development of comprehensive academic offerings – it would be necessary to engage in a dialogue with them. It is important to keep in mind that data from students’ use of digital resources provides information on how they relate to the various digital resources. For most students, this is only a limited part of their study programme, which emphasises the importance of engaging in a real dialogue with students and ensuring student democracy.

Preconditions for learning analytics to contribute to student follow-up

In order for learning analytics to strengthen student follow-up, teachers and instructors must use analyses to follow up the same students from whom the data has been collected instead of making adjustments for the next class. It is also essential for teachers, instructors and students to collaborate on interpreting student data and on the implications for further instruction and learning.

7.2.3 Inclusive education and lifelong learning

Inclusion and equal access are important principles in both higher education and tertiary vocational education These principles are based on the UN sustainable development goal of an inclusive and equitable quality education, and that everyone should have opportunities for lifelong learning.[24] Universities, university colleges and tertiary vocational colleges must therefore facilitate good access to study programmes, also for students from diverse backgrounds Meld. St. 9 (2016–2017); Meld. St. 16 (2016–2017). A central goal in the long-term plan for higher education is to strengthen access to flexible educational programmes and to use digital teaching methods as a means of achieving this Meld. St. 5 (2022–2023).

Learning analytics can contribute to inclusion by providing higher education institutions with a better foundation for adapting study programmes and instruction for students with different abilities. There are examples of targeted learning analytics for higher education in other countries to reduce the performance gap between majority and minority students (Johnson, 2018).

In tertiary vocational education, there are many part-time mature students and online students with obligations that make it challenging for them to complete their studies. Learning analytics can provide greater knowledge of the type of adaptations for learning that work best through access to data from students’ learning situations. This can give higher education institutions a foundation for adapting studies in ways that enable more students to succeed, and to increase the access to study programmes for other groups in society.

Preconditions for learning analytics to contribute to inclusive education and lifelong learning

In order for learning analytics to become inclusive, there are digital learning resources and data sources that are suitable and accessible for students with different backgrounds, and data from these sources can be collated. It is also necessary for higher education institutions to have a conscious approach to how learning analytics can promote inclusion, and that they understand how it may also be exclusionary. There is a risk of discriminatory outcomes with the use of artificial intelligence if we are not cautious (Costanza-Chock, 2020; Selwyn, 2022). Measures for reducing such a risk are essential for ensuring that learning analytics help to promote inclusion.

7.2.4 Quality development

The Universities and University Colleges Act states that requirements for quality development involve helping to ensure that society has confidence in the quality of Norwegian higher education. According to section 1-6 of the Act, higher education institutions must have mechanisms to “ensure and further develop the quality of education”. Vocational colleges are required to have systems in place for quality assurance, according to section 4-3 of the Vocational Education Act. Section 4-1 of the Vocational Education Academic Supervision Regulations stipulates that vocational colleges must systematically collect information from students to assess whether each individual study programme has achieved its objectives for quality.

Learning analytics can contribute to the work on quality development by compiling and analysing data obtained from students’ learning activities and other relevant sources. These regulations stipulate that the quality work at the education institutions must be continuous. Learning analytics can be a good tool for succeeding in these efforts. If higher education institutions use learning analytics that provide ongoing information about student learning and activities, the analyses could comprise a continually updated basis for improving the quality of study programmes. In addition, learning analytics can help facilitate the testing of new forms of instruction and assessment.

Learning analytics as a basis for quality development at an organisational level is discussed in more detail in Chapter 7.4.

Preconditions for learning analytics to contribute to quality development work

In order for learning analytics to make a positive contribution to the work on quality development, it must be possible to compile data from a variety of sources. Access to relevant databases will probably vary between higher education institutions and different subject areas, but in many cases, learning analytics would be able to contribute information that can be a starting point for improving a study or course design.

7.2.5 Standard time for completion of studies

One ambition of higher education is for students to complete their education as efficiently as possible Meld. St. 16 (2016–2017). Deviating from the standard completion time for a study programme can be challenging for certain students, and higher education institutions should therefore attempt to identify potential dropouts as early as possible.

It would be relevant to see whether learning analytics could be incorporated as a mechanism to obtain more knowledge of conditions that increase the probability of completing studies, and to implement measures to prevent dropout. Artificial intelligence would be a suitable tool for locating and recognising patterns in data. Data analyses from student activities to predict potential dropout has been used extensively by higher education institutions in other countries. The Expert Group notes that analyses of student data to prevent dropout is less common in Norway, although we are aware of a few examples, such as BI (Norwegian Business School, 2022).

It may be difficult to obtain good information about student participation and student work on the basis of data from learning activities. In many cases, students who are collaborating or using analogue learning resources may appear to be neither active nor well-integrated based on data from the learning activities, even if the opposite is true. For instance, it will often appear that a student has not been very active on the learning platform if they have systematically been looking at resources together with another student while only the other student is logged in. It can feel stigmatising if an intervention is implemented for students who are active in ways that have not been registered by the system. It would also be unfortunate if students were to begin adjusting their behaviour solely to satisfy the system and to avoid interventions.

Preconditions for learning analytics to contribute to completing studies within the standard time

In order for learning analytics to help increase the probability of completion, higher education institutions must make a thorough assessment of the correlation between the data they have obtained on the students and the students’ social and academic integration.

7.3 The value of adaptivity

Giving recommendations on the use of adaptive teaching aids and exams is part of the Expert Group’s mandate. In our first interim report, we gave an account of the relationship between adaptivity and learning analytics. Here we concluded that these two areas are interconnected to a large extent, but that adaptive systems do not necessary fall within the definition of learning analytics. Learning analytics are to an increasing extent based on data from adaptive systems, particularly those used in primary and secondary education. It is therefore useful to view learning analytics and adaptivity in context.

Because adaptive teaching aids are so far mostly used in primary and secondary education, we will mainly refer to adaptivity in a school context. Nevertheless, several of our assessments will also be relevant for higher education and tertiary vocational education.

7.3.1 Adaptive systems for instruction

In this context, adaptivity means that the content of a digital system is adapted to the person using the system. The antithesis of adaptive tests and teaching aids are linear tests and materials. This is because pupils who use them all follow the same sequence of tasks. An adaptive system can also adjust the tempo and presentation of the content and offer individualised feedback. Since these adjustments are part of a digital system, this is done with the aid of algorithms. Algorithms that control adaptations to tests and learning systems may be complex, e.g. when involving machine learning, but in most cases, they are quite simple. What is typical for adaptive teaching aids and tests is that they are divided into smaller parts, which in turn consist of several problems. These parts comprise an overview of the material to be learned or measured.

The knowledge base on the use of adaptive systems is limited, but a growing number of studies have focused their attention on how they affect learning (Egelandsdal et al., 2019; Moltudal et al., 2020).

7.3.2 Differentiated instruction

Differentiated instruction involves tailoring instruction to ensure that all pupils receive the best possible outcome of the instruction (Norwegian Ministry of Education and Research, 2017, Chapter 3.2). Adaptivity is often related to this principle because the primary goal of adaptivity is differentiated instruction. The desired result of adaptive tests and teaching aids is to give pupils assignments at the correct level, so that pupils are either able to learn more (teaching aids) or demonstrate what they know (tests). At the same time, there are some issues concerning the adaptivity of digital tools in terms of differentiated instruction. This is partly due to the fact that differentiations in digital systems are based on algorithms processed by computers, while the idea of differentiated instruction is primarily that the teacher – and the teacher’s relationship with the pupil – is key to this task. In some cases, the objective of algorithms is to do something that the teacher also does, e.g. tell the pupil which problems to solve and in what order. Many suppliers stress that adaptive systems can free up time for teachers. Teachers also use this as an argument for utilising such tools (Baker et al., 2017).

Preconditions for adaptive systems to contribute to differentiated instruction

In order for adaptivity to have a positive impact on differentiated instruction, teachers must be in proximity to their pupils when working with adaptive teaching aids. Studies indicate that pupils learn very little when teachers mostly let them work on their own with such systems. It is therefore recommended that these teaching aids are included as part of a more comprehensive instruction (McTigue et al., 2020; McTigue and Uppstad, 2019). According to Statped – National Support System for Special Needs Education, (2022), this is unfortunately not currently the case: “The intention of the developer is usually for these teaching aids to be used as part of the instruction. In practice, many pupils sat alone in front of a screen, and the teaching aids were used for most of the instruction without teacher follow-up” (p. 2).

7.3.3 Motivation

According to the Core Curriculum, schools should stimulate motivation, and throughout their education, pupils should be given challenges that promote the desire to learn (Norwegian Ministry of Education and Research, Chapter 3, 2017). Curriculum texts regarding assessments for school subjects emphasise the same.

Adaptive systems have the capacity to strengthen motivation through both the adaptations themselves and through certain elements related to the adaptivity, such as advanced reward systems. Good adaptivity will ensure that pupils receive assignments that they have the ability to complete, and the pupils are then motivated through their accomplishments. Skolelederforbundet (2022) [Norwegian School Heads Association] also emphasise mastery and motivation as a benefit of adaptivity and learning analytics. “Learning analytics can help give certain pupils entirely differentiated assignments to strike the proximal development zone with greater accuracy. Similarly, customised and entirely differentiated instruction can create motivation and a strong belief among the pupils in their own abilities” (p. 1).

Mastery is a key part of motivation theory, where it is essential that the task is neither too easy nor too difficult. But although mastery is essential, it is not always sufficient to promote motivation. One example of this is from an article with a literature review of studies comparing linear and adaptive tests, where the adaptive tests showed no showed no significant effect on motivation (Akhtar et al., 2022). In contrast, one study found positive effects on motivation when the adaptivity was reduced to a lower level, where the pupil was given slightly easier tasks than those provided through standard adaptive systems. The same study indicates the importance of giving pupils sufficient information on how the adaptations are made, since adaptivity also implies that pupils with a higher level of performance are given tasks that are more difficult than the ones they receive in linear systems. These pupils would therefore find the adaptive system more difficult than the one they are accustomed to.

Preconditions for adaptive systems to contribute to motivation

In order for adaptive systems to stimulate motivation and mastery, it is necessary to know precisely how difficult a task is, or to have other precise information as a basis for the adaptations. This part of the development work would be costly and demand a high level of competence. When the foundation for adaptivity is poor, there would naturally be significant limitations on the adaptivity itself. This challenge also applies to linear tools. However, the scope of the challenges is broader for the adaptive tools because one would have to have several functioning pathways in an adaptive learning aid or test – not just one.

7.4 The value of data-supported quality development

In its work, the Expert Group has primarily focused on learning analytics that aim to enhance learning for certain individuals or groups, and where the measures are implemented close to the learning situations. Another form of learning analytics is aimed at the organisational level is known as institutional analytics. In this sub-chapter, we refer to the use of this form of learning analytics as data-supported quality development. The aim is to support decisions on quality related to the conditions and adaptations for learning and the design of the learning environment. This form of learning analytics uses aggregate data from learning situations, often in combination with several relevant sources. It is the management and owner level at the schools and education institutions, as well as local and national authority levels that use the information from such learning analytics.

Data-supported quality development will be relevant for all education institutions, regardless of the form and level. In this subchapter, the Expert Group has primarily chosen to discuss data-supported quality development in higher education and tertiary vocational education. The reason is partly that these sectors believe this potential is relevant: “Learning analytics can contribute to a more holistic and coherent design of courses and study programmes” (Sikt, 2022, p. 2). Another reason is the ongoing work of the Committee for Quality Development in Schools, whose mandate is to survey needs and propose changes to tools and data sources that will facilitate quality development in primary and secondary education.

7.4.1 Better quality of education

In the new long-term plan for research and higher education, high quality and accessibility is one of the three goals that will help realise the thematic initiatives for the next decade (Report to the Storting (2022–2023)). The action plan for the digital transformation includes learning analytics as a possible measure for contributing to better education quality (Norwegian Directorate for Higher Education and Skills, 2022).

The quality report for higher education indicates a number of definitions and interpretations of quality in education Meld. St. 16 (2016–2017). However, it includes the following broad ambitions as a basis for the understanding of high-quality education: “Students shall achieve the best possible learning outcomes and personal development, have access to relevant study programmes to sufficiently prepare them for active participation in a democratic and diverse society and for a future professional career, and complete their education as efficiently as possible” (p. 15). These ambitions are also relevant for high quality in tertiary vocational education Meld. St. 9 (2016–2017).

There is a broad range of data sources that can form the basis for work on quality development. Examples include:

aggregate data from learning situations: activity data from students’ use of teaching aids, platforms and other digital solutions

data from the administrative systems of study programmes and registers which contain, among other things, data on the results of completed coursework requirements and final assessments, such as the Common Student System (FS)[25]

data from the Student Survey[26] and other surveys

7.4.2 Data as a resource

More data is now being produced weekly than data produced in the last millennium. This vast body of data affects the way we do everything from research and product and process innovation to the way we develop organisations, design business models, and how we interact with each other. Meld. St. 27 (2015–2016), p. 101)).

Data as a basis for value creation has been a topic of organisational literature for quite some time, often under headings such as “the data-driven organisation” (Andersen et al., 2018). At the national level, data has been highlighted as the starting point and foundation for the development of the modern society in the white paper Data as a resource – The data-driven economy and innovation Meld. St. 22 (2020–2021). The white paper outlines the Norwegian data policy: “The Government’s principles for data policy should underpin the efficient sharing and use of data within safe and responsible parameters and should ensure that value created from data benefits the private sector, the public sector and society” (p. 8). A separate chapter has also been devoted to data as a resource in the digitalisation strategy for the public sector, which states that data can be better utilised as a resource by the public sector, and that this opens up entirely new methods of solving problems (Ministry of Local Government and Modernisation, 2019).

The Digital Agenda for Norway addresses the large amount of data produced by the education sector, and the importance of utilising this data to improve quality Meld. St. 27 (2015–2016). The white paper highlights the link between data-supported quality development and the good use of technology in instruction. Data from digital resources such as learning platforms, teaching aids, tests and other systems are relevant sources for such quality development.

In the Ministry of Education and Research’s Strategy for digital transformation in the higher education sector utilising data on the knowledge sector is one of the six strategic focus areas (Norwegian Ministry of Education and Research, 2021b). This strategy indicates several challenges in this context: The knowledge and value creation potential of data from the knowledge sector has not been sufficiently utilised. The culture of data sharing has been poorly developed, and there is a lack of common standards for metadata. There are also many data owners in an ambiguous landscape, and the benefits may often appear in places other than where efforts have been made. One of the strategy’s focus areas has therefore been to establish systems and infrastructure for data capture, sharing, storage and the reuse of data on the knowledge sector. It is both desirable and necessary for enterprises to share and exchange data internally and between themselves, as the combination of several datasets will often provide the basis for entirely new and much broader insights than that provided by a single dataset.

7.4.3 The use of aggregate data

In contrast to learning analytics that occur in close proximity to learning situations, data-supported quality development is based on aggregate data that is not directly identifiable. This means that the risk to privacy is generally low for this form of learning analytics. The risk to privacy is also lower because measures are directed at groups or systems for education and not at the individual level.

One example of quality development work with the aid of aggregate data could be exploring various questions about what promotes good learning. This may involve questions about the type of learning platform functionalities, collaboration solutions and learning resources that could contribute to a higher quality of education. Similarly, questions about which learning activities and assessment methods, and which types of assignments and coursework and the order of these would be relevant to explore on the basis of aggregate data. In its input report to the Expert Group, the BI Norwegian Business School explained how they use data analyses on the use of video instruction to obtain a stronger foundation for decisions on the continued use of video at the institution (BI Norwegian Business School, 2023).

Preconditions for data to contribute to better education quality

A good use of data within and across higher education institutions requires an appropriate infrastructure and holistic system architecture (Ministry of Local Government and Modernisation, 2019). This is referred to as a “shared digital foundation” in both the Ministry of Education and Research’s Strategy for digital transformation in the higher education sector, and in the action plan for this strategy (Norwegian Ministry of Education and Research, 2021b; Norwegian Directorate for Higher Education and Skills, 2022). The action plan gives the modernisation of a shared student system a key role in the development of the digital foundation. Data from different systems must be open, i.e. accessible in a form and format that makes it possible to share the data with others and to collate them.

7.5 Summary of the Expert Group’s assessments

The research summarised by the Expert Group in the first interim report indicates a number of potential benefits of learning analytics for pupils and students, for teachers and instructors, and for those who have a responsibility to ensure that schools and education institutions offer high-quality education. One important pedagogical value of learning analytics involves having a clear and systematised insight into the academic development of pupils and students. It is difficult to determine the greatest value of learning analytics in a Norwegian context, but we would like to emphasise three areas that we believe are particularly relevant.

Firstly, we believe that learning analytics has a strong potential to enable pupils and students to gain greater insight into their own learning during their learning processes. As mentioned, this requires the collection of relevant data from the pupils’ and students’ work in their subjects, and that data analyses are effectively communicated to pupils and students. Secondly, we have assessed that teachers and instructors would have much better opportunities to adapt their instruction if they have the sufficient and relevant information to assess their own teaching. Learning analytics can assist with this. The third area where we believe learning analytics could have a significant potential over time for improving instruction and promoting learning is in the work on quality development in schools and education institutions.

8 What pedagogical and ethical challenges are associated with learning analytics?

In the Expert Group’s first interim report, we explained the four primary dilemmas related to learning analytics:

teachers’ and instructors’ need for information about pupils and students to support learning, balanced against the protection of pupils’ and students’ data.

how learning analytics affect the balance between learning through interaction and learning as an individualised process

the balance of centralised support and autonomy when drawing conclusions on learning analytics

the balance between the competence of teachers and instructors required by learning analytics and the actual competence of the education sector

Based on these discussions, the status description in chapter 3 and the input we have received along the way during our work, we have identified a few areas where learning analytics either augment existing challenges or introduce new ones. In this chapter, we will point out specific pedagogical and ethical challenges that we believe are necessary to address. as well as our assessment of these.

8.1 Restriction of content and working methods in instruction

One important pedagogical challenge discussed in the first interim report is the risk that learning analytics may contribute to a restriction of the content and working methods in education. To summarise, this involves concerns that learning analytics may lead to an increased use of individual work methods and less emphasis on the more exploratory and reflective parts of study subjects. The input we have received also points out that learning analytics will appear less relevant when there is a need to apply competence to more complex problem-solving tasks (Norwegian Association of Graduate Teachers, 2023).

Save the Children Norway (2023) has also questioned whether the extensive use of learning analytics in schools “may not be line with what “Fagfornyelsen” [Curriculum Renewal] would entail – that pupils would have a greater opportunity to explore topics in more depth, experience a higher degree of participation, have a more practical approach to subjects, and to work across subjects on topics” (p. 5). They also believe there is a risk that learning analyses may break with some of the values and principles in the core curriculum and with many of the competence aims stated in the subject curricula.

8.1.1 Restrictions to subjects and competence

The Expert Group notes that there is widespread concern that learning analytics could draw more attention to “what can be counted or measured”:

There is a major concern that learning analytics are being developed on a foundation that is too narrow to be measured or adapted to pupils’ learning. We should reflect on how the concept of competence in the LK20 National Curriculum will be addressed in learning analytics. How, for instance, will problem solving, critical thinking and other aspects be reflected in learning analytics? Skolelederforbundet, 2022, p. 2) [Norwegian School Heads Association]

The concept of competence in Norwegian schools states that competence is “the ability to acquire and apply knowledge and skills in order to master challenges and solve problems in both known and unknown contexts and situations” (Norwegian Ministry of Education and Research, 2017, Chapter 2.2). Furthermore, it is emphasised that competence entails both the understanding of and ability for reflection and critical thinking. Our experience with current tools and practices indicates that it is unlikely that information from learning analytics will be able to cover the breadth of the Norwegian concept of competence in the near future. We believe that it is more likely that the analyses will provide information about pupils’ abilities to acquire and apply their skills and knowledge rather than their abilities for reflection and critical thinking. At the same time, rapid developments in the field of artificial intelligence have made it more difficult to predict the potential areas of use for information from learning analytics in the near future. Certain skills and areas of knowledge would also be more suitable for learning analytics than others: “Also in individual subjects, we see that learning analytics may be sensible for certain partial subjects and not for others (e.g. in mathematics, where computational problems are fine, but it would be less appropriate for more complex problem solving)” (Norwegian Association of Graduate Teachers, 2022, p. 3).

Academic areas with content that can be divided into clear and measurable areas of knowledge and skills, and where there is an algorithm that can determine whether the answer fits, is significantly easier than offering learning analytics to someone in more open academic areas that are more suitable for exploration. In mathematics, several learning resources are currently planning to use learning analytics, for instance, within the four basic arithmetic operations, algebra and geometry. We have yet to see a meaningful interpretation of data associated with more inquiry-based areas of the subject, such as creative solutions to open-ended problems. This may also be related to the type of data that is primarily forms the basis for learning analytics in today’s resources. Many digital teaching aids and resources currently contain a large number of closed problems. These are problems where all questions are followed by a limited number of pre-defined answers, such as multiple choice or assignments where different words, numbers or images must be placed in a certain sequence. The Expert Group does not believe that closed problems are problematic in themselves, but it does see the need to continue to explore other methods of designing assignments and measuring competence. Using technology to practice simple skills and knowledge may also have an educational value, but this should not be given too much room in instruction.

There is a market-driven development of learning technology in Norwegian education, something the Norwegian Privacy Commission emphasises in its report (NOU 2022: 11). This means that developers and suppliers will largely prioritise areas of a study where it is both academically and technically simple to develop digital resources. The Norwegian Association of Graduate Teachers (2023) also points out that the freely accessible learning arena NDLA[27] limits the selection of good learning resources in upper secondary instruction for marketing reasons. The Expert Group therefore believes that we need national schemes that stimulate the development and purchasing of digital learning resources that reflect the breadth of school subjects throughout the education pathway.

Furthermore, the Expert Group is concerned that current digital learning resources are nearly always offered in Norwegian Bokmål. The National Parents’ Committee for Primary and Secondary Education (2022b) shares the same concerns:

Many pupils in Norwegian schools are multilingual or have various challenges that entitle them to special education instruction. The situation regarding physical and digital teaching aids is challenging, as publishers and business models do not take such things into account. No do they consider statutory rights regarding Sámi, Bokmål, Nynorsk and universal design. What impact will language and linguistic variations have on our pupils with the utilisation of DLA (learning analytics)? (p. 2)

Save the Children Norway (2023) has also expressed concerns about this: “We are concerned that DLA (learning analytics) do not capture pupil diversity in a positive sense, and we question whether DLA may possibly reinforce differences among the pupils inappropriately. We would particularly like to emphasise Sámi pupils’ right to instruction in the Sámi language” (p. 14).

It has been well documented that the scope of and access to teaching aids and learning resources available in Sámi and Nynorsk are not good enough Prop. 57 L (2022–2023). It is therefore reasonable to assume that there is no real access to learning analytics for Sámi or Nynorsk. This is regrettable, given the requirements set out in the regulations and the rights of the pupils. Sections 6-2 and 6-3 of the current Education Act stipulate that:

In the Sámi districts, everyone of primary and secondary school age has the right to learn the Sámi language and the right to be taught in Sámi. Outside the Sámi districts, at least ten pupils in one municipality who wish to learn the Sámi language and be taught in Sámi have the right to such instruction as long as there are at least six pupils left in the group. […] Outside Sámi districts, Sámi pupils of compulsory school age have the right to instruction in Sámi. […] Sámi pupils in upper secondary schools have the right to instruction in Sámi.

Given these rights, there must be a sufficient amount of varied teaching aids and learning resources in the Sámi language. In its investigation of Sámi pupils’ right to learn Sámi and be taught in Sámi, the Office of the Auditor General concluded that the lack of Sámi teaching materials diminishes educational instruction services for these pupils Dokument 3:5 (2019–2020).

With respect to Nynorsk, section 9 of the Education Act states that teaching aids must be available in both the Bokmål and Nynorsk dialects at the same time and for the same price. This is referred to as the parallelism requirement. In the consultation for the new Education Act, many consultation bodies have mentioned that digital resources other than teaching aids, such as writing programs, should also be covered by the parallelism requirement. In the legislative proposal on the new Education Act, the Ministry’s assessment and proposal is to maintain that the parallelism requirement should only cover that which can be defined as a teaching aid. However, it also proposes a new rule that writing programs should support both Bokmål and Nynorsk Prop. 57 L (2022–2023).

The Expert Group believes that special attention must be paid to the development of learning resources in both Sámi and Nynorsk to ensure that schools are able to fulfil the statutory rights of the pupils.

8.1.2 Less varied and more individualised working methods

Throughout their educations, pupils and students will encounter varied and inquiry-based working methods. Exploratory or inquiry-based learning and competence has also been significantly emphasised in many subjects in the National Curriculum LK20/LK20S. The Expert Group has assessed that it is difficult to see how learning analytics can strengthen an inquiry-based approach to learning with the resources and digital teaching practices we have today. It is essential that the scope of variation and inquiry-based working methods are not reduced in practice by enabling the collection of data for learning analytics to determine the type of learning activities pupils are offered and participate in. The Norwegian Association of Graduate Teachers (2023) has posed a question on this particular point: “How can we facilitate a sensible use of learning analytics without simultaneously implying, for instance, guidelines suggesting that we choose digital rather than analogue teaching aids?” (p. 2).

We have seen such unintended changes in a Norwegian context earlier with respect to digitalisation. The fact that there is increasingly more individual work in Norwegian classrooms (Gilje et al., 2020), is not the result of intended change, but rather that the one-to-one access makes it more “natural” for each pupil to log in with their username and work individually. Variations of working methods also means that large areas of learning must take place through interactions with others. The Norwegian Association of Graduate Teachers (2023) ask: “Will the opportunity for digital learning analyses lead to an increased use of digital, and especially adaptive teaching aids, and what may this increased use of adaptivity do to the learning community of the classroom?” (p. 3). If learning analytics are to counteract the predominance of individual working methods, which we have seen signs of in the fully digital classroom, this would place substantial demands on both digital teaching practices and learning resources (Blikstad-Balas and Klette, 2020).

The opportunity to choose freely between different academic resources is not currently particularly widespread in either primary and secondary education, the vocational college sector, or in the university and university college sector. In primary and secondary schools, it is becoming increasingly more common to purchase licenses for entire “package solutions” where a supplier delivers resources for all relevant subjects (Rambøll, 2023). The Expert Group believes that this is a solution which, at best, safeguards totality at the expense of flexibility. We believe that teachers and instructors should be able to freely choose between a broader range of resources in order to increase the opportunities for local adaptations and variations. This would require teachers and instructors to have the time, competence and capacity to learn how to use them. It would also require making sufficient information about how the digital resource has been developed available to them.

8.2 Links between learning analytics and the National Curriculum

The Expert Group has received several suggestions indicating that there must be a clear link between learning analytics and the National Curriculum if learning analytics is to have real value in primary and secondary education. This was also clearly demonstrated in the assessment of barriers to learning analytics in primary and secondary education (see Chapter 3.4.4). In order for digital teaching materials and analyses to function adequately as a basis for decision making for teachers, teachers must be aware of which parts of the National Curriculum the different resources will help develop. A broad range of different resources are currently available in today’s market. However, it is not always clear how well the various resources harmonise with the National Curriculum or the values of Norwegian schools. In Norwegian-produced digital teaching aids that are tailored to the various levels and subjects, it can also be challenging to know which parts of the National Curriculum the resources are meant to contribute to. There are many contexts in the National Curriculum, both between core curriculum and the subject curriculums, and between subject curricula. Linking digital resources to the National Curriculum is therefore a complex process.

We fully understand why teachers and school leaders wish to learn more about which parts of the National Curriculum a resource with a functionality for learning analytics would be aimed at before they begin using it. Exactly who will be doing the work to ensure coherence between the National Curriculum and the digital resources, and whether it is at all possible to link all the digital resources to competence aims remains unanswered. We also mention the latitude among teachers and at the local level for interpreting these links in the National Curriculum and how the schools wish to put this into practice.

8.2.1 Link to competence aims

It has long been common among Norwegian schools for different teaching aids to signalise both the subject, year and academic topics they profess to cover (Askeland et al., 2013; Tønnesen, 2013). Several teaching aids developed after the introduction of the National Curriculum for Knowledge Promotion in Primary and Secondary Education and Training (Kunnskapsløftet – LK06/LK06S) have also included different competence aims in the National Curriculum and linked these, for example, to different chapters in the teaching aids. Textbooks have therefore long been viewed as an interpretation of the “curriculum’s perspective on subjects and knowledge” (Tønnessen, 2013, p. 149).

When applying for a grant from the Norwegian Directorate for Education and Training to develop teaching aids, there is a requirement for teaching aids to be developed for use in instruction and to cover all or parts of the competence aims stated in the National Curriculum. This is because teaching aids in the Education Act are defined as materials that cover significant parts of the subject’s curriculum. In this sense, it is reasonable to expect that digital teaching aids in a given subject also give an indication of the competence in the subject that the supplier believes the digital resource will help to develop. The Expert Group believes it is important for school owners and schools to be given information that provides a basis for selecting and using teaching aids. At the same time, we also see certain issues in linking parts of digital teaching aids – specifically where there is a functionality for learning analytics – to specific parts of subject curricula or competence aims.

For one thing, it is not possible, with the broad concept of competence stated in the National Curriculum, to assume that only one method for measuring and developing pupils’ knowledge and skills is sufficient. A supplier will therefore seldom deliver more than a few concrete suggestions for how one or more competence aims can be measured or worked on. Competence aims should function as targets for a competence that pupils should have the opportunity to develop over time, and pupils develop different knowledge and skills along the way towards reaching these objectives. Therefore, these are not things that can simply be ticked off a list as each is completed. We believe there is a risk that the sector will continue to view them in this way if individual modules involving resources with a functionality for learning analytics are directly linked to specific competence aims. In addition, the competence aims for a subject are related to each other, and to the introductory parts of the curriculum. It is thus beneficial for pupils’ learning to work in larger contexts rather than just one individual competence aim after the other.

Secondly, there will be a need to break up these competence aims into smaller and more fragmented units to create a more appropriate structure for including smaller modules of the teaching aid. Thus, the understanding of the curriculum becomes narrower, which could lead to splitting instruction in the subject into more parts than would be desirable.

The Expert Group believes that instead of linking parts of the teaching materials and analyses to specific competence aims, they should describe the areas of knowledge and skills in a subject that the analytics in question could help develop. We believe it is essential to develop methods of linking the learning analytics to the content in a way that promotes rather than narrows teachers’ didactic reflection and their work on the what, how and why aspects of learning. As with all other learning resources brought into the classroom, teachers must always determine which specific competence aims the learning resource would be relevant for. This method of linking learning analytics to content would also be more flexible considering the future changes in primary and secondary education in the National Curriculum.

The Expert Group specifies that this challenge applies to how the link between the content of the teaching aid and competence aims are presented to the user, and to not the use of metadata that supports the technological functionality of the solution.

8.2.2 Commercial considerations

Furthermore, there is a question of whether there are commercial providers who will be permitted to define parts of their own services that they believe correspond to different parts of the National Curriculum. Although we fully understand that the sector would prefer such overviews, we believe there is reason to warn against permitting commercial actors to be given even greater power to define such aspects for Norwegian schools than they already have. Not only are suppliers focused on pupils’ learning, but also whether developing various types of technology is actually profitable in a market under pressure. We believe there is a real danger that the monopoly tendencies we have already seen in digital instruction will intensify if all suppliers must link all content to the National Curriculum. It can quickly become worthwhile to link as much content as possible to give the impression that they are offering a “complete package solution” – a tendency that is already evident among the major suppliers today, where several have promised to give teachers “everything they need for the new curricula”, and similar promises. For a smaller supplier who is perhaps able to deliver a good product that is only relevant for limited areas of a school subject, this type of linkage could be a disadvantage. The Expert Group believes that suppliers should be required to provide a realistic description of the relevance of their resources to pupils’ development of knowledge and skills in a subject.

8.3 Academic freedom and decisions on learning analytics

Systematic learning analytics require adaptations to be made at a more central level than by individual teachers. At the same time, it must be possible to adapt the use of learning analytics to the nature of the subject, professional judgement and local conditions. This is where difficulties may arise. Since academic and pedagogical freedom is regulated unequally and the sectors have varying structures, assessments would also differ between primary and secondary education and higher education.

8.3.1 Academic freedom in higher education

In higher education, academic freedom as a principle remains strong. Academic freedom implies freedom in the teaching role, but with corresponding responsibilities. Section 1-5, fourth paragraph of the Universities and University Colleges Act stipulates that the teacher or instructor has an independent academic responsibility for the content and structure of the instruction within the frameworks set by the institution. The long-term plan for research and higher education specifies that academic freedom does not pose an obstacle for developing measures at the authority level to realise education policy objectives.

NOU 2020: 3 Ny lov om universiteter og høyskoler [NOU 2020: 3 The new Act relating to Universities and University Colleges] summarises a few restrictions on teachers’ academic freedom as follows: “Teaching must, however, be designed to lead students towards the applicable exam or degree. A study plan may also place restrictions on the free choice in the instruction. Teachers will regardless have freedom with respect to the presentation of materials and perspectives” (p. 128).

Making decisions on which learning analytic tools should be made available for teachers would therefore lie within the institution’s frameworks. Today, most institutions have also determined which digital learning platforms teachers must use in their instruction, and these often contain opportunities for certain learning analytics. Apart from requirements for the use of certain learning platforms, the Expert Group is not aware of institutions requiring teachers to use these platforms for learning analytics or to adopt the functionality of learning analytics in other systems. This indicates a practice whereby teachers and instructors make independent decisions within the frameworks set by the institutions.

The Expert Group believes that the way in which each teacher or instructor performs learning analytics within these frameworks falls under their academic freedom and responsibilities. This entails an assessment of academic presentations and working methods that are most appropriate for certain courses. However, relevant competence is required for assessing the learning analytics. Universities Norway (2023) sees the need for a collaboration in the sector on teaching resources and information about the possibilities, limitations and risks of learning analytics. The organisation points out that if common guidelines are to be drawn up, the sector must be involved, e.g. through Universities Norway’s bodies.

8.3.2 Methodological freedom in primary and secondary education

The current competence-based National Curriculum has not set guidelines for specific working or teaching methods (Norwegian Ministry of Education and Research, 2017). The way in which teachers plan and conduct their instruction is therefore a professional responsibility.

NOU 2015: 8, which is the foundation for the revision of the National Curriculum in 2020, specifies that the freedom to choose methods must be based on well-founded and research-based decisions:

[…] teachers have a professional responsibility to choose subject content, ways of working and organisation that are based on research relevant for pupils’ learning and adapted to the particular group of pupils. This means that teachers’ professional autonomy involves a responsibility for making well-reasoned and research-based choices of methods and approaches in their teaching. (p. 78)

The Expert Group notes that these professions are adamant that learning analytics must fall under professional autonomy. “Teachers must themselves be able to choose the situations in which they want to incorporate learning analytics as part of the planning work for the instruction, based on an assessment of subjects and learning situations to which this would provide real added value, and for which groups of pupils” (Norwegian Association of Graduate Teachers, 2023, p. 2).

We emphasise that the assessment from the first interim report stating that the decision to use resources with a functionality for learning analytics should be made as close as possible to where instruction is taking place. At the same time, we realise that there is a high demand among teachers for guidance and support in making these pedagogical decisions. Many teachers have expressed uncertainty about how learning analytics work, when and how they would be appropriate for teaching and learning, and what is needed to ensure that learning analytics are responsible. The Expert Group believes there is a strong need to ensure a better overview of resources with a functionality for learning analytics that is available for primary and secondary school instruction, and to develop good support structures for assessing the relevance and quality of the resources. We emphasise that it is essential to have national frameworks that can ensure that all learning analytics in Norwegian schools are performed in accordance with legal and ethical frameworks that protect the privacy of pupils to the greatest extent possible. In addition, teachers must have the necessary competence to make good pedagogical decisions on learning analytics.

8.4 When decision support becomes the decision

In the first interim report, we asked whether information from learning analytics could be perceived as more authoritative than other information that teachers have about pupils’ and students’ learning. We are concerned that digital presentations and visualisations based on automatic calculations may have too great an influence on pedagogical decisions. This concern was also expressed in the input from the sector: “Experience has shown that teachers become more passive and place too much trust in data/results obtained from digital learning” (Møre og Romsdal County authority, 2022, p. 1).

The Norwegian Data Protection Authority (2022b) has also noted the risk that learning analytics can become an automatic decision-making system, even if it is not intended as such:

Another risk is that the system in practice could be used as an automatic decision-making system, even if it is not intended as such: In other words, some teachers may accept recommendations from the system without making independent assessments. This may be due to a heavy workload, insufficient knowledge of the algorithm, insufficient insight into how the system works, etc. We can also imagine a situation where recommendations from the system are perceived to be so good that teachers do not feel they can disprove the system. (Chapter 4)

For learning analytics, the issue would be particularly relevant in the delineation between decisions on adapting instruction and decisions on pupil and student learning. We will illustrate how information from learning analytics may function as a decision rather than as decision support by describing two examples.

8.4.1 Decision support becomes a measurement of competence

Let us take a hypothetical example: A class in year 9 uses a fully digital learning resource for the subject of Norwegian, which includes a multiple-choice test for the genre of short stories. If the teacher uses the results systematically and looks at how continued instruction and the follow-up of an individual pupil should be adapted, this would be in accordance with good learning analytics used for decision support. For the teacher, the test may have contributed important information about what pupils have learned about the genre of short stories, as well as what the teacher needs to spend more time on before the pupils can write or analyse their own short stories. Here, learning analytics have supported the teacher in their future work. Learning analytics alone have not “decided” anything, but rather provided information about future decisions. Yet we can also imagine another possible use of the same data.

If the teacher, who is about to make a half-year assessment or coursework grade in the subject of Norwegian, only months later goes back and focuses on the results of a test for a specific pupil as documentation of the pupil’s competence in the short story genre, the analysis is used as a decision regarding the pupil’s competence at a specific point in time. In this case, the test score does not provide information about future work. Instead, it has become a measurement of what the pupil knows or does not know about short stories.

The Expert Group believes there is reason for concern that the information from learning analytics can become a decision rather than a support for the teacher. This may present challenges in that a number of digital resources today offer results and overviews of pupils’ academic answers, but it is not made clear whether this is intended as a basis for adapting the instruction or as a measurement of what the pupil has done. It reinforces the problem that several suppliers suggest that they can provide an “overview” over what pupils know, understand, need to practice, or similar formulations.

The Expert Group also believes it is important for pupils and students to be aware of whether the data collected on their learning is meant to provide insight into their learning processes and their academic progression along the way, or whether the data will be used to document their competence. In order to provide this information, suppliers must state which of the two objectives the resource will help to achieve.

8.4.2 Narrow analyses are broadly interpreted

Another example of how learning analytics may be given too much emphasis: If a pupil repeatedly displays an average achievement of objectives in various parts of a digital teaching aid in a subject, there is a risk that this will contribute to preventing the pupil from being assessed as anything but average in this subject. There is therefore a risk that the measurable aspects of the subject, where there is an opportunity for learning analytics, would have disproportionate importance in the overall assessment of the pupil’s total competence. This is in itself problematic, as there are no teaching aids that can measure all aspects of a subject. If the digital teaching aid is meant to cover all aspects of subject’s curriculum, it would still be problematic to lean on information from the learning analytics for decision making, as pupils have the right to demonstrate their competence in several different ways[28] . When learning analytics form the basis for an assessment, it would be necessary to comply with the principles of fairness and accuracy stated in the General Data Protection Regulation to ensure that pupils’ and students’ overall competence are assessed correctly.

In order for learning analytics to function as intended, teachers and instructors must have sufficient prerequisites to assess the information from the analyses and draw independent conclusions based on their professional and academic judgment. Among other things, this would require suppliers to make information available about how the factors being measured are part of a greater whole. Union of Education Norway (2022) believes it is especially important for teachers to have the appropriate prerequisites for assessing adaptive teaching aids: “It is […] essential that teachers are enabled to assess functionality and the database/data sources for each adaptive teaching aid in order to determine what teaching aid says about a pupil’s/student’s academic level” (p. 1).

8.5 Data that is irrelevant, misleading or difficult to interpret

Through the use of digital learning resources, data is collected on pupils and students that may be irrelevant, misleading or difficult to interpret. If learning analytics are to be accurate and relevant, it is necessary to have data that provides information about learning and analyses that draw the correct conclusions. In our discussions in the first interim report, we mentioned examples of mechanisms that challenge these conditions, such as collaboration using shared logins, as well as pupils and students who manipulate the systems.

In this context, we are concerned with the quality of the data, and not necessarily the quantity. It is important to emphasise that although the amount of data may be relevant for performing good learning analytics – especially if it involves machine learning – more data would not in itself imply a higher quality analysis. The supplier Neddy (2023) mentions this in its input to the Expert Group:

Yes, learning analytics require a continuous collection of activity data, but do good learning analytics really require large amounts of data? We believe that this discussion must primarily focus on what can be described as high-quality activity data, i.e. more accurate data, and whether this would increase the quality of the analysis. What if we primarily think qualitatively and not quantitatively here, before concluding that more data equals better insight? (p. 16).

In the following, we have highlighted certain aspects that give cause for concern about the quality of data and analyses. All aspects are also scenarios that indicate a risk of failing to comply with the principles of fairness and accuracy in the data protection legislation.

8.5.1 Data as a basis for uncertain inferences about learning

One example of data that is difficult to interpret is data on time expenditure, which tell us that a student has spent an unusual amount of time on an assignment. This may imply that the student has found the assignment difficult. However, there may also be several other explanations. For instance, the student may have taken a break and done something else while time was automatically being measured, or the student may have found the assignment so interesting that they decided to look up other resources to learn more about it. Pupils have also expressed this in their discussions with the Expert Group: “It’s okay that you can see what we’ve answered, but not how much time we spent on it. We may make mistakes, take breaks, and so on. So that would be completely wrong” (pupil in year 9).

Similarly, it would be difficult to determine what the number of attempts a pupil needed for solve a problem before it was correct says about the pupil’s learning. Pupils that the Expert Group have spoken with explain that they may have correctly understood and correctly used a calculation strategy but still repeatedly gotten a wrong answer as they made a small error in their calculation. This is a good example of something any math teacher would have immediately noticed, but that a machine would not necessarily have identified. It illustrates the necessity of having a teacher close at hand when pupils are working in such systems.

The problem here is not that the data provides incorrect information, but rather that it is difficult to arrive at good conclusions about learning based on the data, unless such a relationship has been demonstrated through research and testing. The Expert Group believes that there is a need for a thorough pedagogical discussion on which data should be included in learning analytics, and what conclusions could be drawn on the basis of the analytics in each case. This requires a high degree of transparency from suppliers and developers of resources with functionalities for learning analytics. Teachers and instructors must take an active role in the use of learning analytics to reduce the risk of incorrect conclusions. We emphasise that responsible learning analytics would primarily involve a teacher’s professional assessment of the conclusions before they have pedagogical consequences.

8.5.2 Data as a basis for incorrect conclusions

Examples of misleading data include when several pupils are working together to solve problems in the same login, or when pupils use strategies when solving problems that result in misleading data. The Norwegian Association of Graduate Teachers (2023) mentions this in its input to the Expert Group: “Pupils log in with each other’s usernames or they help each other, which leads to incorrect data. Some of the assignments are designed such that pupils can randomly press a key until they get it right” (p. 2).

We have also received input from parents who say that pupils are using strategies for assignments that do not reflect their academic competence. For instance, they may identify a pattern in how the correct answers are formulated, or they may intentionally give the wrong answers to get easier problems. We are also aware of instances where pupils using resources with strong reward systems begin collecting as many points as they can rather than solve the assignment problems.

Adaptive teaching aids are adjusted according to pupils’ earlier answers. If a pupil has received help or has collaborated with others when working with the teaching aid, future assignments may have an artificially high level of difficulty. This level will gradually readjust itself to the pupil’s level if the pupil continues working alone. However, the pattern of the pupil’s answers may be interpreted incorrectly by the algorithms of the teaching aid. This is especially applicable to younger pupils who receive help from their parents at home. The Norwegian Union of School Employees (2023) ask: “And when the pupils work together or get help at home, what answers do they then get from the adaptive assignments?” (p. 1).

To avoid wrong conclusions, it is important that those who are following up the results of learning analytics are aware of the type of data used and the context from which it was collected. The Expert Group believes there is reason to be critical of the results of the analytics if adaptive teaching aids are used for collaborative assignments with a shared login. We believe that adaptive models must take into account that pupils may work together and be given help. The system must therefore quickly “reset”, and sequences with strong performances must be valued and viewed in context with the progression report. For example, achievement peaks could show what the pupils are capable of when receiving help, as opposed to what they can achieve on their own. In this way, such achievement peaks can provide information about a pupil’s potential.

8.5.3 Does the data really provide information about learning?

The last category of data we must address is irrelevant data. Large amounts of data are collected on pupil and student learning situations for learning analytics, which may have an uncertain pedagogical value. The supplier Neddy (2023) has posed some good questions on the matter:

Today, we see that information about when and for how long an activity is performed is used as an indication of learning outcome. Furthermore, information that is easy to manipulate is used as a basis for personal adaptations. Why is this information currently being used for learning analytics? Is it tradition that is holding us back in this field? Perhaps such data points are not good data points for learning analytics (p. 20).

Time data on when and for how long a pupil or student has been working is easy to dismiss as irrelevant for learning analytics – and one could ask why it is collected at all. Many pupils have also expressed that they feel uncomfortable knowing that teachers have access to information about what time of day they do their schoolwork: “I don’t think it’s okay for teachers to see when you’re working. The most important thing is to hand in your homework, so it doesn’t really matter when you do it” (pupil, year 7).

Pupils and students have also expressed concerns that access to time data in some places may lead to an unwritten “absentee limit” on learning platforms that could conceivably impact their relationship to a teacher or instructor – and in the worst case also play a role in the academic assessment of the pupil or student.

The Expert Group believes that data on what time of day a certain pupil or student does their work would have limited pedagogical value and should therefore not be collected for the purpose of learning analyses. When information about pupils and students is not relevant for pedagogical or administrative reasons, and pupils also express concerns about this information being available, it would also be highly problematic to process the data from the privacy perspective.

8.6 Monitoring and increased stress

The Expert Group devoted considerable space in the first interim report to a discussion on the dilemma of the need for data as opposed to data protection. One of the challenges brought up in this discussion was the discomfort that many pupils and students experience when an education institution collects, shares and analyses data from their activities on digital devices. Such information flow may affect the relationship and trust between a teacher/instructor and a pupil/student. Many students and pupils have expressed that continuous data collection has given them a feeling of being under constant evaluation. This leads to greater pressure and stress for many young people. Save the Children Norway (2023) has called for a public debate on this issue:

Save the Children Norway calls for a public debate on the use of personal data for learning and development in schools, which is not only linked to sectoral goals for learning outcomes, but also raises more ethical questions about children’s autonomy, their right to privacy, and their right to speak, write and play freely without having their activities registered, shared and used, even if this could be beneficial for pupils’ learning and at the aggregate level for the education sector or other public services (p. 4).

8.6.1 Fear of trial and error

Pupils and students should feel secure in being able to try things, even when they are uncertain as to whether they will succeed. They should feel willing to tackle challenging tasks as this can enhance learning – even if they make a few mistakes along the way (Kapur, 2008). When traces of the learning process are saved, this may make some pupils unwilling to try and fail, and they may feel uneasy knowing the teacher has access to this information:

I think it’s difficult knowing the teacher will see everything. We feel insecure and afraid of making mistakes. You should have the chance to make mistakes without someone seeing them. You may understand it later, and then maybe the teacher won’t notice. It’s like if you were to hand in a draft every time you write something. Then the teacher can see everything you’ve done wrong before you figure it out. What could happen is that teachers don’t focus on your end product, but instead on the mistakes you make along the way (pupil, year 9).

In the interim report, we discussed the risk that the willingness of pupils and students to try and fail may be impacted by having data from digital learning activities saved and used for learning analytics. The Expert Group notes that there are concerns in the sector that learning analytics may lead to changes in pupils’ behaviour for this reason. The Norwegian Union of School Employees (2023) gives examples of pupils who have written down vocabulary words from an app for language learning instead of using the potentials of the app to practice – because they didn’t want the teacher to see when and how often they practiced for a vocabulary test. In a privacy context, this type of altered behaviour is referred to as a chilling effect:

The fear of being monitored, or that someone will use information about us for purposes we are unaware of may lead to changes in our behaviour. When we lose control over who knows what about us, we are forced to consider the uncertainty factor. As a result, we may begin to think through and re-evaluate what we write, who we are in contact with and what we do. This self-regulation due to fear of surveillance is called a “chilling effect”. Norwegian Data Protection Authority, 2020a, Chapter 7).

In order to counteract the risk that pupils will change their behaviour when their personal data is used for learning analytics, national authorities must stimulate privacy-promoting functionalities in the resources. Furthermore, it is necessary that this use is included as part of an assessment practice that contributes to a safe and inclusive learning environment. We emphasise that pupils and students must have the opportunity to try and fail during their learning processes without having the school and education institutions save this data and use them for learning analytics. It is crucial that schools, vocational colleges and higher education institutions are aware of this responsibility.

8.6.2 The experience of constant assessment

Viken Youth Council (2023) is concerned that learning analytics may contribute to increased pressure on young people if they feel that are in a situation of constant assessment that does not give the full picture of their competence. The Norwegian Union of School Employees (2023) talks of their experiences: “When they [pupils] never know exactly when they are being assessed, or if they feel that are being assessed all the time, this creates an unnecessary level of stress” (p. 1). Save the Children Norway (2023) formulates this as follows:

There must be an evaluation of what part of the pupils’ work that can be assessed and what part does not need to be assessed. With the introduction of digital teaching aids and tablets in Norwegian schools, pupils may feel as though they are always being assessed. Pupils need a break from these continuous assessments. It is important for all pupils to have the courage to express their opinions and participate in the exchange of opinions, and to explore various dilemmas in a secure community of their teachers and fellow pupils (p. 13).

The National Curriculum points out that “schools and teachers must balance the need for good information about pupils’ learning and the unwanted consequences of various assessment situations” (Norwegian Ministry of Education and Research, 2017, Chapter 3.2). According to the National Curriculum, consequences of the unfortunate use of assessments are that it may prevent the development of a good learning environment, and that it may negatively affect the self-confidence of certain pupils. Concerns about systems “remembering all mistakes” were well-articulated by young people at an input meeting the Expert Group held with Sentralt ungdomsråd i Oslo [Umbrella Youth Council of Oslo]: “If you say something stupid during a class, the teacher will forget about it. It’s a lot worse if these mistakes are saved so that the teacher never forgets stupid mistakes”.

This comment contains an important point about the difference between humans and machines. A teacher can systematically disregard mistakes that are of no importance, interpret context (e.g. understanding that a pupil is having a particularly difficult time just before a test), and distinguish between significant and insignificant indications of what a pupil is capable of. This ability is also part of the reason for the system of classwork grades in primary and secondary education and training, where subject teachers are all given substantial autonomy precisely to emphasise what they believe is significant information about pupils’ competence. It is essential that teachers have a wise approach to how they emphasise information from learning analytics, and that there are always people involved who can actually ensure all pedagogical aspects of the instruction. This approach must also be applied to pupils to ensure that they do not feel that “everything they do” can be used against them when determining grades.

Nor should students in higher education and tertiary vocational education have to worry about whether data being collected today – especially data obtained from their activities on learning platforms – will affect them negatively during their education or in the future. In the vocational college sector, student associations have expressed concerns about the close connections between study programmes and the industry. The National Union of Students in Higher Vocational Education and Training in Norway (2022) points out that students may feel it is invasive when analytics from their use of the learning platform are followed up by the education institution. Students fear that information about their learning activities and their behaviour may be passed on to the labour market and impact their future recruitment and employment relationships.

The Expert Group believes that this must be assessed thoroughly on principle to determine what type of data can be used for learning analytics. More knowledge is also needed on how continuous data collection and analytics may impact pupils and students. Here, we see a clear need for more practice-based research on the extent of the learning analytics and the actual impacts that the collection, analyses and further use of data has on pupils’ and students’ experience of teaching and instruction, and on their learning outcomes.

8.7 Random and planless use of data

One of the main challenges noted by the Expert Group is that large amounts of data are currently being collected from digital learning resources for undetermined educational purposes. For pedagogical, ethical and privacy reasons, this is questionable if, in connection with learning analytics, data is collected without a plan for how the information will be used to follow up pupils and students. Much of the input we have received stresses that there are few systematic learning analytics in today’s primary and secondary education, higher education, or tertiary vocational education. In the survey of learning analytics in primary and secondary education and higher education, we have seen that this practice currently appears to be ad hoc and unmethodical (Rambøll, 2023). Nor have surveys identified municipalities that are working systematically with the functionality of learning analytics (FIKS, 2023).

8.7.1 Run by enthusiasts

Random and planless practices with learning analytics may be due to the problem that its use relies entirely on so-called digital enthusiasts, something we also problematised in the first interim report. Primary and secondary education has long relied on teachers with a particularly strong interest in digital competence and specific digital tools (Egeberg et al. 2017; Gudmundsdottir and Hatlevik, 2018). How teachers incorporate digital technology in their instruction, and to what extent, has in many cases been up to each individual teacher. This is unfortunate, as it leads to inequalities in the instruction, with varying opportunities for pupils to develop their digital competence.

There is a similar issue with respect to learning analytics. As Rambøll (2023) summarises, learning analytics appear to be “entirely dependent on teachers and school leaders with a special interest in these tools” (p. 14). Several informants have stated that they must use their leisure time to acquire and learn about relevant resources, and to develop a teaching plan where the resources can best be employed. We emphasise that a good, systematic learning analysis requires sufficient competence, time and capacity to become familiar with the use of these resources.

8.7.2 Lack of knowledge

One of the reasons for the random practices with learning analytics may be a lack of knowledge in the sector. The Expert Group has noticed that not only is term ‘learning analytics’ unfamiliar to many, but also its definition: “Currently, the use of learning analytics is not particularly systematic or evidence based. Many may not know what it is” (Skolelederforbundet, 2022, p. 1) [Norwegian School Heads Association]

As the Universities Norway (2023) points out, the low prevalence of learning analytics may be due both to a lack of knowledge and access to tools and opportunities. It is understandable that little knowledge will result in little use, although this is not necessarily a problem in itself. The issue is that the existing use is not systematic, which makes it difficult to build a good evidence and knowledge base on learning analytics. Another challenge is that resources with a functionality for learning analytics are already collecting large amounts of data that are not used for good pedagogical purposes. The Expert Group believes that there is a strong need for a clear and explicit framework to determine what type of information that can be collected, and for what purpose. We also believe that teachers and instructors need greater competence in learning analytics to ensure that they have the sufficient ability to make pedagogical decisions based on the learning analytics.

8.8 Summary of the Expert Group’s assessments

Research and input from various actors indicate several barriers to good learning analytics. We are especially concerned when pupils and students say that learning analytics make them feel as though they are constantly being monitored and that they do not have enough freedom to make mistakes. We also believe there is a danger that the resources used to collect and analyse data could themselves lead to more individual work, and that this could narrow down the subject content and working methods.

Another issue is the interpretation of the analytics. It is highly problematic that much of the data collected for learning analytics have an ambiguous pedagogical value and can be misleading or difficult to interpret. This problem is exacerbated by the fact that analytics intended for decision support for teachers and instructors are in many cases perceived as authoritative and become decisions instead. We have assessed that although the research does indicate that learning analytics have promising pedagogical advantages, we are still far from a practice in today’s schools and education institutions that outweighs the disadvantages.

9 Participation in learning analytics

Pupils and students have the right to participate in matters that concern them. This is emphasised in several current regulations and is considered a key value of the Norwegian education system. Learning analytics use digital traces from pupils and students as a source and may provide important insight into their learning. Both aspects of learning analytics concern pupils and students to a large extent.

On the one hand, learning analytics can create better conditions for the participation of pupils and students, partly by increasing their insight into their own learning, which can make them better equipped to answer questions about learning and teaching practices. On the other hand, learning analytics can also make it more challenging for pupils and students to participate and contribute if they are not given the opportunity to gain such insight and an understanding of how this concerns them. In this chapter, the Expert Group will describe the right to and experience of participation in education, and we will cite the input we have received from pupils, parents, students and their representatives. We will also assess how participation should be safeguarded in learning analytics.

9.1 Pupil participation

Pupils have the right of participation according to several of our current regulations, such as the Norwegian Constitution, the Education Act, and the Independent Schools Act. A pupil’s capacity to ensure their own interests and to participate develops over time with age. This applies to both their own learning and privacy issues (UN Committee on the Rights of the Child, 2021). The Education Act and its regulations determines frameworks and provides guidelines for pupil participation that applies to all teaching and instruction. According to the National Curriculum, pupils shall have the opportunity to participate in decisions regarding their own learning, and to participate actively in the assessments of their own work and their own competence, as well as their academic and social development. Schools shall promote a commitment to democratic values, and pupils shall experience democracy in practice through their schooling. This means that they must have influence and be able to have a say in matters that concern them through various forms of participation and contribution.

Knowledge of pupil participation is limited. Utvalget for kvalitetsutvikling i skolen [The Committee for Quality Development in Schools] writes in its knowledge base:

Despite several decades with a greater focus on the participation of children and young people, there is limited research on pupil participation in general, and on pupil democracy and pupils’ individual participation in Norwegian schools. For instance, a recently published research summary on youth participation in Norwegian municipalities indicates that there is little peer-reviewed research on this topic, and that young people are rarely given opportunities to voice their opinions. (NOU 2023: 1, p. 156)

A report prepared on behalf of the Norwegian Association of Local and Regional Authorities (KS) and the School Student Union of Norway states that those who have successfully facilitated pupil participation understand that pupil participation is about more than simply making decisions (Faannessen et al., 2022). They recognise that participation is a key part of pupils’ learning and understanding of democracy, and they know that secure relationships are a prerequisite. By facilitating processes to involve pupils and to develop competence in the area of pupil participation among teachers, leaders and school owners, they are also creating a broader and deeper insight into how their own practices must be adjusted according to their developmental needs.

Experience of participation

The Pupil Survey[29] is annual web-based survey on pupils’ school and learning environments. This is conducted at all primary and lower secondary schools in years 7 and 10, and in the first year at upper secondary schools. The purpose of the survey is to give pupils the chance to give their opinions on learning and well-being at their schools. Results of the survey are used by schools, school owners and national education authorities to improve the schools. The results of the 2022 Pupil Survey show that year 7 pupils have expressed, to a much larger extent than year 10 pupils, that they are able to help decide how they will work on different subjects, that their teachers facilitate pupil council work, and that the school listens to pupils’ suggestions.[30] Pupils in the first year of upper secondary school express a higher degree of co-determination in their responses to the same questions than year 10 pupils in lower secondary school. Pupils in vocational education programmes experience a higher degree of participation than upper secondary school pupils in higher education preparatory studies. Although the Pupil Survey provides an indication of the types of opportunities for participation pupils feel they have in Norwegian school, it does not provide detailed insight into what these opportunities imply in practice.

9.2 Student participation

Student bodies have a right under section 4-1 of the Universities and University Colleges Act to be heard in all matters concerning the students. Student bodies at vocational colleges have a corresponding right according to section 14 of the Vocational Education Act. Student participation is rooted in democratic principles, pedagogical considerations and agreements on study programmes. This is required to ensure that students can become active participants in their own learning. This is also emphasised in the white paper on vocational college education, which states:

Strengthening the position of student democracies at vocational colleges will enable students to influence the education environment and the academic content of their study programmes, and to contribute to improving vocational colleges even further. An attractive vocational college means more engaged students who will help determine its direction and development. Meld. St. 9 (2016–2017), p. 7)

Experience of participation

The Student Survey (Studiebarometeret[31] ) is a national survey that is sent out to more than 70,000 students each autumn. The survey asks students for their opinions on the quality of their study programmes at Norwegian universities and university colleges. The aim of the Student Survey is to improve the work on quality development in higher education, and to provide useful information about study programme quality. There are similar surveys for vocational college students.[32]

In the 2022 Student Survey, just under 40 per cent of students responded that they had the opportunity to offer input on the content and structure of their study programme to a high degree (Hauge et al., 2023). It is worth noting that there are substantial differences between the education institutions. Just over 30 per cent responded that they had this opportunity to a low degree. Among vocational college students, approx. 60 per cent answered that they had the opportunity to offer input to a high degree (Øygarden and Stensby, 2022). A similar rate completely agreed that their vocational college facilitated participation through student representatives and local student councils. The Student Survey does not provide detailed insight into the type of participation this entails or what forms of participation the students actually utilise.

9.3 Input on participation in learning analytics

In conversations the Expert Group has had with the School Student Union of Norway, they have said that pupils should be given the same information about themselves that the teacher has. In addition, they emphasise that the pupil council should have access to aggregate data on their school if they are to contribute to better pupil participation. They believe that parents should also receive sufficient information but that they should have less access at the upper secondary level than at the primary and lower secondary level.

The Youth Panel in Møre og Romsdal (2023) believes it is important for youth to use existing meeting places to have their voices heard on the issue of learning analytics. Such places include the School Student Union of Norway and its national conference, Elevtinget, as well as the pupil council, and both municipal and county authorities. They emphasised that pupil surveys and evaluations should be developed that can provide a broader understanding of pupils’ opinions. They also believe that there is a need for a national youth council to ensure pupil participation.

Vestland ungdomsutval (2023) [Western Norway Youth Council] notes that teachers should discuss learning analytics with their pupils in each class: “In this way, teachers will know which learning tools to use that pupils find useful, as well as what information to use and how it should be used for their pupils” (p. 1).

In its input report to the Expert Group, Universities Norway (2023) has specifically expressed concerns about student participation in learning analytics:

Learning analytics can be a useful tool and a supplement to quality assurance work. However, it neither can or should replace student participation. Students must also be included when assessing the types of learning analytics that are needed and when, and therefore what type of data should be collected (p. 1).

The National Union of Students in Higher Vocational Education and Training in Norway (2022) emphasises that it is important for these tools to be used on the students’ terms, and that good student participation in the processes associated with learning analytics is actively facilitated.

9.3.1 Discussions with pupils on learning analytics

Through its discussions with pupils, the Expert Group has received good input, suggestions and ideas on learning analytics. We have spoken with pupils from several classes in different year levels at three schools. See more information about how the Expert Group has involved children and young people in Chapter 2.4.2.

To summarise, some of the pupils say they find it stressful to know that there is a machine that saves their answers, and that the teachers can see what they have been working on. Others think it is a good idea for the teacher to follow their work progress and contribute to better learning by adapting their instruction and providing pupils with the help they need. It is worth noting that the pupils have different perspectives on giving teachers and schools insight into how they are working on different school subjects. This emphasises the need to engage in dialogue with each individual class and each individual pupil.

The quotes in box 9.1 by pupils in years 7 and 9 provide examples of both the stressful and the motivating aspects of using digital teaching aids as described by the pupils.

Textbox 9.1 Reponses by pupils in years 7 and 9 regarding stressful or motivating aspects of digital teaching aids

“I think you feel more stressed. You don’t want to make a mistake because the teachers will see it.”

“It’s okay that they see what we answer. If I make some mistakes, I can just ask them anyway. That motives me to work even harder.”

“It’s not good to feel afraid of what you’ve done, but it’s good that a teacher can see what you’re able to do. Then you can improve and become better at what you’re working on.”

“It’s good that they can see whether we’re on the right track and give us feedback on that.”

“It’s a positive thing that the teacher can see what we’re answering if it can be used along the way to get help.”

“It’s a form of stalking, since they know exactly what I’m doing all the time. You become more alert when you know that a teacher can see what you’re doing. You can also become very cautious. Sometimes you feel afraid of doing anything.”

“The teacher can adapt their teaching to each individual pupil when they see what we can and can’t do.”

“I don’t think about it that much. It’s not that important. You may have written something wrong and had the wrong idea, but you learn from it.”

Many pupils have expressed that teachers should not have access to information about what the pupils do in their leisure time. Several have stated that they are uncertain as to which school employees can see what the pupils are doing when using the tools. Box 9.2 displays some of the response we received to our question of what type of data on the pupils the school should potentially not have access to.

Textbox 9.2 Responses from pupils in years 7 and 9 about which data on the pupils the school should not have access to

“That which has nothing to do with school. That’s what comes to mind. […] That which has to do with school is collected to help us.”

“Assignments where you share things about yourself. For example, that you take the bus from this or that station. It’s not okay for them to save that and share it.”

“It is very important that they [the teachers] can’t see personal things. We do a lot of personal things on the pc too. No one should ever see my chat log.”

“I wouldn’t have liked it if they [the teachers] had known which YouTube videos I watch or what skin I wear in Minecraft.”

“What happens if my mother uses the pc while I’m at the gym and I was in the middle of an assignment? Would something be registered about me that isn’t me?”

“It’s how I answer and get to the answer that is what is important to know, not everything else.”

Nearly all of the pupils say that the feedback they receive from their teacher has a far greater impact than feedback they are given by a machine through various digital teaching aids and apps. A few mentioned some positive aspects of automatic feedback. See the pupils’ responses in Box 9.3.

Textbox 9.3 Responses from pupils in years 7 and 9 about automatic feedback

“It’s real feedback [if the teacher says so]. You have a relationship with the teacher, and you don’t know who has given you feedback on the application.”

“They [the apps] are just programmed to tell you. Teachers only say something if they want to. Teachers actually mean what they say. Machines just say the same thing over and over, so what teachers say means more to you.”

“You can’t talk to a program. It’s better to get it from the teacher. I would rather raise my hand and say that I needed help.”

“I would rather get it from a program because then you avoid favouritism and that it becomes unfair. It is never completely objective when a teacher gives you feedback. Some teachers really like a certain person and will give that pupil a better assessment. So it’s better with machines.”

“Even if a program nags at you each day, it doesn’t matter as much as when a teacher does it.”

“I think it’s very similar. I feel that you get feedback from the teacher and that the same thing happens with a machine.”

Most of the pupils are sceptical to allowing parents access to everything the pupils do in digital teaching aids. Many believe that the pupils themselves should be able to show their parents what they do, and that parent-teacher conferences are a suitable place for them to get information about how the pupil is doing. Other pupils believe that it is fine for parents to have access, saying that parents often want to know how things are going and to help their own children with their schoolwork. See the pupils’ responses in Box 9.4.

Textbox 9.4 Responses from pupils in years 7 and 9 about what parents should have access to

“I think that the parent-teacher conference is the place where the parents get the info. I don’t want to be monitored by anyone else.”

“They shouldn’t see the feedback we get, even if we’re their kids. I’m the one who does or doesn’t do something, and I have to accept the consequences if it’s poor. It’s my private life and it would be troublesome if they were to know everything.”

“Parents should see more. They can see more of what we are working on, but not how we’ve done it.”

“I want to decide what mum and dad should see. Maybe I’m the one who should have access?”

The pupils are generally positive to the idea that teachers can see how they work and what they are working on. However, they feel differently about developers having access to data on how pupils are working. However, some pupils are positive to having developers use the data to further develop their own solutions. See the pupils’ responses in Box 9.5.

Textbox 9.5 Responses from pupils in years 7 and 9 about what developers and suppliers should have access to

“Those who are developing should not be able to know what we’re doing. We don’t know them, so I don’t think they should have access.”

“It’s uncomfortable to know that they know a lot. You lose all control.”

“They can actually have access to whatever they want, but they should not sell it to others.”

“It’s important to think about who actually needs my data and what it should be used for. I don’t actually know who shouldn’t have access to it, but it’s intended for the school, so should anyone else have access to it at all?”

“I wouldn’t care if my English text from year eight was saved and used by someone for something that could be good for someone else.

The pupils also express that they feel motivated by being able to combine digital and analogue resources. None of the pupils want a school that only uses analogue teaching aids. Nor do they want a school where digital feedback replaces an ongoing dialogue with teachers.

9.4 The Expert Group’s assessments

Participation in learning analytics requires pupils and students to understand and have good insight into what type of data about them is collected, as well as the type of information they can receive from learning analytics. The Expert Group believes it is important for pupils and students to receive adapted and comprehensible information, so that they can consider the issues regarding learning analytics. Through the dialogue and input, we have found that there are significant differences in the type of information that pupils, parents and students receive from the various schools and education institutions.

The Expert Group believes that national authorities must make efforts to make sure that all school owners and education institutions ensure that pupils, parents and students are given the information they need, and that they have the opportunity to participate and be involved in decision making. We also believe that schools and education institutions must actively facilitate good student participation when learning analytics are performed. In order for students and pupils to feel confident that their data is collected in a responsible manner, it is essential that they are clearly informed of their rights in connection with data collection in a manner that is adapted to their age and level of maturity. Facilitating the opportunity for pupils and students to check whether their data is used for learning analytics, to a greater extent than is currently the case, would also help increase participation. Furthermore, it is important that participatory bodies for pupils and students are included in decisions on learning analytics.

10 The need to regulate learning analytics

The overall aim of regulating learning analytics is to help ensure that the value of learning analytics is achieved, while simultaneously reducing privacy risks. In this chapter, the Expert Group will first assess the privacy risks of learning analytics, adaptivity and the use of artificial intelligence. We will then discuss the existing legal bases in the regulations for processing personal data for learning analytics. As an extension of this, we will assess the need for regulatory changes.

10.1 Privacy risks related to learning analytics, adaptivity and artificial intelligence

The risk associated with privacy entails the danger that the rights and freedoms of pupils and students are not sufficiently safeguarded. The General Data Protection Regulation requires national supervisory authorities to create a list of categories of methods of processing personal data which, by definition, involve a high risk. On the list that the Norwegian Data Protection Authority (2019) has compiled of such processing methods, we find this example: “The processing of personal data to evaluate learning, mastery and well-being in schools or kindergartens. This includes all levels of education, from primary and lower secondary school to upper secondary school and higher education. The reason this type of processing is associated with high risk is that it involves children and young people who are considered vulnerable, and because continual evaluation entails invasive, systematic monitoring. The Norwegian Data Protection Authority has thereby assessed that learning analytics and adaptivity that involves processing personal data entails a high risk.

Artificial intelligence challenges the rights of pupils and students in new ways. Artificial intelligence is becoming increasingly widespread in study programmes. Therefore, the Expert Group will discuss a few risk scenarios that may arise with learning analytics and adaptivity that include artificial intelligence. Our starting point will be Article 5 of the General Data Protection Regulation and the fundamental principles[33] , where four of these are particularly challenged by learning analytics and adaptivity with artificial intelligence. These are the principles of fairness, transparency, data minimisation and accuracy.

10.1.1 Fairness

For the processing of personal data in learning analytics and adaptivity to be fair, it must be predictable and comprehensible to pupils, teachers and parents, and it must not be done in a concealed, manipulative or discriminatory manner. One characteristic of artificial intelligence is that it is an innovative and complex technology that we cannot readily understand the scope of.

One risk scenario worth pointing out is that the data controller does not decide the extent to which a legal basis applies and may therefore be in danger of unjustly depriving students and pupils of their right of co-determination. Co-determination in this context may be placed on a scale from deciding whether learning analytics will be performed to having a say on when and how the learning analytics will be performed. Another risk scenario related to the principle of fairness is that algorithms that are not monitored may develop biases in the database and lead to discrimination. Ensuring that a machine learning model behaves fairly and does not discriminate is a challenging task. A third risk scenario is when learning analytics and adaptivity lead to unreasonably differential treatment. We could say that the entire purpose of learning analytics is to support a form of differential treatment: to assist the teacher in assessing how their instruction can be adapted to different pupils and students, and groups of these. The question is therefore not whether the model is treating people differently, but whether it does so correctly, reasonably, and without discrimination. A fourth risk scenario is when learning analytics may lead to a form of chilling effect and stress for pupils and students. By chilling effect, we mean that our awareness of the fact that the things we say and write are being registered and analysed causes us to change our behaviour. As a result, we may begin focus more on what we write, what we do and who we are in contact with – and we begin to curb our behaviour.

10.1.2 Transparency

For the processing of personal data in learning analytics and adaptivity to be in line with the principle of transparency, both the supplier and the data controller must be able to explain data flow – how the data is used in the solution – and algorithms in a comprehensible manner. One characteristic of machine learning is the algorithms are dynamic and programmed to weigh responses in different ways. It is also a characteristic for business models to have little transparency.

A risk scenario that becomes apparent with regard to transparency is that the data controller knows too little about what the learning analytics entail to configure them correctly while also fulfilling their duties to pupils and students, such as providing them with information about the processing. Another aspect of this risk scenario is that students, pupils and parents know too little about what learning analytics involve to be able to safeguard their rights. In general, this could increase the risk of mistrust in learning analytics as a method, which could then lead to more opposition.

10.1.3 Data minimisation

For the processing of personal data in learning analytics and adaptivity to be in line with the principle of data minimisation, it must be possible to limit the amount of collected personal data to that which is necessary for achieving its purpose. One characteristic of machine learning is that it requires large amounts of information to train the algorithms. This is generally counter to the data minimisation principle. In addition, there is a continuing need, which can raise the threshold for initiating deletion or anonymization.

One risk scenario that stands out with respect to data minimisation is that the need for a large amount of data, which could result in the collection of a great deal of information on pupils’ and students’ activities, regardless of relevance and necessity. Another risk scenario is that personal data is not deleted after the purpose of the processing has been achieved, because this data is needed to train the model.

10.1.4 Accuracy

For the processing of personal data in learning analytics and adaptivity to be in line with the principle of accuracy, personal data that is processed must be accurate and give the correct impression of the person from whom the data is collected. One characteristic of machine learning is that the algorithms are meant to improve themselves over time. This creates a need for continuous “training” based on information from a large number of people.

A risk scenario that stands out with respect to accuracy is that the database develops biases that result in discriminatory algorithms. Another risk scenario is when the information being measured about the pupils and students constitutes an inaccurate and deficient database. This may occur if the information does not reflect what the pupil or student has actually done, or when an adaptive learning resource adapts to the pupil or student incorrectly. This may cause the analyses to present an inaccurate picture of the actual situation, which may influence decision making. A third risk scenario is when a teacher or instructor misinterprets the analysis provided by the system due to poor insight into what the analysis is based on, and because the teacher or instructor does not have sufficient competence to interpret the data.

Textbox 10.1 Privacy consequences the Norwegian Data Protection Authority has identified for learning analytics

In the report from the Norwegian Data Protection Authority’s sandbox, the Norwegian Data Protection Authority described several privacy consequences of learning analytics:

chilling effect

risk of inaccurate information

risk that the technology causes pupils unwanted stress

special categories of personal data that require a special legal basis

processing of third-party information

danger of decision support systems becoming decision systems

(Norwegian Data Protection Authority, 2022b)

10.2 Introduction on the legal basis and necessary processing of personal data in learning analytics

In section 5.2, the Expert Group described the relevant data protection legislation for learning analytics. In order for the processing of personal data to be lawful, the processing must have a legal basis in Article 6(1) of the GDPR. Some of the legal bases presuppose the existence of provisions in national legislation that can serve as a legal basis. Before the Expert Group assesses the extent to which existing provisions in Norwegian legislation are suitable as a supplementary basis for processing personal data in learning analytics, we will review which legal bases in the data protection legislation are likely to be used for the purpose of processing personal data in this context.

10.2.1 Legal obligation or task carried out in the public interest

The two bases the Expert Group considers relevant in the GDPR are Article 6(1)(c), processing which is “necessary for compliance with a legal obligation”, and (e), processing which is “necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller”.

As the Expert Group explained in Chapter 5, a basis in legislation will probably not fall under the legal obligation category if the basis entails a considerable degree of free choice as to how the processing of personal data is to be performed in order to fulfil this obligation. For instance, provisions that exclusively authorise someone to do something will not be covered by the legal obligation category in Article 6(c) (Kotschy, 2020). In such cases, the processing will instead fall under to the category “task carried out in the public interest”. If the legal basis in the legislation is found in the legal obligation category, the data subjects (pupils and students) will also lose the right to object to the processing pursuant to Article 21. This indicates in favour of exercising caution regarding the use of legal obligation as legal basis.

Several of the provisions that constitute the relevant legal bases for learning analytics are formulated as obligations in relation to schools or educational institutions. However, these are mainly obligations that largely facilitate the exercise of professional judgment in terms of the manner in which these obligations are to be fulfilled, e.g., in connection with formative assessments and differentiated instruction. This indicates that the processing of personal data linked to these provisions is less compatible with the GDPR’s legal obligation. In addition, the Expert Group finds that learning analytics is not a required tool to achieve the obligations in the legislation. It is possible to fulfil the requirements of the legislation without learning analytics.

In terms of provisions on quality development and quality assurance work, there are other arguments as to whether the provisions fall under the legal obligation category. If learning analytics will be essential or have greater value than other methods for quality development, the processing of personal data could fall under the legal obligation category. As of today, however, we do not find examples of learning analytics that are of such crucial importance to the work on quality development that could it entail a legal obligation. The provisions on quality development in primary and secondary education and training involve, among other things, facilitating local adaptation in terms of how quality development occurs (Prop. 57 L (2022–2023), section 56.5.2). This possibility of adapting the quality assurance work to local conditions will in any case indicate that the provision should not be considered a legal obligation.

The Expert Group’s assessment is that the Article 6(1)(e) of the GDPR – “processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority […]” – is the relevant basis for learning analytics.

10.2.2 On the necessity of processing personal data in learning analytics

For the processing to be legal, it is not sufficient that the processing is related to a “task carried out in the public interest or in the exercise of official authority”. According to the GDPR, it must be necessary. The Regulation does not provide further guidance on what is required for the criterion of necessity to be met. The general linguistic understanding of the processing being necessary would indicate that the purpose cannot be achieved absent the processing in question. There must be an objective connection to the collected data and the manner in which it is processed, in order to fulfil the purpose. As the Norwegian Ministry of Justice and Public Security has noted in other contexts, it is not sufficient that the data are useful; they must be of significance to the work, either on their own or when compiled (Prop. 59 L (2017–2018), section 4.1.3.2).

The ECJ has examined necessity under Article 6(1)(e) and whether the processing in question goes beyond what will be necessary to achieve the objective thereof (C-439/19 Latvijas Republikas Saeima [Grand Chamber], 2021 paragraph 109). The Court also refers to recital 39, where it is stated that the data “should be processed only if the purpose of the processing could not reasonably be fulfilled by other means”.

Currently, it is fully feasible to meet the requirements for differentiated instruction, formative assessment, the educational institutions’ tasks related to educational provision and the requirements for quality assurance work, without learning analytics. However, the Expert Group believes that learning analytics has the potential to improve and enhance the pedagogical follow-up of pupils and students and the work on quality in a manner that will contribute to achieving the objectives of primary and secondary education and training, tertiary vocational education and higher education.

There are several sources of information on pupils and students that, jointly, form the basis for pedagogical decisions and which are used in the work on quality development. The Expert Group does not believe that learning analytics alone will be able to solve the pedagogical tasks referred to in the legislation, but envisages that it will make a valuable contribution in some cases. If learning analytics is to serve as one such source of information, this will require that the learning analytics offers sufficient pedagogical value and that the disadvantages to data protection are sufficiently limited. It is the pedagogical value related to having to process personal data that must be justifiable. The same applies when learning analytics occurs in connection with the further development of subjects and teaching plans, as the value of the processing in relation to achieving the aim must be justifiable.

The Expert Group will highlight some aspects that characterise learning analytics, which we believe will increasingly necessitate the processing of personal data in this context:

the processing gives pupils and students better insight into their own learning

the processing provides teachers and instructors with a better basis for adapting instruction and making pedagogical decisions

the processing gives teachers and instructors a better basis for providing feedback to pupils and students

the processing offers a better basis for quality assurance work in schools, in the municipality, the county authority or at the educational institution.

10.3 Legal basis in the legislation governing the processing of personal data in learning analytics for primary and secondary education and training

The fact that pupils and teachers can utilise learning analytics in line with the purposes as defined in the Education Act, Regulations to the Education Act and the National Curriculum is a prerequisite for both the success and lawfulness of learning analytics (Vestfold and Telemark County Authority, 2022).

The Expert Group finds that there are primarily three provisions that are relevant to discuss as legal bases for the processing of personal data in learning analytics in schools. These are the provision on differentiated instruction in section 1-3 of the Education Act, provisions on formative assessment in the Regulations to the Education Act and the provision on quality development in s 13-3e of the Education Act. A key point is that the processing of learning analytics will largely not, in itself, constitute differentiated instruction, formative assessment or work on quality development. However, learning analytics can constitute one of several sources of information relied upon to better succeed with differentiated instruction, formative assessment or work on quality development. The Expert Group also emphasises that the broad statutory purpose in section 1-1 of the Education Act must be viewed in the context of the specific legal bases.

10.3.1 General provisions on the processing of personal data

Provisions on the processing of data

In 2021, a general provision on the processing of personal data was adopted in section 15-10 of the Education Act. The first paragraph of the provision grants municipalities, county authorities and educational institutions the right to process personal data “including personal data as mentioned in articles 9 and 10 of the GDPR, when this is necessary to perform tasks pursuant to the Act”.

Section 15-10, first paragraph does not constitute an independent supplementary legal basis and the provision does not extend the right to process personal data Prop. 145 L (2020–2021), section 2.4.2.5). For the specific processing of personal data, it is the provisions of the Act and Regulations pursuant to the Act that constitute supplementary processing grounds and which govern rights and obligations. In the preparatory works, the Norwegian Ministry of Education and Research writes that the question of what personal data will be necessary for the school owner or others to process must be assessed based on the purpose of the individual obligation or task Prop. 145 L (2020–2021), section 2.4.2.2).

During the consultation process, it was asked whether section 15-10 precludes the use of article 6(1)(e) “public interest”, as the wording does not mention that it can be used as a legal basis for processing. In its proposal, the Norwegian Ministry denied that the provision precludes the use of Article 6(1)(e) as a legal basis (Prop. 145 L (2020–2021), section 2.4.2.5).

Access control

Section 22A-2 of the Regulations to the Education Act stipulates requirements for access control when processing personal data that are based on a legal basis in the legislation. According to the provision, municipalities and county authorities have a duty to ensure that persons employed by the undertaking only have access to data that are necessary for the purposes stipulated in section 15-10.

10.3.2 Provisions on differentiated instruction

According to section 1-3 of the Education Act, the training must be “differentiated according to the abilities and aptitudes of the individual pupil, apprentice, candidate for certificate of practice and training candidate”. In the preparatory works it is stated that both the organisation of the school, pedagogical methods and the progress in the instruction shall be adapted according to the prerequisites and abilities of the pupils (Proposition to the Odelsting Ot.prp. nr. 40 (2007-2008), section 3.2). Differentiated instruction is a basic and overriding principle in the instruction. The provisions on differentiated instruction require schools to differentiate the instruction according to the pupils’ prerequisites, but do not enshrine an individual right for the individual pupil. In the preparatory works to the provisions, the data protection consequences of processing personal data for the purpose of providing differentiated instruction have not been considered.

As part of differentiated instruction, a duty is included to provide intensive instruction (early intervention) to pupils in grades 1 to 4. Section 1-4 of the Act imposes a duty on school owners to provide suitable intensive instruction in reading, writing and mathematics so that they achieve the expected progress. The provision does not express how the early intervention is to be carried out. The Norwegian Ministry’s statement in the preparatory works to the provision is illustrative of this:

The Norwegian Ministry specifies that there is no definitive answer as to how intensive instruction shall be carried out. Different methods and pedagogical approaches may be suitable. It will be a pedagogical and didactic task to decide which measures are necessary and expedient in relation to the individual pupil. How the intensive instruction should be arranged must be assessed in the light of, among other things, the needs of the pupil and other measures in the instruction. Prop. 52 L (2017–2018), section 3.5.1)

The duty to provide differentiated instruction is fulfilled through teachers’ choices of methods and tasks. However, the provision on differentiated instruction and early intervention does not state what methods are suitable. The provision on differentiated instruction establishes a broad objective of educational practice.

The Expert Group finds that differentiated instruction is a relevant legal basis for processing personal data in learning analytics that is necessary to “perform a task carried out in the public interest” in Article 6(1)(e) of the GDPR. The provision on differentiated instruction falls under the category in Article 6(1)(e), among other things, because the provision establishes a general and broad principle, which does not clarify specific guidelines as to how the instruction should be differentiated in relation to the pupils. The provision does not provide further guidance on what renders learning analytics necessary processing in order to carry out differentiated instruction.

However, using differentiated instruction as a legal basis for processing personal data in learning analytics entails a challenge in that the provisions on differentiated instruction make it difficult for pupils to predict what personal data will be processed and how. In addition, the provision does not contain any information on how the principles of purpose limitation, data minimisation and accuracy will be safeguarded in the processing. When processing personal data about children, it is important that the legal basis is clearly formulated, due to their vulnerable position.

The Expert Group finds that the provision is not suitable for specifying that the processing of personal data in learning analytics can be used in connection with differentiated instruction. Codifying one of the methods teachers can use to differentiate instruction would be contrary to the general scheme of the statutory provision. Moreover, such a specification could give the impression that learning analytics plays a particularly important role in differentiated instruction, which we believe is not currently the case.

10.3.3 Provisions on the right to assessment

The right to assessment is laid down in Regulations pursuant to the Education Act. Section 3-2 grants pupils the right to “formative assessment, final assessment and documentation of the instruction”. According to section 3-3, the purpose of the assessment is to enhance learning and express the pupil’s competence. Pursuant to section 3-3, the basis for assessment is the pupil’s participation and activity in light of the objectives in the National Curriculum.

According to section 3-10 of the Regulations pursuant to the Education Act, “[a]ll assessment that takes place before the end of the instruction” is formative assessment. According to the Regulations, the formative assessment must be an “integral part” of the instruction. In its circular, the Norwegian Directorate for Education and Training elaborates that the wording “integral part” entails that “pupils and apprentices must be assessed, or assess their own work, where this is natural in the instruction, without this necessarily being planned or occurring at fixed and agreed intervals” (Norwegian Directorate for Education and Training, 2021a). The Norwegian Directorate for Education and Training further elaborates that the wording “integral part” means that “pupils and apprentices must be assessed, or assess their own work, where this is natural in the instruction, without this necessarily being planned or occurring at fixed and agreed intervals”. In other words, formative assessment refers to a set of different methods and ways of working to aid pupils in the learning process.

In the description of formative assessment, there are several elements that fit what the Expert Group identifies as important characteristics of learning analytics. The description of formative assessment in the section 3-10 of the Regulations is also recognisable in terms of the purpose of learning analytics. The pupil shall participate in the assessment of their own work and reflect on their own learning and academic development. They shall also understand what they need to learn, what they master, what is expected of them and how they can further work to enhance their competence.

The Expert Group finds that the processing of personal data in learning analytics in connection with formative assessment will fall under the legal basis of “performing a task carried out in the public interest” in Article 6(1)(e). Learning analytics can be one of several ways of carrying out formative assessments. Furthermore, we have found that formative assessment will not cover all relevant contexts where learning analytics takes place. Examples could be that information from learning analytics is used as a basis for didactic assessments, e.g., selecting relevant and motivating working methods and teaching aids, or other matters not directly related to the purpose of formative assessment.

Because formative assessment is to be an integral part of instruction, it can be difficult to separate formative assessment from other pedagogical activities. Avoiding such a distinction is also part of the purpose of the emphasis in the Regulations that assessment must be an integral part of the learning activities. Linking learning analytics specifically to formative assessment may also send unintended signals that the formative assessment should to a greater extent be based on learning analytics, even if this is not the intention. This can have unfortunate consequences, as it is precisely the breadth and variety in how pupils show and develop their competence that is important.

Another challenge in using formative assessment as a legal basis for learning analytics is that formative assessment is closely linked to the final assessment. According to section 3-15, second paragraph of the Regulations, “competence that the pupil has shown during the instruction” must be part of the assessment when the mark awarded for classwork is determined. It will then be the teacher’s task to decide whether the competence the pupil has shown provides relevant information about the pupil’s competence at the end of the instruction. Based on current practice, it would be unfortunate to signal that learning analytics provides sufficient breadth in the assessment of pupils’ competence that it can be used as a basis for determining the mark awarded for classwork.

The Expert Group believes that the provision on formative assessment is a relevant legal basis for some forms of learning analytics. It is a strength that the provision indicates how the pupil should be involved in the formative assessment and what the pupil should understand. This procedure will help ensure that the data that are processed in learning analytics through formative assessments are accurate and adequate. Nevertheless, the provision on formative assessment will not make it predictable how personal data may be processed in connection with learning analytics. The Expert Group finds that the provision on formative assessment is not a suitable place to specify the legal basis for processing personal data in learning analytics. Such a specification may indicate that learning analytics, compared to other methods, plays a particularly important role in formative assessments, which the Expert Group does not believe is currently the case. The Expert Group therefore believes that specifying the provision on formative assessment for the processing of personal data in learning analytics could send the wrong signal.

10.3.4 Provisions on quality development

Section 13-3e of the Education Act contains a duty on the part of the municipality and the county authority to work on quality development. Private schools have a corresponding duty to work on quality development in section 5-2b, where the school board is responsible. The purpose of the quality development work is for the school owners and the schools to use knowledge about the learning environment and learning outcomes at the schools to assess how the education can be improved Prop. 81 L (2019–2020), section 12.4).

The provision in the section 13-3e, second paragraph of the Education Act reads as follows: “The municipality and the county authority shall ensure that the schools regularly assess the extent to which the organisation, differentiation and implementation of the instruction contribute to achieving the objectives set out in the Curriculum for Knowledge Promotion in Primary and Secondary Education and Training. The pupils shall be involved in this assessment”.

Information from learning analytics may represent one of several sources for the quality development work. The purpose of the quality assurance work is for the school owners and the schools to use knowledge about the learning environment and learning outcomes at the schools to assess how the instruction can be improved (Prop. 81 (Bill) (2019–2020), section 12.4). Neither the provision on quality development nor the related preparatory works state what personal data or how personal data are to be processed in connection with quality development.

To more closely examine what the work on quality development entails, we can look to discussions about school self-evaluation. Previous provisions on quality development used the term school self-evaluation, but the removal of this term was not intended to alter the content of the provision. The Education Act Committee noted that the content of school self-evaluation is unclear (NOU 2019: 23). The Expert Group on school performance indicators noted a need to prepare a harmonised interpretation of the specific academic content in section 13-3e (Expert Group on performance indicators, 2021). School self-evaluation was one of the topics in the Norwegian Directorate for Education and Training’s joint national inspection for 2014–2017, and what was examined in the inspection expresses how national authorities have understood the term (NOU 2023: 1). In the inspection, the Directorate investigated whether schools have a broad and comprehensive assessment of goal attainment for pupils, assess whether changes can contribute to greater goal attainment by pupils, conduct regular assessments and ensure broad participation (Norwegian Directorate for Education and Training, 2019).

The Committee for Quality Development in Schools believes that the work on quality development must be based on a broad platform of knowledge that builds on the experiences of the professional community, relevant research and the local context (NOU 2023: 1). Information from learning analytics will represent one of several sources of information for quality development. The fact that the provision’s professional content has not been determined makes it difficult to clarify the degree to which the provision is suitable as a legal basis for learning analytics. Furthermore, the Expert Group believes that the legal basis for learning analytics in quality development primarily falls under “performing a task carried out in the public interest” in Article 6(1)(e) of the GDPR.

In general, learning analytics used for quality development could have less intrusive data protection consequences than the processing of personal data that form the basis for decisions concerning the direct follow-up of a pupil. For quality development, it will also often be sufficient to process de-identified information.

The Expert Group does not consider it suitable to specify that learning analytics may be an instrument in section 13-3e when carrying out quality development. Such a specification would be contrary to the general scheme of the statutory provision. The statutory provision only stipulates that schools shall work on quality, but not how this is to be done.

10.3.5 Need for legislative amendments

There are broad provisions in the Education Act which enunciate the objectives of education, differentiated instruction, assessment, pupils’ benefit from the instruction and the school owner’s responsibility for quality development, and which refer to contexts where learning analytics may be relevant. In Chapter 10.3.1, the Expert Group referred to the general provision in the section 15-10 of the Education Act which emphasises that the school owner may process personal data, including those mentioned in Articles 9 and 10 of the GDPR, when this is necessary to carry out a task in the Act.

Determining that learning analytics may be in accordance with the overall purposes of the Act is not necessarily challenging. However, determining accordance is not sufficient. As shown in Chapter 5.2.1, the GDPR and the ECHR set requirements for how the legal basis for processing personal data is designed. The legal basis “must lay down clear and precise rules governing the scope and application of the measure in question” (C-439/19 Latvijas Republikas Saeima [Grand Chamber], 2021, paragraph 105).

The Expert Group believes that it would be beneficial if the legislation clarified that learning analytics may contribute to carrying out statutory duties imposed on the schools. The Expert Group has shown that learning analytics can have pedagogical value and be a relevant method for differentiating instruction, providing formative assessment and for the work on quality development.

A key argument in favour of specifying the legal basis in the Education Act for learning analytics is to address the pupils’ need for predictability regarding how their personal data are processed when the processing may have significant consequences for their privacy. A clearer basis may contribute to making it easier for school owners to ensure their responsibilities when processing personal data in learning analytics. The need for a clearer legal basis is precarious as it concerns the processing of data in ways that pupils find invasive. In addition, special categories of personal data described in the Article 9 of the GDPR may be processed, e.g., health data related to learning difficulties or the like. In order to prevent disproportionate interference with children’s privacy, the legislation must reflect the challenges raised by learning analytics.

A clearer basis in the legislation could help prevent personal data from being used for purposes other than those for which they were collected. A more precise legal text can offer pupils and parents a clearer sense of boundaries in terms of their own rights. There is, however, a risk that any clarification of the legal basis for learning analytics will remain symbolic provisions devoid of real content, and which fail to establish guidelines for how the processing of personal data should take place. Another risk in clarifying the basis for processing personal data for learning analytics is that it could be perceived as an opportunity to carry out more extensive learning analytics without ensuring corresponding data protection requirements.

In connection with the consultation process prior to the adoption of section 15-10 of the Education Act, the Norwegian Data Protection Authority stated that the legal basis in the Education Act “should set out fixed, objective, statutory criteria for the processing if it is to comply with the data protection legislation’s requirements for a legal basis” (Norwegian Data Protection Authority, 2021a, p. 1). It is difficult to answer how the legal basis in the legislation can be designed to ensure that the processing of pupils’ personal data in learning analytics occurs in a manner that safeguards their privacy.

An important question is how the legal basis can contribute to ensuring that the processing of personal data is necessary for the purpose. A general starting point is that the school owner, as controller, must be able to demonstrate that processing personal data in learning analytics has a pedagogical value that outweighs the data protection consequences it entails. However, it is not only the necessity of the processing that makes it lawful, the legal basis should also make it easier for the school owner, as controller, to observe the principles of accuracy, purpose limitation and data minimisation.

The Expert Group believes that there is a clear need to elucidate the legal basis for learning analytics. The basis must contribute to ensuring that the processing of the pupils’ personal data in learning analytics occurs in a more predicable manner.

10.4 Legal basis in the legislation governing the processing of personal data in learning analytics for universities and university colleges.

The possible legal bases for processing personal data in learning analytics are found in various parts of the legislative framework governing higher education. The Expert Group considers that there are mainly provisions in three areas that are relevant as a legal basis for the processing of personal data in learning analytics in higher education.

Firstly, there is a general provision on the processing of personal data in section 4-15 of the Universities and University Colleges Act. Second, sections 1-3 and 1-5 contain provisions on the institutions’ tasks and professional responsibilities. The third area of legislation that is relevant as a legal basis for learning analytics is the provisions on the institutions’ quality assurance work in section 1-6 of the Universities and University Colleges Act, in addition to the provisions on quality assurance work in the Regulations on the quality of programmes of study and the Academic Supervision Regulations.

The Expert Group will discuss whether the provisions in the respective areas are suitable as a legal basis for processing personal data in learning analytics. This involves, among other things, the extent to which the provisions are suitable to fulfil the requirement of a clear and distinct legal basis in Article 6(3) of the GDPR, in light of statutory requirements in the Constitution of Norway and the ECHR.

10.4.1 Provision on the processing of personal data in course management systems

Section 4-15 of the Universities and University Colleges Act has the heading “Obtaining and processing personal data in course management systems”. The first paragraph of the provision stipulates that the educational institutions may process personal data regarding the students “when the purpose of the processing is to safeguard the rights of the data subject, or to fulfil the institution’s tasks and duties under the Universities and University Colleges Act.” The wording in the text of the Act indicates that the provision does not constitute an independent legal basis, but merely expresses what is already stipulated in Article 6(1), cf. Article 6(3), i.e., that personal data may be processed when this is necessary for compliance with a legal obligation or for the performance of a task carried out in the public interest pursuant to acts or regulations. An unresolved question is whether the provision is limited to the collection and processing of personal data that exclusively takes place in course management systems. In the preparatory works, the Norwegian Ministry states that the term “course management systems” is used as “a common name in the bill to include all course management systems in the sector” Prop. 64 L (2017–2018), section 7.2.2).

The provision’s second and third paragraphs list the types of data the institutions can process when necessary in relation to the purpose in the first paragraph. The second paragraph lists data that will typically be necessary to process upon admission to a study programme, such as names and marks from upper secondary education, universities and university colleges, retrieved from public authorities. The second paragraph mentions “information about health, social issues and other sensitive information which the student him/herself has given to the institution”.

During the consultation round for the current section 4-15, both the University of Oslo (UiO) and the Norwegian University of Science and Technology (NTNU) expressed in their consultation responses that the processing of personal data regarding students will increasingly take place in contexts other than in the traditional course management settings. NTNU first states that the proposal for the provision in section 4-15 is unlikely to meet the needs of the educational institutions, before summarising the uncertainty surrounding the processing basis for, among other things, learning analytics as follows: “Current areas where the legal basis may be uncertain are the processing of personal data in connection with learning analytics, development and quality improvement in education, access control, exchange of information between institution and place of professional practice, as well as storage of answers in the plagiarism control system” Prop. 64 L (2017–2018), section 7.2.3).

When the Norwegian Ministry discussed the consultation responses, it stated that the provision will not cover all the institutions’ needs for processing personal data, and also noted that was not the purpose of the bill Prop. 64 L (2017–2018), section 7.2.4). In the same place, the Norwegian Ministry pointed out that “broad ‘all-encompassing’ authorisations cannot be granted without further investigation of consequences and needs”. Nor has it been considered in the preparatory works whether data in course management systems can be used in learning analytics. Based on the provision’s wording and statements in the preparatory works, the Expert Group believes that the provision in its current form in section 4-15 does not clarify the extent to which personal data can be processed in learning analytics.

10.4.2 Provisions on the institution’s tasks, as well as academic freedom and responsibility for instruction

Section 1-3 of the Universities and University Colleges Act sets out the institutions’ duties at an overarching level. According to section 1-3 letter (a), universities and university colleges shall “provide higher education based on the foremost within research, academic and artistic development work, and experience-based knowledge”. In the preparatory works, the Norwegian Ministry writes that the provision on the institutions’ tasks notes what is needed to achieve the ambitions set out in the statutory objective of the Act, and that the provision will not involve a regulation of specific rights or duties Prop. 111 L (2020–2021), sections 2.3.5.1 and 2.3.5.2).

Pursuant to section 1-5, first paragraph, the institutions have academic freedom and responsibility for, among other things, instruction being conducted in accordance with “recognised […] pedagogical and ethical principles”. This also means that the institutions are responsible for the academic follow-up of the students. The Universities and University Colleges Act Committee noted the importance that the legislation “does not set limitations for new and varied ways of teaching and learning” (NOU 2020: 3, p. 194). The Expert Group believes that the regulations should also ensure that learning analytics safeguards student privacy, which presupposes a clear legal basis.

The Norwegian Ministry considered proposing a provision that specifies that the institutions are responsible for good academic follow-up of the students, but concluded that such a specification was not necessary as long as the responsibility is already stated in section 1-5, first paragraph Prop. 74 L (2021–2022), section 7.4). The extent to which learning analytics can be carried out in the academic follow-up of students has not been a topic in the preparatory works. In the Expert Group’s view, section 1-5 on academic freedom and responsibility constitutes a legal basis for “performing a task carried out in the public interest” i Article 6(1)(e) of the GDPR.

The provision in section 1-5 stipulating that the institutions must ensure that instruction takes place in line with “recognised […] ethical and pedagogical principles”, does not make it particularly predictable how the personal data of students are to be processed in learning analytics. It would not be appropriate for the provision containing the institutions’ overarching task to list only one of the means institutions can use to ensure that instruction takes place in accordance with recognised ethical and pedagogical principles. Therefore, the Expert Group does not find that section 1-5 is a suitable place in the legislation to specify that the institutions may process personal data in learning analytics.

The Expert Group finds that section 1-5 is a relevant legal basis for processing personal data in learning analytics. At the same time, the Expert Group believes that the provision does not provide a clear enough basis for processing personal data in learning analytics.

10.4.3 Provisions on quality assurance work

In section 1-6, universities and university colleges are required to have a “satisfactory internal system for quality assurance that will ensure and further develop the quality of the education”.

According to section 2-1, third paragraph, NOKUT shall, in consultation with the sector, issue regulations on criteria for the quality assurance work of the institutions. The further details on the content of the quality assurance work are regulated in section 2-1 of the Regulations on the quality of programmes of study:

Universities and university colleges shall manage the responsibility for the quality of education by way of systematic quality assurance work that ensures and contributes to developing the quality of the study programmes. Furthermore, the institutions shall facilitate ongoing development of the quality of education, be able to detect declines in quality in the study programs and ensure satisfactory documentation of the quality assurance work. The institutions shall ensure the quality of all matters that impact the quality of studies, from information to potential applicants to completed education.

In addition, section 4-1 of the Academic Supervision Regulations contains requirements for systematic quality assurance work:

1) The institution’s quality assurance work must be based on a strategy and cover all substantial areas of significance for the quality of students’ learning outcomes […]. 4) The institution shall systematically collect information from relevant sources in order to be able to assess the quality of all study programmes. 5) Knowledge obtained from the quality assurance work shall be used to develop the quality of the study programs and detect any declines in quality. Declines in quality must be rectified within a reasonable period of time. 6) Results from the quality assurance work shall be included in the platform of knowledge for assessment and strategic development of the institution’s overall study portfolio.

According to the guide that NOKUT has prepared, the above-mentioned requirement will also entail an assessment of whether students actually achieve the desired learning outcomes (NOKUT, 2022). NOKUT also notes that the institutions shall collect sources from all matters of relevance to the quality of studies. At the same time, they specify that the institutions should not collect more information than is necessary to inform and assess the quality of studies.

Information from learning analytics can represent one of several bases that may be relevant for the quality assurance work at institutions in higher education. The statutory tasks of the institutions presuppose their processing of personal data. However, the manner in which such processing should take place in order to comply with the data protection legislation has not been addressed. The Expert Group believes that the legislation should clarify that processing personal data in learning analytics is relevant in the quality assurance work. In the case of quality assurance work, it will primarily be relevant to use data at an aggregated level as a basis for decisions, which contributes to minimising the data protection risks involved.

The Expert Group believes the legal basis for processing personal data in learning analytics for quality assurance work should be clarified. The Expert Group also believes that it is relevant to consider the provisions on quality assurance work as a legal basis for the “performance of a task carried out in the public interest” in Article 6(1)(e) of the GDPR.

10.4.4 Need for legislative amendments

In the sections above, the Expert Group has shown that relevant provisions in the current legislation governing higher education do not provide a clear legal basis for processing personal data in learning analytics. The discussions in the proposition prior to the adoption of section 4-15 show that it is uncertain in what situations the institutions are permitted to process personal data in learning analytics. In its comments to the Expert Group, Sikt notes that the unresolved legal basis for the university and university college sector to carry out learning analytics has been an obstacle (Sikt, 2022). The Expert Group believes that the provisions on the institutions’ tasks in section 1-5 and the provisions on quality assurance work contain relevant legal bases for processing personal data in learning analytics.

The Expert Group believes that there is a need to clarify what is needed for institutions to process personal data in learning analytics, both for processing in pedagogical practice and in quality assurance work.

10.5 Legal basis in the legislation governing the processing of personal data in learning analytics for vocational colleges

Several of the provisions that are relevant for assessing whether there is a suitable legal basis in the vocational college legislation are designed according to similar provisions in the Universities and University Colleges Act. In the following sections, the Expert Group assesses whether provisions in the vocational college legislation are suitable to fulfil the requirement of a legal basis in Article 6(3).

10.5.1 Provisions on the processing of personal data and the content of vocational college education

Section 4 of the Vocational Education Regulations is entitled “Collection and processing of personal data by the vocational colleges”. The first paragraph of the provision contains a right to process personal data regarding students “when the purpose of the processing is to safeguard the data subject’s rights, or to fulfil the school’s obligations pursuant to the Vocational Education Act”. In the consultation paper on which section 4 is based, it is noted that processing personal data is necessary in course management systems and during the admissions process for the vocational colleges (Norwegian Ministry of Education and Research, 2019). In the second paragraph of section 4 of the Regulations, it is expressly stated which personal data may be used in the processing to achieve the purposes stated in the preceding paragraph. Among other things, it mentions: “name, national identity number […] work experience and marks from upper secondary education, vocational colleges, universities and university colleges and subjects and journeyman’s certificates retrieved from public authorities”. The provision does not address the right to process other personal data, e.g., from students’ learning activities and the like.

According to section 4 of the Vocational Education Act, vocational college education shall be “based on knowledge and experience from one or more professional fields and be in accordance with relevant pedagogical, ethical, artistic and scientific principles”. The Vocational Education Academic Supervision Regulations contain requirements for the content and form of vocational college education. Section 2-1 sets out a broad requirement that the education should, among other things, have “instruction, learning and assessment forms that are suitable for the students to achieve the learning outcomes”. The Expert Group finds that the provision is a relevant basis for processing students’ personal data in vocational colleges in learning analytics.

The provision does not describe in more detail which types of processing of personal data may be relevant for the suitable instruction and learning methods. Thus, the provision makes it difficult for students to predict how their personal data will be processed in instruction and learning situations. The Expert Group believes that the provision does not provide a clear legal basis for processing students’ personal data in learning analytics. The Expert Group nevertheless finds that it would not be appropriate to specify that learning analytics may be included in the instruction and learning methods in section 2-1 of the Vocational Education Academic Supervision Regulations. Learning analytics is just one of several instruments that can help ensure that students achieve learning outcomes. Codifying this one instrument may give the impression that learning analytics takes priority in terms of choice of instruction and learning methods, which is not the intention.

10.5.2 Provisions on quality assurance work

Section 5 of the Vocational Education Act addresses accreditation and quality assurance. The fifth paragraph of the provision stipulates that vocational colleges shall have “satisfactory internal quality assurance systems”, and grants the Norwegian Ministry the right to issue regulations relating to the “requirements for quality assurance systems and quality assurance work”. According to section 5, sixth paragraph (d), the Norwegian Ministry may issue regulations on “requirements for quality assurance systems and quality assurance work”.

The requirements for the vocational colleges’ systematic quality assurance work are specified in section 4-1 of the Vocational Education Academic Supervision Regulations. In order to assess whether each individual education meets the quality assurance target, the vocational colleges must, according to Section 4-1, third paragraph, “systematically collect […] information from students, employees, representatives from the professional field and any other relevant sources”. However, NOKUT’s guidance on the provision specifies that the institution should not collect more information than is necessary to inform and assess the quality of studies (NOKUT, 2020). In the consultation paper that accompanied the bill, there is no discussion of what this information may entail or how it may be processed (NOKUT, 2019). The absence of data protection discussions renders the provision unclear with regard to the extent to which it may constitute a legal basis for the processing of personal data in learning analytics for quality assurance work.

10.5.3 Need for legislative amendments

The provisions that are relevant for processing personal data for learning analytics in vocational colleges govern the processing of personal data at different levels of detail. The Expert Group believes that this causes uncertainty in situations where neither the legislation nor the consultation process have assessed the frameworks for and consequences of processing personal data.

The Expert Group believes that the legal basis for learning analytics in vocational colleges should be clarified. This applies both to the vocational colleges’ pedagogical practice and to their quality assurance work. The Expert Group believes it is important to clarify the legal basis, as a large proportion of students in the vocational college sector are online students, where analyses of student activity are particularly relevant. For all vocational college students, and online students in particular, it is important that the legal basis in the legislation increases the predictability as to how personal data are processed in learning analytics.

10.6 Summary of the Expert Group’s assessments of legal bases

The Expert Group finds that the current provisions on processing personal data in learning analytics are unclear. A general challenge is that the provisions that determine tasks in legislation where learning analytics may be relevant largely fail to ensure predictability for pupils and students in terms of how the personal data are processed. Moreover, the data protection consequences of learning analytics are hardly mentioned in the preparatory works to these provisions. For primary and secondary education and training, higher education and tertiary vocational education, there is a glaring need to clarify the legislation so that it indicates when there is a right what is required to process personal data in learning analytics.

For the primary and secondary education and training, the Expert Group’s assessment is that the provisions that determine the relevant tasks of differentiated instruction, formative assessment and quality development, can serve as legal bases for processing personal data in learning analytics. The Expert Group believes that the basis for processing personal data in learning analytics should be clarified, but does not find that the provisions that determine the tasks are suitable places to include specifications about learning analytics.

Regarding higher education, the Expert Group finds that the provisions on the institutions’ tasks and quality assurance work could be relevant legal bases for processing personal data in learning analytics. The Expert Group believes that the grounds for processing personal data in learning analytics must be clarified, but does not consider it appropriate to specify the actual provision that determines the institution’s tasks and responsibilities.

The Expert Group finds that the provisions on vocational colleges’ instruction and learning methods and the provisions on quality assurance work are relevant legal bases for processing personal data in learning analytics in tertiary vocational education. The Expert Group believes that the processing basis must be clarified, but nevertheless finds that the provision on the vocational colleges’ instruction and learning methods is not a suitable place to include such specification.

Part 3
The Expert Group’s recommendations and proposals

11 The objectives of the Expert Group’s recommendations

Figure 11.1

The discussions in the first interim report showed that there is a need for clear frameworks that ensure justifiable learning analytics. These frameworks must safeguard the privacy of pupils and students, while at the same time create sufficient scope of action to utilise the potential. The Expert Group has found that there is a considerable demand in the education sector for clear guidelines on matters pertaining to data protection and artificial intelligence. Regarding matters such as quality assessment and decisions regarding use, a greater degree of support and guidance is desirable.

Developing good practice that includes learning analytics means facilitating professional discussions regarding purpose, impact on learning processes, instruction situations and roles. Such discussions must incorporate professional, pedagogical, ethical and data protection aspects. Drawing on the professional environment in this manner enhances knowledge, awareness and competence about learning analytics, adaptability and the use of artificial intelligence.

In this report, we have based our work on the knowledge of what kinds of learning analytics are currently carried out in Norwegian education, what needs the actors themselves express that learning analytics can address, what characterises data quality in learning analytics, and how learning analytics is regulated in the current legislation. Through assessing how learning analytics can enhance learning and improve instruction, which pedagogical and ethical challenges are associated with learning analytics, how participation should be ensured, and the need to amend the legislation, the Expert Group has drawn up recommendations and proposals that will contribute to safeguarding trust, reducing risk and building good pedagogical practice.

11.1 Safeguarding trust in the education sector

In Norway, public authorities enjoy a high level of trust among the population. Privacy is a value that contributes to safeguarding and building trust in society (NOU 2022: 11). It has been important for the Expert Group to take a position on privacy in the context of pedagogical, ethical and legal issues. Learning analytics can both challenge and strengthen trust in the education sector, and the Expert Group has emphasised preparing recommendations and proposals that contribute to secure and justifiable learning analytics with a clear pedagogical purpose.

It is worth noting that trust in how the school safeguards children’s privacy is significantly lower than in other public agencies (Norwegian Data Protection Authority, 2020b). Part of the reason for this may be that a lot of attention has been paid to several breaches of the Personal Data Act in the school sector (NOU 2022: 11). Nevertheless, it is our clear impression that there are high expectations that schools are to be able to adequately solve privacy challenges. In its comments to the Expert Group, the Parents’ Committee for Primary and Secondary Education (2022b) states as follows:

[c]hoices we as parents make in our spare time are often characterised by naivety and trust that someone will ensure that no one tampers with my data. When we send our children to school, many of us have a higher expectation of what considerations the school will make in relation to our children. The school as a public institution must set the standard! (p. 3)

In higher education and tertiary vocational education, student representatives have expressed that they largely perceive that their personal data are in safe hands with the educational institutions.

Data protection is not just about personal data being secure. It strengthens trust if the educational institutions process pupil and student data in ways where the purpose and procedure are predictable and comprehensible for pupils and students. In the case of learning analytics, the processing of data will not be self-explanatory, as in digital learning resources with simpler functionality. If schools and educational institutions enable pupils and students to understand and properly use personalised functionality, this contributes to increasing their trust and autonomy.

Taking to steps that contribute to strengthening trust that privacy is safeguarded throughout the educational pathway is one of the main goals of the Expert Group’s recommendations and proposals. First and foremost, trust must be strengthened through improved data protection practices. The Expert Group recommends clarifying the legal basis for learning analytics and preparing a code of conduct and guidelines for data protection in education to help achieve this goal. Recommendations regarding predictability, fairness, transparency and participation in learning analytics are key to ensuring trust.

11.2 Risk reduction

Decisions regarding learning analytics involve various risk assessments. Risk reduction is often associated with exercising caution. However, the Expert Group notes that standing still is not risk-free either, particularly in the field of technology. Innovations in artificial intelligence have created fertile ground for curiosity, exploration and innovation in the education sector, but also the need to ask critical questions. We wish to facilitate experiences with learning analytics within as secure and justifiable frameworks as possible, but recognise that the improvement and development of teaching practice through the use of new forms of technology always involves a certain risk.

Learning analytics requires that school owners and educational institutions assess the risk by using information about pupils’ and students’ activities, behaviour, performance and background to enhance learning and improve instruction. This entails considering the data protection consequences in relation to other values in society. An important task is to identify measures that reduce the data protection risk to an acceptable level. The school owners and the institutions must also assess the risk of not utilising the potential of learning analytics to enhance learning and improve instruction. Although the platform of knowledge on learning analytics in Norway is inadequate, we do find indications of untapped potential. As we showed in the first interim report, research and development projects point to pedagogical gains by having access to information about pupils’ and students’ professional development through learning analytics. Realising these gains requires innovation and the development of good resources with functionality for learning analytics. This is a market with considerable investment costs and development involves a high risk for developers and suppliers (ICT-Norway, 2023).

Making school owners and institutions better able to safeguard pupils’ and students’ privacy is a clear goal of the recommendations put forward by the Expert Group. The Expert Group wishes limit both an unnecessarily restrictive and an uncritical approach to learning analytics in the education sector. We also wish to stimulate development that to a greater extent brings out the inherent value of learning analytics. Recommendations on a code of conduct and guidelines for data protection and a clarification of legal bases can contribute to the former, while recommendations on competence development, guidance services, grant schemes and usage-based price models can contribute to the latter.

11.3 Developing good practice

A common concern about the use of technology in education is whether it contributes to good learning and instruction. An important task for the Expert Group has been to investigate how learning analytics can support the objectives of education and enhance learning. Several of our recommendations and proposals concern facilitating good pedagogical practice by increasing free choice and support for quality assessment in learning analytics and offering various guidance services and contributing to competence development.

A large part of the data generated in learning situations involving the use of digital devices can constitute a valuable pedagogical resource. Giving pupils and students insight into their own learning and giving teachers and instructors a better basis for differentiating instruction and follow-up are two main areas of value. The third main area of value concerns learning analytics as a suitable instrument for conducting computer-supported quality assurance work in education. Analyses of pupil and student data can also be relevant at an even more general level. In the Norwegian Government’s strategy for digital competence and infrastructure in kindergartens and schools, it is noted that researchers can use aggregated data to develop knowledge about learning, which the authorities can use to adapt the use of instruments to improve the schools’ situations (Norwegian Ministry of Education and Research, 2023). In the strategy for digital transformation in the university and university college sector it is a stated ambition that data from this sector is used to contribute to streamlining and strengthening education and research and to generate more innovation and value creation (Norwegian Ministry of Education and Research, 2021).

Frameworks and guidelines that safeguard scope of action and professional discretion will support the improvement and further development of pedagogical practice. Thereby, the Expert Group wishes to promote good learning analytics and pedagogical practice by recommending frameworks and guidelines for learning analytics for primary and secondary education and training, higher education and tertiary vocational education, respectively. For primary and secondary education and training, we have emphasised providing teachers and schools with a better overview and free choice when using digital resources, strengthening the basis for assessing the quality and suitability of the resources and facilitating competence development. In higher education and tertiary vocational education, we have emphasised free choice, guidance and competence development.

11.4 Overview of the content of the recommendations

In the following chapters, the Expert Group will present four main recommendations to support good and justifiable learning analytics throughout education system. Primary and secondary education and training, higher education and tertiary vocational education each receive their own recommendations. The reason for this is partly that the sectors do not have the same purposes, structures, traditions and practices, but also that they have different experiences with learning analytics, and their needs for learning analytics clearly differ from each other. Nevertheless, it is important to view the four recommendations in context. This is primarily because an educational pathway should involve a certain degree of continuity and predictability, but also because the recommendations influence and build on each other. For example, recommendations on legal basis are closely related to the recommendations on creating a code of conduct and guidelines on data protection. In order for learning analytics to be used as an instrument at all, the legal basis must be in place. However, it is the code of conduct and guidelines that will ensure a justifiable practice.

11.4.1 The Expert Group’s four main recommendations on learning analytics

The first recommendation is to clarify the legal basis for learning analytics in primary and secondary education and training, higher education and tertiary vocational education. The purpose of this recommendation is to clarify when the processing of personal data in learning analytics is lawful, create better predictability and provide guidance.

The second recommendation is to prepare a data protection code of conduct in primary and secondary education and training. The purpose of this recommendation is to strengthen pupils’ privacy and facilitate good data protection practices and increase data protection awareness and competence.

The third recommendation is to establish a framework for good learning analytics in primary and secondary education and training. The purpose of this recommendation is to strengthen the free choice of pupils and teachers and to provide a better basis for pedagogical decisions regarding learning analytics to enhance learning.

The fourth recommendation is to develop broad guidelines for good and justifiable learning analytics in higher education and tertiary vocational education. The purpose of this recommendation is to facilitate good data protection practices and justifiable learning analytics that promote student learning and enhance the quality of education.

12 Legal basis for learning analytics

Processing personal data about pupils and students is key to many forms of learning analytics. In order for schools and educational institutions to have the right to process such personal data, there must be a legal basis. In Chapter 10, the Expert Group assesses various legal bases for learning analytics.

The Expert Group’s experience is that there is considerable uncertainty in the education sector regarding what legal basis they have for processing personal data in learning analytics. This uncertainty has different effects. In primary and secondary education and training, the Expert Group’s impression is that it varies between schools and municipalities as to what is considered lawful and justifiable use of resources with functionality for learning analytics. The range extends from relatively uncritical use – as also confirmed by the Norwegian Privacy Commission – to a more restrictive approach. This leads to considerable unpredictability for pupils, parents and suppliers (Parents’ Committee for Primary and Secondary Education, 2022a; ICT-Norway, 2023). In higher education, the Expert Group has found that the uncertainty regarding legal basis means that learning analytics is not included in pedagogical practice, as the strategic level of service providers and institutions limits access to functionality and resources. For the vocational colleges, the situation is somewhat more unclear, but the Expert Group’s impression is that the practice varies greatly between the different educational institutions.

The Expert Group believes that there is a need to clarify the legal basis for processing personal data in learning analytics. This will enhance predictability for pupils and students regarding how their personal data are processed throughout the educational pathway. In this chapter, we will present relevant options for clarifying the legal basis in the legislation governing the three levels of education. A clearer legal basis will provide a less ambiguous starting point for further practice involving learning analytics in the education sector.

12.1 Some broad challenges

The Expert Group points to three broad challenges in amending the legislation to clarify the legal basis for learning analytics.

The signalling effect – a clearer legal basis is not meant as an invitation

The purpose of clarifying the legal basis is not about more learning analytics being a goal in itself. It would be unfortunate if a legal basis was perceived as an invitation to carry out learning analytics that has limited pedagogical value.

The legal basis must take account of a rapidly developing technology

Changes happen quickly in the field of learning analytics. Legal bases that are too technology-specific and aimed at specific ways of processing personal data risk quickly becoming outdated and irrelevant.

More complexity in the legislation can result in less predictability

The existing provisions that the Expert Group has considered as possible bases for learning analytics are found in different parts of the legislation. By clarifying the legal basis, there is a risk of unnecessarily increasing complexity and further muddling the legislation. The consequence of more convoluted legislation is less predictability for pupils, students and controllers.

12.2 The Expert Group’s recommendations on the legal basis for learning analytics in primary and secondary education and training

The Expert Group recommends clarifying the legal basis for learning analytics in primary and secondary education and training. In this section, we present a proposal for how the legal basis for processing personal data in learning analytics can be clarified in the legislation governing primary and secondary education and training.

This proposal concerns establishing a separate provision on processing personal data in learning analytics, provided the processing is justifiable. As learning analytics, by definition, entails a high risk for data protection and as the processing can be invasive for the individual, the Expert Group believes that learning analytics should be regulated in a separate provision. In March 2023, the Norwegian Ministry presented a proposal for a new Education Act, where it has mainly been proposed to continue the provisions that have been discussed in this report Prop. 57 L (2022–2023). The Expert Group’s proposal for a provision is based on the general scheme in the proposal for a new Education Act. Our proposal will also apply to corresponding provisions in the Independent Schools Act.

12.2.1 Provision on the processing of personal data in learning analytics

Against the backdrop of the high risk for pupils’ privacy, we believe that a separate provision should be established that clarifies that pupils’ personal data can be processed in learning analytics. A separate provision is suitable to clarify that pupils’ personal data may be processed in learning analytics when this is necessary to solve the tasks in the Act, e.g., in connection with differentiated instruction, formative assessment and quality development.

The Expert Group believes that the provision should be included as a separate paragraph in section 25-1 of the Education Act, the provision on the processing of personal data in the proposal for a new Education Act, which in turn is a continuation of section 15-10 Prop. 57 L (2022–2023), section 54.5.2). The Expert Group described the current section 15-10 in section 10.3.1. The first paragraph of section 25-1 has a purely pedagogical purpose. The section clarifies that it is permissible to process personal data when this is necessary to perform a task in the Act, which is already stipulated in Article 6 of the GDPR. In addition, section 25-1, second and third paragraphs establish independent legal bases that apply to the processing of data in connection with changing schools and to prevent absence from education. Section 25-1 contains both broad guidelines and independent legal bases for processing personal data. The Expert Group finds that these statutory provisions are suitable for including a section on processing personal data in learning analytics.

The Expert Group is of the opinion that the provision on the processing of learning analytics will first constitute an independent legal basis according to Article 6(1)(e) of the GDPR if the provision establishes a basis for “the performance of a task carried out in the public interest or in the exercise of official authority”. If the provision only describes the processing of personal data in learning analytics and the tasks in the Act where the processing of personal data in learning analytics may be necessary, the provision will not constitute an independent legal basis. Our proposal will not constitute an independent legal basis, but it will establish clearer frameworks for the processing of the existing statutory tasks and duties.

The Expert Group emphasises that the term learning analytics will not be suitable in a statutory provision. The term learning analytics does not have a clear definition, there are different interpretations within the sector, and through the Expert Group’s meetings with young people it has become clear that pupils are not familiar with the term. However, this can be resolved by describing the types of processing of personal data that learning analytics entails.

In the following paragraphs, we will show how a provision on learning analytics can describe the processing of personal data, the tasks where it may be necessary to process personal data in learning analytics according to the law, and what qualifies as justifiable processing of personal data. We will then assess how the purpose of the processing can be specified and any other criteria that should be determined. Finally, we will describe what is required for the provision to constitute either an independent legal basis in accordance with the GDPR or a clarification of the existing bases in the legislation.

The provision should describe the processing

A separate provision on learning analytics in acts or regulations should describe how the processing of personal data is carried out. One question is whether the provision should contain an overview of all possible data that can be processed for learning analytics. The Norwegian Ministry of Education and Research discussed the issue before the Storting adopted the general provision on processing personal data in section 15-10 of the Education Act. The Norwegian Ministry concluded that it would not be appropriate for the provision to contain an overview of all types of data that may be processed pursuant to the Act. Firstly, the Norwegian Ministry believed that it is not possible to create an exhaustive list, and second, it was stated that a non-exhaustive list is more suitable in regulations or as guidelines Prop. 145 L (2020–2021), section 2.3.2.3). The Expert Group believes that the same assessment is relevant for learning analytics. It is not appropriate to create an overview of all types of personal data that can be processed in learning analytics. We propose to use the wording “machine analysis and alignment” to include the processing of personal data using artificial intelligence. This wording also covers the processing of personal data in learning analytics in primary and secondary education and training.

Necessity

The Expert Group is of the opinion that the provision should describe the tasks in the Act where it may be necessary to process personal data. We have identified the tasks differentiated instruction, formative assessment and quality development as relevant, but the provision should not exclude the possibility that other tasks may also be relevant. There is a risk that a provision which allows for the processing of personal data in learning analytics to be carried out to solve tasks in the Act could be interpreted such that it also allows for the processing of personal data for other tasks in the Act that are not relevant. The Expert Group emphasises that the proposal is not intended to facilitate increased processing of personal data in learning analytics in connection with performing other tasks in the Act where learning analytics is neither necessary nor justifiable. For tasks pursuant to the Act, school owners will have a processing basis if the requirement of necessity is met according to Article 6(1)(e) of the GDPR, cf. Article 6(3).

Justification

In addition to the provision having to describe the actual processing, the Expert Group is of the opinion that the provision must require that the processing be justifiable. The requirement for justification is about emphasising that the processing must be subject to the condition that the pedagogical and ethical aspects of the processing have been assessed. The interference with the individual pupil’s privacy can be significant in the case of learning analytics, and this requires a thorough assessment of the pedagogical and ethical aspects of the specific processing.

The Expert Group believes that processing personal data in learning analytics raises issues of a pedagogical and ethical nature. This has made it necessary for the text of the Act to emphasise that the processing must be ethically and pedagogically justifiable, so that the wording clearly conveys what is guiding for the assessment. What constitutes justifiable processing of personal data in learning analytics must be assessed individually in relation to the specific processing and in the light of recognised pedagogical and ethical principles. The Expert Group is of the opinion that the assessment of what constitutes justifiable processing in the individual case presupposes a minimum requirement of an assessment of the ethical and pedagogical challenges the group discussed in Chapter 8. At the same time, we recognise that the challenges will change over time, and that this will affect what should be included in the assessment of justification.

Specify the purposes of the processing

The current broad purpose of learning analytics derives from the objectives of education and the individual statutory tasks. Further specifying the purposes for the processing of personal data in a separate provision will clarify the legal bases for learning analytics.

The Expert Group notes that there is currently little experience-based knowledge on which specific purposes it is appropriate for learning analytics to have. Learning analytics is a complex practice that can be used for different purposes. In this report, we have identified several purposes that we believe have sufficient pedagogical value to justify the processing of personal data. The purposes we have identified do not constitute an exhaustive overview of the types of processing that will have sufficient pedagogical value. Therefore, the Expert Group does not believe it is suitable to include these purposes in the text of the Act. The following clarifications can nevertheless provide guidance on what constitutes justifiable and necessary processing of personal data in learning analytics. The details are based on the assessments in chapters 7 and 8.

1) Insight into the pupils’ learning

The Expert Group believes that learning analytics, which provides insight into pupils’ learning, can be useful for pupils and teachers.

Learning analytics that provides information to pupils is tempered by the fact that the teacher is involved and supports the pupils in interpreting the information from the analysis when necessary. Such information can, e.g., be presented through a visualisation, recommendations or a report that provides an overview of what the pupils have mastered and can contribute to participation and reflection on their own learning. We believe that this form of learning analytics and the teacher’s involvement must be differentiated according to the pupil’s age and maturity. Learning analytics that provides information to the teachers can form a basis for differentiating the instruction and providing formative assessments. Such information can, e.g., be presented through a visualisation, recommendations or a report that provides an overview of the pupils’ learning activities and academic progress.

2) Feedback and suggestions

The Expert Group believes that learning analytics that provides pupils and teachers with feedback and suggestions for instruction and learning based on pupil data can have pedagogical value.

For the pupils, this may entail that they receive recommendations from the learning resource regarding what they should work on next. Another example is that a pupil is assigned tasks in an adaptive resource based on how the pupil has previously solved tasks. Appropriate use of adaptive resources requires that the teacher has the opportunity to maintain a certain overview of how the pupils work with these resources. For example, the resource must facilitate so that the teacher can detect if pupils, for various reasons, receive feedback or suggestions that are incorrect in relation to the pupil’s actual needs. For the teachers, learning analytics can provide recommendations on learning activities and subject content based, among other things, on certain preferences, subjects, topics or methods. Learning analytics can also contribute to streamlining and individualising feedback to the individual pupil.

3) Work on quality development

The Expert Group believes that learning analytics, which provides information on learning and instruction, can be useful in quality development work in schools and with school owners.

Aggregated information on pupils’ academic development and on teaching practice can serve as a relevant source for the work on quality development. This will mainly apply to information that is comparable over time and across schools. Information that provides a broad description of the status quo will also be useful to support the schools in their quality development work.

Establish additional criteria for processing personal data in learning analytics

As shown in section 5.2.1, Article 6(3) of the GDPR allows for the stipulation of limitations in the legal basis for processing personal data. Such criteria could include storage limitation, further processing, accuracy and data minimisation. The Expert Group notes that learning analytics can involve processing personal data in different ways for different purposes. This makes it difficult to determine specific limitations that apply to all forms of learning analytics. Nevertheless, we believe it is crucial for pupils’ privacy that personal data that identifies pupils is not processed more than is necessary in relation to the purpose. The provisions that determine the tasks concerning differentiated instruction, formative assessment and quality development are open in terms of the types of information collected to carry out the tasks. Therefore, we propose that the proposal for a provision expressly states that the degree of personal identification shall not be greater than is necessary in relation to the purpose. This is a specification of the data minimisation principle in the Article 5(c) of the GDPR, and we note that it will rarely be necessary in relation to the purpose to process directly identifying personal data, especially in the case of quality development. This wording has also been used in other acts and regulations.[34]

Regarding further processing of personal data for purposes other than those stipulated in the legislation governing the education sector, the Expert Group is aware that this is carried out to further develop software, among other things. This is a commercial purpose on the part suppliers of digital resources and such further processing takes place without a legal basis (Bouvet, 2021; NOU 2022: 11). We believe this situation is unsustainable. However, the school owners are not able to do anything about this individually and it is therefore necessary for the national authorities to initiate a dialogue with the suppliers in order to clarify the issue.

The Expert Group also supports the Norwegian Privacy Commission’s proposal that:

[…] the Norwegian Government must initiate a broad investigation of digital tools that are currently in use in Norwegian schools and how they impact children’s privacy. Such an investigation should apply to all types of teaching aids and other methods and tools used in the teaching context. What control and monitoring possibilities these tools provide, what knowledge it is possible to extract from the data that are collected and stored and how the knowledge is used for the benefit of pupils and educational institutions, should be elucidated in such an investigation. Furthermore, it should be assessed how the collected personal data are further processed for various purposes. (NOU 2022: 11, pp. 136–137)

12.2.2 The Expert Group’s proposal on the legal basis for learning analytics in primary and secondary education and training

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in primary and secondary education and training. The proposal is based on the general scheme in the proposal for a new Education Act Prop. 57 L (2022–2023). The provision shall be inserted in the Education Act and in the corresponding provision in the Independent Schools Act:

The Expert Group proposes including a new paragraph in section 25-1 of the Education Act on the processing of personal data in learning analytics and the tasks in the Act where such processing will be necessary. Proposed new paragraph:

“Municipalities, county authorities and training establishments may process personal data about pupils and apprentices by means of machine analysis and alignment where this is ethically and pedagogically sound and necessary to perform tasks and duties in the Act and regulations pursuant to the Act. Examples of such tasks and duties may be to adapt the instruction, the work on quality development in section 17-12 and formative assessment in section 3-10 of the Regulations pursuant to the Education Act. The degree of personal identification shall not be greater than necessary for the purpose in question.”

12.3 The Expert Group’s recommendations on the legal basis for learning analytics in higher education

The review of the existing legal bases for processing personal data in learning analytics in higher education in section 10.4 showed that there is a glaring need to clarify the basis for processing personal data in learning analytics. The Expert Group believes that the provisions on the tasks and responsibilities of the institutions are not suitable for specifying and constituting a legal basis for processing personal data in learning analytics.

The Expert Group recommends clarifying the legal basis for learning analytics in higher education. In this section, we present two proposals to strengthen the legal basis for processing personal data in learning analytics in higher education. The first proposal concerns creating a separate provision on processing personal data in learning analytics. The second proposal concerns specifying one of the provisions on quality assurance work, to specify that the institution may process personal data in learning analytics in the quality assurance work. The proposals by the Expert Group will not constitute independent legal bases within the meaning of the GDPR because the proposals do not impose their own tasks on the institutions that make it necessary to process personal data. However, the proposals will establish clearer frameworks for processing personal data in learning analytics that occurs to carry out tasks in the legislation.

12.3.1 Provision on the processing of personal data in learning analytics

Section 4-15 of the Universities and University Colleges Act contains provisions on the processing of personal data. The first paragraph of the provision stipulates that the educational institutions may process personal data “when the purpose of the processing is to safeguard the rights of the data subject, or to fulfil the institution’s tasks and duties under the Universities and University Colleges Act.” This does not constitute an independent legal basis as the provision merely reiterates what is stipulated in Article 6(1)(e) and (3), i.e., that personal data may be processed when the “processing is necessary for the performance of a task carried out in the public interest” and the task is laid down in national legislation. The Expert Group is of the opinion that section 4-15 on the processing of personal data is a suitable place to establish a provision on the processing of personal data in learning analytics.

In the following paragraphs, we will show how a provision on learning analytics can describe the processing of personal data and the tasks where it may be necessary to process personal data in learning analytics according to the law, and what qualifies as justifiable processing. Next, we will consider how the provision will specify the purpose of the processing.

The provision should describe the processing

A clarification of the provision presupposes that the processing of personal data is described. As the Expert Group showed in section 12.2.1, it is neither appropriate to use the term learning analytics in a provision nor to list all the information that would be relevant to process in learning analytics. This assessment also applies to the processing of personal data in higher education. We propose to use the wording “machine analysis and alignment” to include the processing of personal data using artificial intelligence. This wording also covers the processing of personal data in learning analytics in higher education.

Necessity

The Expert Group is of the opinion that the provision should describe the tasks in the Act where it may be necessary to process personal data. The Expert Group believes that processing personal data in learning analytics is relevant, among other things, to fulfil the following tasks and duties, but that the provision should not exclude the possibility that other tasks may also be relevant:

The responsibility for offering higher education based on the foremost within research in section 1-3a, and ensuring that teaching maintains a high professional level and is conducted in accordance with recognised scientific, pedagogical and ethical principles in section 1-5, first paragraph.

Having a satisfactory internal system for quality assurance in section 1-6, and requirements for systematic quality assurance work in section 2-1 of the Regulations on the quality of programmes of study and section 4-1 of the Academic Supervision Regulations.

There is a risk that a provision that mentions examples of tasks for which learning analytics may be relevant could be interpreted such that it also allows for personal data to be processed for other tasks in the Act that are not relevant. The Expert Group emphasises that this proposed wording is not intended to facilitate increased processing of personal data in learning analytics where this is neither necessary nor justifiable. For tasks pursuant to the Act, the institutions will have a processing basis if the requirement of necessity is met according to Article 6(1)(e) of the GDPR, cf. Article 6(3).

Justification

The Expert Group believes that processing personal data in learning analytics raises issues of a pedagogical and ethical nature. This has made it necessary for the text of the Act to emphasise that the processing must be ethically and pedagogically justifiable, so that the wording clearly conveys what is guiding for the assessment. What constitutes justifiable processing of personal data in learning analytics must be assessed individually in relation to the specific processing and in the light of recognised pedagogical and ethical principles. The Expert Group is of the opinion that the assessment of what constitutes justifiable processing in the individual case presupposes a minimum requirement of an assessment of the ethical and pedagogical challenges the group discussed in Chapter 8. At the same time, we recognise that the challenges will change over time, and that this will affect what should be included in the assessment of justification.

In addition to the provision having to describe the actual processing of personal data, the Expert Group is of the opinion that the provision must require that the processing be justifiable. The requirement for justification is about clarifying that the processing must be subject to the condition that the pedagogical and ethical aspects of the processing have been assessed. The interference with the individual student’s privacy can be significant in learning analytics, and this requires a thorough assessment of the pedagogical and ethical aspects of the specific processing.

Specify the purposes of the processing

The current broad purpose of learning analytics derives from the objectives of education and the individual statutory tasks. Further specifying the purposes for the processing of personal data in a separate provision will clarify the legal bases for learning analytics.

The Expert Group notes that there is currently little experience-based knowledge on which specific purposes it is appropriate for learning analytics to have. Learning analytics is a complex practice that can be used for different purposes. In this report, we have identified several purposes that we believe have sufficient pedagogical value to justify the processing of personal data. The purposes we have identified do not constitute an exhaustive overview of the types of processing that will have sufficient pedagogical value. Therefore, the Expert Group does not believe it is suitable to include these purposes in the text of the Act. The following clarifications can nevertheless provide guidance on what constitutes justifiable and necessary processing of personal data in learning analytics. The details are based on the assessments in chapters 7 and 8.

1) Active learning

The Expert Group believes that learning analytics, which provides students with insight into their own learning, can be useful in their learning process.

Information regarding which activities the students have carried out and what academic results they have achieved in various subjects during their educational pathway, can contribute to self-regulation, participation and involvement. The Expert Group believes that students should be involved in learning analytics processes at the educational institution. The students must also receive sufficient guidance to be able to understand, interpret and make use of the information from learning analytics in order to meet the objective of increased insight into their own learning processes and active learning.

2) Student follow-up

The Expert Group believes that learning analytics that offers information about learning and teaching can have pedagogical value for teachers.

Data about students from digital resources can provide information to teachers about how the students use the resources available to them. This can support instructors in following up students and structuring teaching. For learning analytics with this purpose, it is key that teachers and students work together to interpret the information and jointly determine what significance the analyses will have for the further development of teaching.

3) Quality assurance work

The Expert Group believes that learning analytics that provides information on students’ academic development and on teaching practice can serve as a relevant source for the quality assurance work at educational institutions.

The Expert Group believes that the general rule must be that the data processed for quality assurance work must be de-identified. Processing personal data for quality assurance work requires the necessary guarantees in accordance with Article 89 of the GDPR to ensure pupils’ and students’ rights and freedoms. The guarantees shall ensure that technical and organisational measures have been introduced to ensure compliance with the principle of data minimisation. Such measures include pseudonymisation, de-identification, aggregation or anonymisation. The decision on what degree of identification is permitted must be based on a risk assessment linked to the types of personal data included in the analysis.

12.3.2 Specification of the provision on quality assurance work

The provisions on quality assurance work are divided between three different sets of acts and regulations. Section 1-6 of the Universities and University Colleges Act lays down the broad provision that the institutions “shall have a satisfactory internal system for quality assurance that will ensure and further develop the quality of the education”. The objectives and requirements for the content of the quality assurance work are stipulated in section 2-1 of the Regulations on the quality of programmes of study and section 4-1 of the Academic Supervision Regulations. These provisions specify to a greater extent how the institutions should work on quality development. In order to support the general scheme in the current legislation, a specification to clarify the basis for processing personal data in learning analytics would be best suited to be included in the provision that regulates how the quality assurance work should take place.

Section 4-1 of the Academic Supervision Regulations states, among other things, that the “[i]nstitution shall systematically collect information from relevant sources in order to be able to assess the quality of all study programmes”. The wording is open in terms of what kind of information is collected for the quality assurance work. Therefore, the Expert Group believes that the provision should explicitly show that the degree of personal identification should not be greater than what will be necessary for the purpose, as discussed in more detail in section 12.2.1. The student unions have been concerned that personal data are processed in systems which unnecessarily identify the individual student. A requirement that the degree of personal identification must be necessary for the purpose clarifies how the Expert Group believes the principle of data minimisation in Article 5(1)(c) of the GDPR should be specified in connection with quality assurance work.

12.3.3 The Expert Group’s proposal on the legal basis for learning analytics in higher education

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in higher education. The provisions shall be inserted in the Universities and University Colleges Act and the Regulations pursuant to the Act:

The Expert Group proposes inserting a new paragraph in section 4-15 of the Universities and University Colleges Act on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:

“The educational institution may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties include quality assurance work and the responsibility to ensure that instruction is provided in accordance with recognised ethical and pedagogical principles, cf. section 1-5.”

The Expert Group proposes specifying the provisions on quality assurance work in section 4-1 of the Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:

“The institutions may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”

12.4 The Expert Group’s recommendations on the legal basis for learning analytics in tertiary vocational education

The Expert Group recommends that the legal basis for learning analytics in vocational colleges should be clarified. The Expert Group notes that the assessments in section 12.3 on processing bases for higher education are largely transferable to the processing of personal data in learning analytics in tertiary vocational education.

The Expert Group also believes that it is appropriate that the design of the legal basis for processing of personal data in learning analytics be identical for tertiary vocational education and for higher education. Therefore, the Expert Group will in this section only present changes in proposals for legislation and refers to section 12.3 for a description of the background for the recommendations.

12.4.1 Provision on the processing of personal data in learning analytics

The processing of personal data at vocational colleges is regulated in section 4 of the Vocational Education Regulations. The Expert Group proposes to add a new paragraph to the provision that applies to the processing of personal data in learning analytics.

The Expert Group believes that processing personal data in learning analytics is relevant, among other things, to fulfil the following tasks and duties, but that the provision should not exclude the possibility that other tasks may also be relevant:

Quality assurance work pursuant to section 5 of the Vocational Education Act and section 4-1 of the Vocational Education Academic Supervision Regulations

The requirement that the vocational colleges must have learning and teaching methods that are suitable for the students to achieve the learning outcome in section 2-1 of the Vocational Education Academic Supervision Regulations

12.4.2 Specification of the provision on quality assurance work

The Expert Group believes it is also relevant to specify the provision on quality assurance work in vocational colleges in a similar way to what the Expert Group has proposed in the legislation governing higher education, cf. section 12.3.2. Section 5 of the Vocational Education Act states that vocational colleges must have systems for quality assurance. The detailed content of the requirement for the vocational college’s quality assurance work is regulated in the Vocational Education Academic Supervision Regulations. Section 5, sixth paragraph, letter d) of the Vocational Education Act states that the Norwegian Ministry has the right to issue regulations on requirements for quality assurance work. Section 4-1, third paragraph of the Vocational Education Academic Supervision Regulations states, among other things, that the vocational colleges must “systematically collect […] information from students, employees, representatives from the professional field and any other relevant sources”. The Expert Group believes it would be appropriate to specify in the same provision that vocational colleges may process personal data using learning analytics in connection with quality assurance work when this is necessary and justifiable.

12.4.3 The Expert Group’s proposal on the legal basis for learning analytics in tertiary vocational education

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in tertiary vocational education. The provisions shall be inserted in the Vocational Education Act and the Regulations pursuant to the Act:

The Expert Group proposes inserting a new paragraph in section 4 of the Vocational Education Regulations on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:

“The vocational colleges may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties may be quality assurance work and having learning and instruction methods that are suitable for the students to achieve the learning outcomes, cf. section 2-1 of the Vocational Education Academic Supervision Regulations.”

The Expert Group proposes specifying the provisions on quality assurance work in section 4-1, third paragraph of the Vocational Education Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:

“The vocational colleges may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”

12.5 Summary of the Expert Group’s recommendations and proposals on the legal basis for learning analytics

Primary and secondary education and training

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in primary and secondary education and training. The proposal is based on the general scheme in the proposal for a new Education Act Prop. 57 L (2022–2023). The provision shall be inserted in the Education Act and in the corresponding provision in the Independent Schools Act:

The Expert Group proposes including a new paragraph in section 25-1 of the Education Act on the processing of personal data in learning analytics and the tasks in the Act where such processing will be necessary. Proposed new paragraph:

“Municipalities, county authorities and training establishments may process personal data about pupils and apprentices by means of machine analysis and alignment where this is ethically and pedagogically sound and necessary to perform tasks and duties in the Act and regulations pursuant to the Act. Examples of such tasks and duties may be to adapt the instruction, the work on quality development in section 17-12 and formative assessment in section 3-10 of the Regulations pursuant to the Education Act. The degree of personal identification shall not be greater than necessary for the purpose in question.”

Higher education

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in higher education. The provisions shall be inserted in the Universities and University Colleges Act and the Regulations pursuant to the Act:

The Expert Group proposes inserting a new paragraph in section 4-15 of the Universities and University Colleges Act on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:

“The educational institution may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties include quality assurance work and the responsibility to ensure that instruction is provided in accordance with recognised ethical and pedagogical principles, cf. section 1-5.”

The Expert Group proposes specifying the provisions on quality assurance work in section 4-1 of the Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:

“The institutions may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”

Tertiary vocational education

The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in tertiary vocational education. The provisions shall be inserted in the Vocational Education Act and the Regulations pursuant to the Act:

The Expert Group proposes inserting a new paragraph in section 4 of the Vocational Education Regulations on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:

“The vocational colleges may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties may be quality assurance work and having learning and instruction methods that are suitable for the students to achieve the learning outcomes, cf. section 2-1 of the Vocational Education Academic Supervision Regulations.”

The Expert Group proposes specifying the provisions on quality assurance work in section 4-1, third paragraph of the Vocational Education Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:

“The vocational colleges may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”

13 Code of conduct for data protection in primary and secondary education and training (School Code of Conduct)

In the vast majority of cases, learning analytics in primary and secondary education and training requires the processing of personal data. Data protection must therefore be safeguarded for learning analytics to be justifiable. The right to privacy is enshrined in international and national legislation. The legislation in this area is general and risk oriented. This means that for processing that is considered to be invasive (high risk), there must be mechanisms other than the text of the Act in place to safeguard the basic principles of the GDPR.

The Expert Group’s clear opinion is that there are currently inadequate arrangements to safeguard privacy in Norwegian schools. This is also supported by the assessments of the Norwegian Privacy Commission (NOU 2022: 11). Many of the provisions in the area of data protection are vague and ambiguous, entailing that it can be difficult to translate them into practice. Inadequate safeguarding of data protection can threaten the public’s trust in schools. The Expert Group believes there is a need for greater governance of learning analytics than is found in the legislation, and that the best solution is to draft a code of conduct to safeguard data protection in schools.

13.1 What is a data protection code of conduct?

A data protection code of conduct is a collection of guidelines that enterprises within an industry or sector agree to observe. The guidelines can be designed as several different measures. The purpose of the code of conduct is for the provisions in the legislation to be supplemented through specific requirements for, among other things, organisational, technical and pedagogical measures to achieve satisfactory data protection. In addition to guidelines, additional mechanisms can also be included in such a code of conduct. Such mechanisms may include various support functions that are necessary to ensure good implementation of the guidelines. Developing and working on the basis of such a code of conduct is an important part of achieving the goal of the code of conduct as it facilitates competence development, a shared understanding and equal practice.

In the health and care sector, a data protection and information security code of conduct has been drawn up with great support in the sector. In the health and care sector, this is referred to as Normen (Health and Care Sector Code of Conduct).[35] In the GDPR, the intention is for such a code of conduct to be approved by the Norwegian Data Protection Authority. In order for a collection of guidelines to be approved as a code of conduct, there are a number of formal requirements that must be met. The Health and Care Sector Code of Conduct does not meet all these criteria, and is therefore not categorised as a code of conduct.

13.2 The need for a code of conduct for data protection in schools

The need for a code of conduct to safeguard data protection in schools has been put forward in several contexts. The Norwegian Privacy Commission recommended that state authorities be required take the initiative to draw up a data protection code of conduct for the school and kindergarten sector (NOU 2022: 11). The Commission notes that a data protection code of conduct can better enable municipalities and county authorities to safeguard their processing responsibilities.

The Education Act Committee recommends that codes of conduct be drawn up for the education sector, and highlights learning analytics as a particularly relevant area for a code of conduct:

For municipalities as controllers, it may be relevant to have codes of conduct pertaining to learning analytics. For example, general codes of conduct can be drawn up for the purchase and use of tools for learning analytics, or in relation to a particular tool. Furthermore, publishers and other processors can draw up codes of conduct to ensure compliance with the requirements in the GDPR that apply to them. This could involve codes of conduct on privacy by design. (NOU 2019: 23, p. 70)

We have received several comments calling for a code of conduct to safeguard data protection, including from the industry organisation ICT-Norway (2023):

ICT-Norway supports the intention regarding standard-setting guidelines that can ensure uniform application of the legislation across municipalities and suppliers. A data protection code of conduct for schools can contribute to a more uniform application of requirements for data protection and security for digital teaching aids and make it easier for suppliers to understand what requirements apply to their products. (p. 2)

In order for a code of conduct to aid school owners in complying with the data protection legislation, it should be aimed at the entire life cycle and all target groups of the technology. The entire service life cycle spans from planning, via design, coding, testing, commissioning, to use, management and decommissioning. The most important target groups are developers, suppliers, school owners, school administrators, teachers, pupils and parents. We stress that the different target groups have different needs. However, they also have an urgent and overlapping need for fixed guidelines governing data protection in schools. Therefore, it is hugely beneficial if the content of a code of conduct corresponds to the needs of all these target groups for guidance and assistance in the various phases of the service life cycle.

13.3 Three conditions for the School Code of Conduct

We will highlight three special conditions for the School Code of Conduct. The first condition is that the development of the code of conduct should be based on existing materials and measures in the school sector. The remaining conditions concern two areas to which particular attention should be directed – the data protection risk when using open resources and resources that are autonomous systems.

Building on existing work

There are already several initiatives and measures in place to strengthen data protection in schools. Among them are the Norwegian Association of Local and Regional Authorities’ (KS) project SkoleSec[36] and the Norwegian Directorate for Education and Training’s resource pages on data protection in schools[37] . These resources include extensive guidance material, templates for contracts and assessments, tools for competence development and development projects to support school owners in safeguarding their processing responsibilities. These resources have largely been developed in collaboration with the sector, and the Expert Group believes they form a good starting point for drawing up the School Code of Conduct.

License-based and open resources

The code of conduct is intended to regulate all resources with functionality for learning analytics that schools use, regardless of whether or not they are license-based. Before the schools adopt license-based resources with functionality for learning analytics, they must undertake a comprehensive procurement process. However, services that are openly available online are prevalent in schools. Using open resources often occurs without the approval of management or others with data protection competence. This entails a major privacy risk for pupils (Bouvet, 2021).

The Norwegian Privacy Commission investigated services in schools where “payment is made in the form of children’s personal data” (NOU 2022: 11). The Commission’s work clearly demonstrates that the safeguarding of pupils’ privacy is a major challenge in relation to such services. One of the reasons why the use of open resources is high risk is that data protection impact assessments are generally more thorough when procuring a license-based resource compared to when the school uses an open resource. Another reason relates to commercial matters. The Norwegian Data Protection Authority (2021b) notes that suppliers of open services reuse personal data as part of their business model. This entails a high risk of personal data being used for purposes for which there is no legal basis. In order to reduce the privacy risk in schools, it is crucial that the code of conduct also addresses open resources.

Autonomous systems

Adaptive teaching aids and assessments make an automated, individual adaptation according to the pupil’s situation with the aid of artificial intelligence. Resources that act without human involvement are referred to as autonomous systems or automated decision systems. Questions regarding who is responsible for the consequences of the decisions made by such systems, and how to delimit such autonomy, have been major topics in discussions on ethics and artificial intelligence in recent years (Norwegian Ministry of Local Government and Modernisation, 2020). As adaptive teaching aids and tests fall under the definition of autonomous systems, we believe it is particularly important to pay attention to them in the code of conduct.

13.4 The Expert Group’s proposal for content in the School Code of Conduct

In this section, we will provide a brief and general description of four proposals that are particularly important for safeguarding data protection in learning analytics:

development and administration of specific data protection requirements in resources that have functionality for learning analytics

preparation and administration of guidance materials for school owners, teachers, pupils, parents, developers and suppliers

development and administration of national data protection impact assessments for resources that have functionality for learning analytics

facilitation of competence development and exchange of experience related to data protection in schools

13.4.1 Specific data protection requirements in resources that have functionality for learning analytics

Key to the School Code of Conduct is the development of requirements for the use and development of resources with functionality for learning analytics, including through privacy by design. The requirements can be directed at controllers as well as developers and suppliers. Currently, it is often the suppliers who in practice assess which functions the tools should have, e.g., storage time and what is visible to the teacher (ICT-Norway, 2023).

In 2021, the Norwegian Directorate for Education and Training was tasked with creating a general guide on which data protection requirements should be imposed on suppliers of digital learning resources (Norwegian Ministry of Education and Research, 2021). Experience from this work and from the development of security requirements in SkoleSec may be useful when data protection requirements are to be developed in the School Code of Conduct. The requirements can also form a starting point for preparing criteria-based guides that school owners can use in the procurement of resources with functionality for learning analytics.

In the specific requirements for data protection, it may be relevant to specify, among other things, what will be necessary to process personal data based on the purposes set out in the legal bases. We have identified the following four data protection principles that have proven to be particularly challenged by learning analytics and artificial intelligence:

fairness (uncritical use of analyses, little co-determination)

transparency (closed business models, dynamic algorithms)

data minimisation (collection of data without a clear plan for use)

accuracy (biases in the supporting data).

The specific requirements for data protection in resources with functionality for learning analytics should be particularly aimed at reducing risk within these four areas. We will highlight some examples of which requirements may be relevant to ensuring fairness, transparency and accuracy, and to safeguarding data minimisation in learning analytics to a greater extent than is currently the case.

Fairness

A basic requirement to safeguard fairness is that the supplier provides information about how the data are processed, in an objective and neutral manner, and that the supplier does not use misleading or manipulative language or design.

To ensure fairness, it will be key to facilitate in order for pupils and parents to exercise their rights. This involves, among other things, incorporating requirements that grant pupils and parents access and the opportunity to rectify and erase data. Fairness is also promoted if the resources have functionality for data portability, which makes it possible to extract data in a reusable format. In addition, the possibility for users to control whether their data are used in learning analytics can enhance fairness. Functionality for enabling and disabling learning analytics must be weighed against pedagogical needs and what is practically feasible to implement. However, the default setting should be that functionality for learning analytics is disabled. Such requirements can contribute to ensuring participation in learning analytics.

Algorithms that contribute to maintaining or amplifying discrimination are a significant threat to fair use of artificial intelligence. Therefore, mechanisms to detect and remedy this are key requirements for consideration. Regularly testing whether the algorithms work in line with the purposes, and adjusting the algorithms to reduce biases are relevant examples of such mechanisms.

Transparency

Transparency around the use of personal data is necessary to preserve pupils’ and parents’ trust in the school and the trust they have in suppliers of learning resources. Requirements that can safeguard transparency are largely linked to information that is aimed at different target groups. Examples of relevant information include

an easily understandable and adequate description of what the solution actually does

an outline of data flow and processing protocol[38]

an explanation of how the algorithm weights variables, and the accuracy of the algorithm

making visible what and whence the information is collected, and how it is interpreted in the analysis

Although the information may be of a complex and technical nature, it must be accessible to school owners, school administrators and teachers with general technical competence. This is a prerequisite for being able to decide whether the best interests of the child have been assessed and safeguarded by the manner in which the resource has been developed. The information must also be possible to convey to pupils and parents. Pictures, icons and symbols can be used to make the information clearer. Animation, video and sound can be good tools for adapting the information to the target groups.

Data minimisation

The principle of data minimisation is about limiting the amount of data collected and processed to what is necessary to achieve the purpose. It is necessary to set requirements for mechanisms that ensure that, by default, only personal data that are necessary for the purpose of the processing are collected. A specification of this could be that the resource contains barriers to the linking of personal data in other systems or that have been collected for other purposes. Another way to comply with the principle of data minimisation is to require the removal or masking of directly identifying data (pseudonymisation) once such identification is no longer necessary. This can apply to, e.g., the training of algorithms and quality development of the resource.

Accuracy

In order to ensure the accuracy of analyses, there should be a requirement that the source data are quality assured and validated prior to being used in learning analytics. The Norwegian Privacy Commission notes that biases can occur when digital tools lack transparency (NOU 2022: 11). Such biases are amplified if the solutions are used uncritically or fed flawed training data Therefore, it will be relevant to require built-in regular testing for biases in the data material, the models or in the use of the algorithms. In addition, there should be a requirement to re-train the algorithms if the accuracy falls below a predetermined threshold.

13.4.2 Guidance material for school owners, teachers, pupils, parents, developers and suppliers

Guidance material that elaborates on the requirements of the School Code of Conduct and supports the various roles in safeguarding their responsibilities is a prerequisite for the code of conduct to function as intended. The guidelines must be developed in collaboration with the school sector and be adapted to the target groups.

There are already many good data protection guides that are openly available and developed by competent actors. In table 13.1, we have listed some of these guides, their target groups, and which phase of the technology life cycle is covered. None of the resources are designed for learning analytics specifically, nor are any of the resources aimed at all target groups of relevance to learning analytics.

Table 13.1 

The Norwegian Directorate for Education and Training and the Norwegian Data Protection Authority:

Dubestemmer.no¹

Udir.no²

Norwegian Association of Local and Regional Authorities (KS):

Skolesec³

KiNS⁴

Norwegian Data Protection Authority⁵

Target group

developers

X

X

suppliers

X

X

X

school owners

X

X

X

X

school administrators

X

X

teachers

X

X

X

pupils

X

X

parents

X

X

X

Phase

development

X

order

X

X

X

implementation

X

X

X

use

X

X

X

X

Addresses

data protection

X

X

(X)

(X)

X

information security

X

X

X

ethics

X

X

X

pedagogical matters

X

X

¹ https://www.dubestemmer.no/

² https://www.udir.no/regelverk-og-tilsyn/personvern-for-barnehage-og-skole/

³ https://www.ks.no/fagomrader/digitalisering/felleslosninger/skolesec/

⁴ https://kins.no/verktoykasse/

5 https://www.datatilsynet.no/rettigheter-og-plikter/virksomhetenes-plikter/innebygd-personvern/programvareutvikling-med-innebygd-personvern/

The Norwegian Data Protection Authority’s guide stands out in the sense that it is the only one that specifically targets service developers. However, if it is to be used in a code of conduct aimed at schools, it must be homed in on the school sector.

The Expert Group notes that guidance material as part of the School Code of Conduct can contribute the following:

Gathering resources and tools

Many useful tools already exist, but are fragmented such that it is difficult to gain an overview. The School Code of Conduct can help create the necessary overview.

Interdisciplinarity

Several of the resources contain both elements of law and technological requirements, but few address requirements related to quality of education. The School Code of Conduct can contribute by contextualising these perspectives, and updates should also occur by way of such contextualisation.

Communication

Guidance linked to the code of conduct can contribute to target group outreach, which can underpin the objective of increased shared understanding across the target groups. An important aspect of this is to ensure that the various guides do not contradict each other.

13.4.3 National Data Protection Impact Assessments (DPIAs)

The School Code of Conduct is a suitable arrangement for identifying a better solution for assessing data protection consequences for learning analytics in school. The Norwegian Data Protection Authority states that processing personal data to evaluate learning, coping and well-being in school always requires a DPIA to be carried out (Norwegian Data Protection Authority, 2019). This requirement will cover the vast majority of forms of learning analytics. There is a considerable need to carry out DPIAs of resources with functionality for learning analytics in a more efficient and qualified manner.

As controllers, all school owners are subject to the same statutory requirements to assess data protection risks, but have different prerequisites for carrying out such assessments. A likely consequence is that pupils do not receive an equal provision of digital learning resources across municipalities because the municipalities fail to use resources due to their lack of capacity and competence to assess privacy risks (Høiseth-Gilje et al., 2022).

The Expert Group notes that there is a great deal of uncertainty among the school owners in the process of assessing whether the use of a resource in the school entails an acceptable privacy risk. This leads to marked differences in what is considered justifiable learning analytics among schools and municipalities (Parents’ Committee for Primary and Secondary Education, 2022a). It also leads to unpredictability for schools, pupils, parents, suppliers and developers (ICT-Norway, 2023). Although the responsibility for the DPIAs is clearly placed with the school owners, in many cases they do not have sufficient competence to carry out a satisfactory assessment. Such an assessment also requires considerable resources from each individual municipality.

A national actor may be responsible for preparing and managing risk analyses and general DPIAs with associated data processor agreements. Nevertheless, it is the controllers who are to carry out the residual risk assessment and local adaptation. Nationally prepared DPIAs of resources with functionality for learning analytics as part of the School Code of Conduct can contribute to countervailing differences, ensuring justifiable assessments of high quality and reducing the use of public resources. It is important that the actor that carries out the assessments has specialised competence in assessing privacy risks, particularly within core public institutions that include vulnerable groups, such as children. According to Article 35(1) of the GDPR, a single assessment of data protection consequences may address a set of similar processing operations that present similar high risks. Therefore, it may be appropriate to prepare joint DPIAs for similar services.

For the procurement of digital teaching aids, the most socio-economically expedient option is considered to be for a single national actor to carry out DPIAs (Høiseth-Gilje et al., 2022). According to a socio-economic analysis of the procurement of digital teaching aids, the current procurement system is inefficient and does not ensure that all pupils have equal access to high-quality digital teaching aids that meet requirements for data protection, information security and universal design (Høiseth-Gilje et al., 2022). The report recommends centralising parts of the activities involved in purchasing digital teaching aids, including obtaining information and carrying out parts of DPIAs and risk and vulnerability analyses of digital teaching aids, as well as drawing up standardised data processor agreements.

The Expert Group recognises that national DPIAs are a powerful instrument that can challenge local self-government and autonomy. There is also a risk that the responsibility for safeguarding data protection in schools may be shifted or pulverised. Therefore, it is important to carry out a thorough investigation of how national DPIAs can be prepared and managed in the best possible manner. We emphasise that the potential impact on the market and connection with the procurement legislation are important components of such an investigation. It will be particularly relevant to identify good solutions to ensure equal treatment of digital learning resources – both licensed and open resources – to avoid distortion of competition.

The Expert Group emphasises that, as it is the school owners who have the statutory responsibility for data protection in schools, an arrangement involving national DPIAs means that the school owners carry out residual risk assessments and any adaptations of the national assessments and agreements. We recognise that there is a need to offer support to school owners in making residual risk assessments, e.g., through centrally prepared guidelines. This is particularly important as many of the country’s municipalities do not have employees with legal competence (Juristforbundet, 2021).

Textbox 13.1 Project on national assessment of data protection consequences¹

KS and the City of Bergen have launched a project to implement and test a national assessment of data protection consequences (DPIA) for Google’s products and services in schools. The aim is to have an overall national DPIA in place by the end of 2023.

Such a broad national DPIA will ensure sufficient competence and capacity in the assessments and enable the municipalities to make assessments of residual risk for the use of the services. In addition to a broad DPIA, the project will prepare an accompanying guide to adapt and anchor the DPIA to the individual municipality.

In addition to having a national broad but thorough DPIA prepared for Google’s services, an aim of the project is to gather experience in order to co-govern and coordinate processes for data protection impact assessments. These experiences will be transferrable to national assessments of similar services, e.g., Microsoft Office 365. The experiences will also be relevant for other services and solutions used in schools.

¹ https://www.ks.no/fagomrader/digitalisering/felleslosninger/skolesec/personvernkonsekvenser-for-googles-produkter-i-skolen-skal-vurderes/

Sharing of municipally prepared DPIAs

Establishing national DPIAs will take time, and the privacy situation in schools is precarious (NOU 2022: 11). According to the Norwegian Privacy Commission, schools have generally failed to safeguard pupils’ privacy and adequately manage their personal data. As mentioned, there are major differences in terms of the assessments carried out by the schools.

Several municipalities have already prepared thorough DPIAs of resources with functionality for learning analytics, which will largely be relevant for other municipalities. Most school owners have similar considerations and needs, and the school sector is generally well suited for sharing and simultaneous use. Until national measures related to DPIAs are in place, a first step should therefore be to arrange for the municipalities to share parts of their DPIAs. This will ensure more equality in education – as it will reduce the likelihood that municipalities assess different resources in completely different ways – and serve as quality assurance for municipalities with less legal competence seeking to build on the assessments of others. The municipally prepared DPIAs can also serve as a starting point when their national counterparts are to be prepared.

13.4.4 Facilitation of competence development and exchange of experience

Safeguarding data protection in learning analytics in schools requires a high level of competence. The ordering competence of school owners varies. Many utilise support functions such as the Norwegian Directorate of Education and Training’s guides for assessment of quality in teaching aids, risk assessment, cloud services and data protection consequences.[39]

One of the Expert Group’s proposals in the School Code of Conduct is to facilitate the exchange of experience and develop data protection competence. A good solution would be to add this function to an already existing network, e.g., the municipal sector’s regional digitalisation networks and other national cooperation arenas between authorities and KS for digitalisation in the developmental environment. The competence development that takes place in such a network must also contribute to guiding suppliers in order to strengthen technological development that is in compliance with the requirements.

In addition to serving as an arena for competence development and exchange of experience, such a network can also facilitate the development of pilot projects and testing environments.

13.5 The Expert Group’s proposal on management of and participation in the School Code of Conduct

In order for the School Code of Conduct to have a regulatory function, the Expert Group believes it is necessary to have considerable support among school owners, schools, developers and suppliers. The School Code of Conduct must also be managed appropriately, with good involvement and anchoring in the school sector. There are several alternative solutions that can contribute to ensuring good management and a high degree of participation.

13.5.1 Management and coordination responsibility for the School Code of Conduct

In order to ensure that the School Code of Conduct is sufficiently anchored in the sector, it is crucial to establish mechanisms for management and coordination that will be suitable to realise the purpose of the code of conduct. This will entail, among other things, developing the data protection requirements in the code of conduct, carrying out national data protection impact assessments, establishing data processor agreements and preparing guidance material.

Governance model

One option for governance is to establish a steering group consisting of representatives from key actors and user groups. This can contribute to ensuring adequate anchoring of the School Code of Conduct. The steering group can be consensus-driven, as in the data protection and information security code of conduct in the health and care sector. We also believe that an important measure will be to include actors with observer status in the steering group, such as representatives from the Norwegian Data Protection Authority, industry organisations, relevant research environments and others. One disadvantage of a consensus-based governance model is that it is less suited for strong governance. Nevertheless, we believe it is essential that the School Code of Conduct ensures coordination and anchoring in the sector, and that a consensus-based governance mechanism would be suitable for this purpose.

Another possible governance model is that a central government actor – e.g., the Norwegian Directorate for Education and Training or Sikt – be given responsibility for administering the School Code of Conduct. A solution involving a central government actor as administrator of the code of conduct could result in clearer national governance, at the same time as there would be a need for mechanisms to ensure ownership and support in the school sector.

A third option is the establishment of a new body with interdisciplinary competence suitable for administering the School Code of Conduct.

Coordination

An option for coordinating the School Code of Conduct is the establishment of an independent secretariat under e.g., the Norwegian Directorate for Education and Training or Sikt. The secretariat may be responsible for following up the decisions made by the steering group and, e.g., preparing proposals for data protection requirements and guidance material. The Health and Care Sector Code of Conduct uses such a coordination model with a working secretariat under the Norwegian Directorate of eHealth.

Another option is for KS to be responsible for coordination. The association is perhaps closer to the school sector and is an interest group in the policy-making process distinct from central government actors.

13.5.2 Support for the School Code of Conduct

The least binding model for support among school owners is an optional commitment to the code of conduct. Such an arrangement would be consistent with the strong local self-government we have in Norway. Several school owners express to the Expert Group that they need significantly more assistance in the area of data protection, which may indicate acceptance of a model involving stronger central governance. The Expert Group notes that there is a high level of motivation on the part of school owners to endorse centralised arrangements that make it more predictable and manageable to handle processing responsibilities.

One model with a stronger incentive for participation on the part of both suppliers and municipalities is to closely link the School Code of Conduct with a national service catalogue for digital learning resources. In the digitalisation strategy for schools, one of the measures is to establish a publicly managed national service catalogue for digital teaching aids (Norwegian Ministry of Education and Research, 2023). The purpose of such a service catalogue is to provide the municipalities and schools with a better overview of the market, while at the same time ensuring free choice and a wealth of options. The catalogue may include an overview and description of digital teaching aids, information about available licences, statistics and analysis of use, as well as assessments in relation to requirements for data protection, universal design and language variant. The data protection requirements included in the code of conduct can be key criteria for inclusion in the national service catalogue. The Norwegian Privacy Commission emphasises that a national service catalogue is an important initiative to strengthen pupils’ privacy if clear requirements for data protection and information security are included as criteria for including the learning resources in the catalogue (NOU 2022: 11). We stress that we have not considered such a link in relation to the procurement legislation and other legislation that regulates the market, but are aware that there may be a need to investigate new interpretations and possible amendments to the legislation.

The Expert Group notes that resources available through national platforms and access services – such as Feide – are already perceived in the sector as being controlled according to statutory requirements. This is incorrect. It is the school owners who are responsible for ensuring that the learning resources meet the data protection requirements, and there is a widespread misconception that Feide relieves school owners of this responsibility. We are concerned that the consequence of the misconception that national access services involve an approval based on statutory requirements will be amplified if a service catalogue devoid of data protection requirements becomes available to schools.

13.6 The Expert Group’s assessments

The Expert Group believes there is a considerable need to develop a code of conduct for data protection in schools. Currently, pupils’ privacy is not adequately safeguarded, and this challenges trust in the school as a public institution. The processing of personal data in primary and secondary education and training occurs on a large scale, including without learning analytics. It is not the analysis of pupil data that, in isolation, necessitates a code of conduct to safeguard data protection in schools, but learning analytics reinforces this need.

The Expert Group does not consider it appropriate to develop a code of conduct that is approved by the Norwegian Data Protection Authority according to criteria for a code of conduct. One of the reasons for this is that the Norwegian Data Protection Authority, in its comments to the Expert Group, has noted that there is a requirement to establish a monitoring body to ensure compliance with the provision on codes of conduct in the GDPR. However, we emphasise that national authorities should periodically assess whether the GDPR’s arrangement for codes of conduct would be suitable for the school sector.

The Expert Group believes that the best solution for better management and safeguarding of data protection in schools is to develop a School Code of Conduct according to a model similar to the Health and Care Sector Code of Conduct[40] . We recognise that the school sector differs from the health and care sector, in part because the State has greater controller functions in the health sector. The Norwegian Directorate for Education and Training (2023) has highlighted this in its consultation comments to the Norwegian Privacy Commission’s report. Nevertheless, we believe that the experience from the work on the Health and Care Sector Code of Conduct will be relevant for the school sector.

National authorities must take responsibility for ensuring that the School Code of Conduct is developed in close cooperation with the sector. At the same time, the Expert Group finds that the code of conduct should be binding for the relevant actors. Centralised support and guidance in the code of conduct must be designed in such a manner that the responsibility for processing continues to clearly rest with the school owners. In our assessment, specific requirements for data protection in resources with functionality for learning analytics, national data protection impact assessments, comprehensive guidance material with explanations and examples, and facilitating competence development in the sector will be necessary components of the School Code of Conduct.

13.7 The Expert Group’s recommendations

The Expert Group recommends that, in cooperation with the sector, a code of conduct be drawn up to safeguard data protection in schools. The School Code of Conduct should include the following minimum requirements:

development and administration of specific data protection requirements in resources that have functionality for learning analytics

preparation and administration of guidance materials for school owners, school administrators, teachers, pupils, parents, developers and suppliers

development and administration of national data protection impact assessments for resources that have functionality for learning analytics

facilitation of competence development on and exchange of experiences from data protection work in schools

The Expert Group recommends that, as part of the School Code of Conduct, specific, verifiable data protection requirements be drawn up for resources that have functionality for learning analytics. The requirements of the School Code of Conduct must be identical for both licensed and open resources. At a minimum, the requirements should be aimed at reducing the risks associated with the following four data protection principles:

fairness

transparency

data minimisation

accuracy

The Expert Group recommends that a national actor, as part of the School Code of Conduct, prepare and administer overall risk analyses, data protection impact assessments (DPIAs) and data processor agreements for resources that have functionality for learning analytics. The Expert Group emphasises that the responsibility for processing rests with the school owners. As the privacy situation in schools is precarious, we recommend as a first step that arrangements be made for school owners to share their analyses and assessments with each other.

The Expert Group recommends that as part of the School Code of Conduct, provisions be made for developing competence on and exchanging experiences related to the work on data protection. It would be beneficial if an already existing relevant network were assigned this role.

The Expert Group recommends that the administration model for the School Code of Conduct include a steering group with representatives from key actors and user groups.

The Expert Group recommends that the School Code of Conduct be based on relevant measures and guidelines that are already firmly anchored in the school sector, but it emphasises that the code of conduct must take a comprehensive approach to data protection in schools.

The Expert Group recommends linking the School Code of Conduct with a national service catalogue for digital learning resources. Such a link must be in accordance with the procurement legislation.

The Expert Group recommends that further work on the School Code of Conduct:

be developed with a realistic level of ambition and include thorough investigations and evaluations along the way

be aligned with existing learning technology standardisation efforts and privacy by design

includes all processing of personal data in schools, including processing that does not have learning analytics as a purpose

involves pupils and parents, where relevant

14 Frameworks for good learning analytics in primary and secondary education and training

Good learning analytics in schools is partly about the extent to which it supports the intentions of the National Curriculum. Therefore, it is necessary to facilitate a teaching practice characterised by variation, also with regard to the use of digital resources. There is also a need to require suppliers of digital learning resources to provide sufficient information about the pedagogical principles on which the resources are based.

The Expert Group believes that schools must be given a greater degree of free choice and a better overview of the digital learning resources with functionality for learning analytics available to them, and that greater provision should be made for schools to pay for use rather than access. They must also be better supported in assessing the quality of resources with functionality for learning analytics, and competence development related to pedagogical practice must be facilitated.

14.1 The Expert Group’s proposals for measures to increase free choice and equal access to learning analytics

In order for the information from learning analytics to be perceived as relevant and accurate for pupils and teachers, the selection of resources must reflect the variation in subjects, methods and pupils’ prerequisites. At the same time, access to a wide range of resources does not imply free choice per se. Teachers increasingly have access to high-quality digital teaching aids, but find it challenging to gain an overview of the selection thereof and identify the teaching aids that are suitable for the current learning situation (Norwegian Directorate for Education and Training, 2022a). A prerequisite for teachers to be able to adapt learning analytics to the distinctive nature of the subject, professional judgement and local conditions is that schools have a good overview of and information about the quality of the resources. Furthermore, access to learning analytics must be equal irrespective of which school the pupils attend. Teachers also need to be able to choose resources based on their suitability and not on whether the resources are part of larger licensed package solutions.

The Expert Group’s clear view is that teachers, school administrators and school owners are requesting a better and more quality-assured overview of what resources are available, what features they have, and to what extent they meet various pedagogical, legal and technical requirements. The Expert Group believes that a national service catalogue for digital learning resources is a good solution for offering such an overview in a number of areas and for stimulating equalising pricing models. We also believe that the grant schemes for the purchase and development of digital teaching aids are important drivers of free choice that should be further developed, and that financial measures aimed at testing and developing resources with learning analytics functionality should be established.

14.1.1 National service catalogue

A national service catalogue for digital learning resources can offer teachers, school administrators and school owners a better overview of the selection of digital resources suitable for use in schools, and increase free choice for individual teachers. Such a catalogue may include an overview and description of digital learning resources with functionality for learning analytics, information about available licenses, statistics and analysis of use, as well as assessments in relation to requirements for data protection, universal design and language variant.

The digitalisation strategy for kindergartens and schools for 2023–2030 states that the Norwegian Government will, in cooperation with KS, establish a publicly administered national service catalogue for digital teaching aids and consider including other digital solutions (Norwegian Ministry of Education and Research, 2023). We believe that the plan to establish a national service catalogue for digital teaching aids is a good starting point for supporting good and justifiable learning analytics in schools.

14.1.2 Pricing models

We have received a lot of comments from teachers, school administrators and school owners about the need for a more flexible pricing model than the current license-based models. Several municipalities have expressly stated a desire to pay for use to a greater extent in order to combine multiple digital teaching aids in the education (Høiseth-Gilje et al., 2022). The Expert Group believes that work should be initiated to investigate how it can become more attractive for suppliers to offer more flexible pricing models. It may also be relevant to investigate whether such pricing models can be linked to the national service catalogue.

Most digital teaching aids are license-based, with an annual price per pupil. This means that there is a relatively high threshold for utilising resources, especially if there are individual teachers who wish to familiarise themselves with the resource, or a few pupils who need a resource for a shorter period of time (KS, 2021). In many cases, it may be preferable to use parts of multiple resources instead of having full access to a few teaching aids for all pupils in the municipality or at the school. A license-based pricing model also entails greater supplier power than is the case for printed textbooks, since access to the teaching aid ceases if the agreement is terminated or expires.

Suppliers of large, comprehensive selections of digital teaching aids are favoured based on the current market structure (Oslo Economics, 2022). Key reasons for this are the procurement legislation and pricing models, as well as high switching costs if you change suppliers. In practice, this means that the large publishers dominate the market. The consequences of such dominance are that the threshold is high for smaller actors, and that established actors have lower incentives to invest in innovation and development. This is particularly relevant for learning analytics, as such functionality can be both costly and competence-intensive to develop, as it is often based on artificial intelligence or other complex technologies.

A further challenge with a licence-based pricing model is that the municipalities find it difficult to combine with grant schemes for digital teaching aids (Oslo Economics, 2022). The reason for this is that the grants are awarded for one year at a time, and several municipalities are sceptical about entering into agreements where they cannot maintain the licence without grant funds due to high switching costs.

In the project Activity Data for Assessment and Adaptivity, a pricing model is tested whereby the participating schools only pay for usage, not access (KS, 2021). An evaluation of the lessons learned from this project will be relevant in a continued investigation of a usage-based pricing model. Such a model requires, among other things, clarifications on how use is to be measured, what impact it may have on the market, and how it can be developed in line with the procurement legislation. Experience from the Activity Data for Assessment and Adaptivity project indicates that there is great potential in using a national service catalogue to enable more flexible access to digital teaching aids and digital solutions using new payment models (Norwegian Ministry of Education and Research, 2023).

The Expert Group believes that a usage-based pricing model will to a greater extent ensure free choice for schools, teachers and pupils when using resources with functionality for learning analytics, and that national authorities should facilitate this.

14.1.3 Grant schemes

The Norwegian learning technology sector is nascent, and we believe it is a state responsibility to contribute to the establishment of a market with fertile ground for both large and small suppliers that ensure innovation and diversity, and that are developed with Norwegian schools in mind. This is particularly important in order to prevent a school-oriented sector from developing based on purely commercial mechanisms. A lack of financial discretion among those who purchase teaching aids increases the use of advertising-financed resources where the payment is pupils’ personal data (ICT-Norway, 2023).

The grant scheme for the purchase of digital teaching aids is a measure under the initiative The Technological Backpack[41] . The Norwegian Directorate for Education and Training has administered the scheme the aim of which has been to give pupils and teachers better access to a multitude of high-quality digital teaching aids (Norwegian Directorate for Education and Training, 2022b). This procurement grant scheme was implemented over the course of four years (2019–2022). Over the four years, NOK 289 million was allocated, and when factoring in the municipalities’ own contributions, teaching aids were purchased in the amount of NOK 571 million during this period. More than 300 municipalities have been allocated funding. In other words, the scheme has resulted in a substantial injection of funds into the market.

In 2018, another grant scheme was launched to develop digital teaching aids for the introduction of a new National Curriculum. The framework for the grant scheme was NOK 23.75 million. The objective of the grant scheme was for the pupils to have access to a multitude of innovative, high-quality teaching aids that support differentiated instruction, and a broader range of digital teaching aids in Nynorsk and Sámi.

In the final report for The Technological Backpack and the Action Plan for Digitalisation, the Norwegian Directorate for Education and Training concludes that there continues to be a need for a government funding scheme for the purchase of teaching aids and that these should be adapted to a hybrid school model involving both digital and printed teaching aids. The Norwegian Directorate for Education and Training also finds that the procurement support scheme provided better access to digital teaching aids in addition to contributing to stimulating the market and the supply of teaching aids.

The Expert Group believes that both the grant schemes for developing and purchasing digital teaching aids should be maintained and further developed. This especially applies where there is no basis for commercial development, and to safeguard pupils who receive instruction in Sámi or Nynorsk. The scheme for development should stimulate innovative use of artificial intelligence (AI) and learning analytics functionality, but also set requirements for data protection and responsible use of AI in line with the recommendations in Chapter 13.

14.2 The Expert Group’s proposal for support for assessing pedagogical quality in learning analytics

For learning analytics to be useful, schools and teachers must have a basis for assessing the quality of the actual analysis and the data included therein. The Expert Group finds that suppliers need clearer information about what the learning analytics is based on, and that better provision should be made to provide specific guidance to schools in assessing the quality of resources with functionality for learning analytics. The need for information applies not only to resources with such functionality, but also to the resources that could conceivably provide data for learning analytics. Assessing the quality of learning analytics is always about having a conscious relationship with the data on which the analysis is based.

Quality is an imprecise and context-dependent term. In this context, we are seeking to identify aspects of the learning resources that concern how suitable they are for different purposes in the instruction. This involves pedagogical and didactic aspects, but also technical aspects such as data quality and user-friendliness. In terms of academic quality, this depends on whether the resource is a teaching aid with pre-defined academic content or a resource such as a communication platform or a pure analysis tool. For learning analytics concerning pupils’ academic progress, the connection to the National Curriculum is always relevant. For learning analytics built into resources without pre-defined academic content, the relationship to the National Curriculum may be more indirect, e.g., if the functionalities support the principles and values.

The school sector is an attractive market for suppliers of learning technology, and the marketing of such products often emphasises a major impact on pupils’ learning outcomes: “Technology actors have far too often set the agenda, often with discourses characterised by simple solutions and quick fixes of complex pedagogical issues” (Erstad, 2022, p. 318). By facilitating support for schools to assess quality, they will have a better basis for critically assessing resources.

14.2.1 Criteria for pedagogical quality in learning analytics

Assessments of pedagogical quality and suitability of resources with learning analytics functionality should be performed in as close proximity as possible to those who will be using the resources. Relevant criteria for quality related to learning analytics should nevertheless be developed centrally to provide good support in this assessment process.

The Norwegian Directorate for Education and Training has developed guidelines to support teachers, school administrators and school owners in assessing and selecting teaching aids.[42] In its digitalisation strategy for schools, the Norwegian Government specifies that it wishes to further develop and disseminate this service (Norwegian Ministry of Education and Research, 2023).

The framework in the guides contains statements expressing favourable qualities of teaching aids within the categories relation to the National Curriculum,pedagogical and didactic quality and prototyping and design. They also list some general quality characteristics of digital teaching aids (Norwegian Directorate for Education and Training, 2021b, Chapter 3.4):

utilises the advantages that digital platforms can provide

has a large repertoire of learning content and working methods

makes it easier for the pupil to use additional senses

safeguards the pupil’s privacy (if the teaching aid generates pupil data, it accounts for what data are collected, for what purpose, and who has access to the data)

has technological solutions that are based on a learning perspective that is in line with the values of the curriculum

The knowledge base for the guides emphasises that learning analytics reinforces the need to assess whether the teaching aids are based on a pupil and learning perspective that is in line with the intentions and values of the National Curriculum (Norwegian Directorate for Education and Training, 2021b).

The Expert Group believes that the content of this material constitutes a good framework for quality assessment, and that explicit criteria for quality in learning analytics should be included. However, we recognise that this measure is non-binding and assumes that the schools choose to avail themselves of it. In order to increase user-friendliness, support and accessibility, a good measure may be to incorporate the guides into a national service catalogue.

We also believe the quality criteria can be used by developers and suppliers to ensure that the resources offered to schools are in line with expectations for quality.

14.2.2 Sufficient information to assess pedagogical quality

Learning analytics is often based on processes and metrics that can be difficult for users to understand. This means that it is difficult for schools and teachers to decide whether the resource supports the values and objectives of the National Curriculum. Many digital teaching aids rely on simple behaviourist principles and individualisation of learning, without this necessarily being clearly stated (Erstad, 2022). In order for users to gain insight into the pedagogical standpoint embedded in the resource, the providers of learning materials must make available sufficient information about how content and functionality support teaching and learning. The main purpose of the information shall be to provide users with good prerequisites for assessing the pedagogical quality and suitability of the resource.

Such information aimed at users can be based on the principles behind explainable artificial intelligence[43] , which is about creating the conditions for understanding the algorithms on which artificial intelligence is based. This does not mean publishing or providing full insight into the code behind the algorithms or data sets, but rather shedding light on which data have had an impact on the analysis, and how important the various elements have been (Norwegian Ministry of Local Government and Modernisation, 2020). Suppliers must also be able to document that the technical specifications in the solutions correspond to the user-oriented information. These specifications will include more detailed information on data types and analytical methods.

There are many actors in the Norwegian market today who refer to their products as “analysis of learning”, “insight into the pupil’s learning”, “overview of what pupils have understood”, “overview of academic progress” and similar wording, without offering sufficient evidence for such claims (Egelandsdal et al., 2019). To remedy this, the Expert Group believes that the suppliers must justify the solutions they have chosen and explain how these solutions actually work.

The Expert Group believes that suppliers must be required to make such information available in order for resources with learning analytics functionality to be suitable for use in schools. Such information may also be required to be included in a national service catalogue. We stress that user-oriented information from suppliers must be adapted to the various target groups. For example, we believe it is unfortunate if the term learning analytics, which is poorly and diversely understood in the school sector, is used aggressively by suppliers of learning technology in their marketing.

14.3 The Expert Group’s proposal on competence development in good learning analytics

In order to use digital resources in ways that contribute to increased adaptation, documentation or variation in teaching, the teacher must become familiar with the academic and technical aspects of the digital resources. This requires a high level of didactic competence in addition to good digital competence and the ability to critically assess each tool.

In the first interim report, we describe how teachers are dependent on being able to critically assess all academic and pedagogical factors, and that they must have sufficient analytical competence to interpret pupil data and analysis representations. They must be able to make assessments about ethics and have a practical understanding of data protection and have the competence to support the pupils in interpreting analyses of their own learning. Comments we have received emphasise that it is important that teachers are enabled to assess the functionality and supporting data in the individual teaching aid, so that they can assess what a teaching aid tells us about the pupils’ academic level (Union of Education Norway, 2022).

14.3.1 Areas of competence for teachers

Competence in learning analytics is based the teacher’s professional digital competence[44] and their academic and didactic competence. The new competence requirements that learning analytics entails can mainly be linked to the competence areas of critical appraisal, ethics and data protection, and analytical competence.

Critical appraisal

Teachers must be able to critically appraise the academic and pedagogical guidelines embedded in learning analytics. This first and foremost presupposes that information about these guidelines is available in the resource, but it also requires some technological competence and understanding of, among other things, how algorithms work and what kind of data are included in the analyses. It is necessary for teachers to be able to ask critical questions about how learning analytics supports the breadth of subjects and the variety of working methods for the pupils. This competence is also key to being able to guide the pupils in utilising feedback from learning analytics regarding them.

Ethics and data protection

Educators must have competence in ethics and practical data protection to determine the appropriate course of action based on the analysed data. The ethical competence related to learning analytics is based on the teachers’ professional ethics and integrity. They must also be familiar with the legislation governing learning analytics. The Expert Group believes it is particularly relevant for teachers to have a good practical understanding of the data protection principles, so that they have applicable knowledge to safeguard the pupils’ privacy at school. An example of an ethical and data protection challenge is that the distinction between school and home becomes blurred in digital learning environments. This must be addressed by teachers with a high degree of ethical awareness.

Analytical competence

Learning analytics provides teachers with different types of performance representations. These representations are often presented in the form of a visualisation, recommendation or report. It is important that teachers have relevant competence in order to understand the underpinning for the representation, and to be able to determine the significance of what the representation shows. This is part of what many refer to as analytical competence[45] , which in short is the ability to explore, understand and use data in meaningful ways. Such competence is key to assessing and translating analytical representation into pedagogical practice, and is highlighted as a core competency for teachers (Sampson et al., 2022).

14.3.2 Schemes for competence development among teachers

There are various schemes for competence development that can help enable teachers to include learning analytics in their practice in a suitable manner. We have chosen to describe three options for competence development, which are neither intended exhaustively nor are they mutually exclusive.

Basic education

Section 2 of the Regulations relating to the Framework Plan for primary and lower secondary teacher training on learning outcomes states that the candidate shall have professional digital competence.[46] Similarly, section 2 of the Regulations relating to graduate teacher training states that the candidate shall be able to use digital tools in teaching, planning and communication.[47] These provisions can be interpreted as including competence in learning analytics. However, there is a need to clarify this in the competence descriptions.

Supplementary and continuing education programmes

A wide range of supplementary and continuing education programmes are offered to teachers. The continuing education programmes are credit-conferring programmes, while the supplementary education programmes are competence-raising schemes that do not confer credits. Learning analytics is a relevant area of competence for such programmes, e.g., in connection with professional digital competence.

Competence packages

Competence packages are brief, independent courses and structured learning resources that teachers, school administrators and other target groups can use for self-development. The Norwegian Directorate for Education and Training has developed competence packages for teachers in various topics.[48] This offer already includes a competence package for artificial intelligence in schools, but this package does not currently specifically address learning analytics.

14.4 The Expert Group’s assessments

The Expert Group believes there is a considerable need to develop clear frameworks for good learning analytics in primary and secondary education and training. Currently, large volumes of pupil data are collected and analysed without a clear purpose, and such frameworks can contribute to clarifying what kinds of data are needed and how to use the analyses to enhance learning. We stress that the purpose of the frameworks for good learning analytics is to increase and support the teacher’s scope of action, not to restrict it.

The Expert Group considers the planned national service catalogue for digital teaching aids to be a suitable instrument for establishing frameworks for good learning analytics. A national service catalogue provides a structure that furnishes schools with a necessary inventory of resources with learning analytics functionality. In addition, the Expert Group believes that the structure of the service catalogue can be utilised to establish a support system for assessing quality in learning analytics, and to facilitate a usage-based pricing model.

The Expert Group notes that a support system for assessing quality is not intended to serve as a national approval scheme for teaching aids. Such a scheme existed in Norway until the year 2000, but it was repealed in part based on grounds of safeguarding teachers’ free choice (Norwegian Directorate for Education and Training, 2021b). However, today’s market for teaching aids is different compared to when this scheme was abolished, and the need for centralised support to assess quality and suitability has grown.

The Expert Group finds that there is a need to develop competence development programmes for teachers and school administrators on learning analytics. We emphasise that expectations as to what kind of competence teachers should develop must be reasonable in relation to the profession. For instance, technological competence at an advanced level and complex data protection assessments must be managed at other levels in the sector. The design of the digital resources must also build on the current competence situation and practice.

14.5 The Expert Group’s recommendations

The Expert Group recommends that national authorities facilitate usage-based pricing models for digital teaching aids, and that a study be initiated on how trials involving usage-based pricing models can be scaled up.

The Expert Group recommends that the national service catalogue for digital learning resources supports good learning analytics in schools.

The Expert Group recommends that centrally defined quality criteria be developed for resources that have functionality for learning analytics. It is teachers, school administrators, school owners and developers who will use these quality criteria. The criteria can build on existing guidelines for assessing the quality of teaching aids.

The Expert Group recommends that suppliers and developers cooperate on using and further developing the quality criteria so that they offer guidance for product development.

The Expert Group recommends that suppliers be required to make available user-oriented information that justifies and explains how the resources work. Suppliers must also be able to document that the technical specifications in the resources correspond to the user-oriented information.

The Expert Group recommends a grant scheme for purchasing and developing digital teaching aids that have functionality for learning analytics. The grant scheme should stimulate innovative learning analytics functionality and artificial intelligence (AI), and must set requirements for data protection and responsible use of AI. Resources must also be required to comply with centrally defined quality criteria.

The Expert Group recommends that funding be announced for innovation, research and development of digital learning resources that have functionality for learning analytics and adaptivity, and funding for research on the use of such resources in authentic learning situations.

The Expert Group recommends measures aimed at student teachers, teachers, school administrators and school owners, so that they can develop competence in learning analytics. Competence in learning analytics and knowledge of artificial intelligence should be included in both basic education and supplementary and continuing education programmes.

The Expert Group recommends that school owners ensure that pupils receive adapted and comprehensible information so that they can consider issues relating to learning analytics. Furthermore, it is recommended that school owners regularly evaluate whether pupils experience that the school is safeguarding their right to participation.

15 Guidelines for good and justifiable learning analytics in higher education and tertiary vocational education

Good and justifiable learning analytics in higher education and tertiary vocational education can contribute to promoting student learning. Learning analytics is a new and complex field in university colleges, universities and vocational colleges.

The Expert Group’s perception is that there is great uncertainty in higher education and tertiary vocational education regarding the processing of personal data in learning analytics. There is also uncertainty as to what the pedagogical benefits of learning analytics might be. Consequently, there is a need to support and guide educational institutions in how learning analytics can contribute to improving study programmes in a way that safeguards students’ privacy.

The Expert Group is of the opinion that broad guidelines for good and justifiable learning analytics in higher education and tertiary vocational education should be developed. Educational institutions can then develop local guidelines based on their national counterparts. We also believe there is a need to develop guidance materials and competence development measures to support good practice and demonstrate the opportunities educational institutions have to use new tools to support student learning.

15.1 The need for guidelines for good and justifiable learning analytics

Higher education and tertiary vocational education are diverse sectors with considerable institutional autonomy. Broad common guidelines and clarifications of principle can nevertheless be of great benefit with respect to learning analytics. Both Sikt (2022) and Universities Norway (2023) have expressed a need for common guidance resources.

Ambiguous legal basis hinders initiatives and knowledge development

As the Expert Group shows in sections 10.4 and 10.5, the legal basis for processing personal data in learning analytics in higher education and tertiary vocational education is ambiguous. In order to familiarise oneself with the opportunities and challenges associated with learning analytics, it is necessary to explore learning analytics on a larger and smaller scale and gather experiences within secure frameworks. Uncertainty regarding what constitutes lawful and justifiable processing of personal data is currently an obstacle to exploring its potential and constitutes a significant barrier to learning analytics, especially in higher education: “An ambiguous legal basis for collecting data and conducting learning analytics prevents activities in this area from getting off the ground” (Sikt, 2022, p. 2).

New learning technology should be tested in realistic pedagogical practice in order for educational institutions to gain experience of opportunities and limitations. This is also true for learning analytics. Here, teachers and educational institutions should trial the existing opportunities they believe are relevant, and their experiences should be documented.

As shown in the overview of research on learning analytics in Chapter 3, few studies have been carried out in the context of ordinary pedagogical practice in Norway, probably because the scope of learning analytics is so limited. The knowledge acquired about learning analytics instead takes place under the auspices of individual teachers or in small-scale and targeted research projects where the researchers themselves often also introduce the technology and structure their teaching around the possibilities for systematically using student data. Experiences from these smaller projects rarely go beyond the research environments, and it is often unclear how transferrable the experiences are across different courses or programmes.

Clear frameworks and proposals for risk-reducing measures, especially with regard to data protection, will better facilitate testing of learning analytics in ways that can make useful contributions to quality assurance work, in the development of teaching practice and in support of students’ learning processes.

Guidelines complement regulatory legislation

In sections 10.4.4 and 10.5.3, the Expert Group shows that there is a need to clarify the legal basis for processing personal data in learning analytics in higher education and tertiary vocational education.

Even with a clearer legal basis, there will still be a clear need for guidelines, including to ensure adequate compliance and data protection practices. If a specification in the legislation allows for a more privacy-invasive processing of personal data in learning analytics than is currently the case, this will necessitate even greater requirements for justifiable frameworks surrounding practical use.

15.2 Three conditions for the guidelines

The Expert Group will highlight three conditions in order for the broad guidelines to fulfil their purpose. The first concerns the establishment of a common path for the sectors, while at the same time safeguarding institutional autonomy. Second, rapid technological development requires frequent updating of the guidelines, and finally the guidelines should cover all the learning analytics resources in use in higher education and tertiary vocational education. In other words, the guidelines should be flexible (to safeguard autonomy), dynamic (to accommodate change) and specific (to cover all resources used).

Common approach with local adaptation

Although the responsibility for good and justifiable learning analytics lies at the institutional level, a number of issues will be common to all educational institutions. It would therefore be appropriate to have a common approach for higher education and tertiary vocational education in line with the needs of the institutions, and with a strong degree of cooperation with and involvement on their part. It must also be possible to adapt the guidelines to local conditions.

Regular updating and evaluation

As the technology that enables learning analytics is rapidly evolving, it is difficult to predict what functionality and resources will be available in just a few years. Moreover, the variations in the pedagogical practice involving learning analytics are limited, which may change with increased experience. The guidelines must therefore be subject to regular revision and further development so that they are always relevant to the typical practice in the sectors. How learning analytics is carried out in university colleges, universities and vocational colleges should also be subject to evaluation, per se.

Encompassing all resources

Resources with functionality for learning analytics become available to educational institutions through procurement processes and self-development, either as joint services or at the institutional level. The guidelines should apply to all the different resources that can be used for learning analytics. Not least, they should also apply to resources that are openly available. The use of the latter entails a significant privacy risk, as ‘payment’ is usually made in the form of users’ personal data (NOU 2022: 11).

15.3 Examples of national and local guidelines

The UK and the Netherlands have developed common national guidelines for learning analytics in higher education (Sclater and Bailey, 2018; SURF, 2019). In the UK, a number of universities have adopted the national guidelines as their point of departure and developed local adaptations, including at Technological University Dublin[49] and the University of Edinburgh[50] . The University of Aalto, Finland has also developed guidelines for learning analytics.[51]

At the University of Oslo, an inter-faculty working group has developed Learning analytics and quality in education at UiO: Proposal for a privacy policy, cf. Box 15.1. The Expert Group believes a good solution is to have working groups at the institutional level that can adapt the broad guidelines to local conditions. It is important that both employees and students are well represented in such working groups.

Textbox 15.1 Description of selected areas from the proposed privacy policy for learning analytics at the University of Oslo (Langford et al., 2022)

Data protection involves (1) personal data, aggregated and pseudonymised data, (2) students’ ethical rights, (3) processing in accordance with the right to privacy and (4) Article 5 of the GDPR on principles for the correct processing of personal data in line with the Regulation.

The legal basis in the GDPR and the principles of specific assessment concern (1) the basis for processing, where Article 6(1)(e) of the GDPR (“performance of a task carried out in the public interest or in the exercise of official authority”) is highlighted as the most relevant of the six principles, (2) the need to establish principles for determining whether Article 6(1)(e) constitutes a legal basis in specific contexts, (3) whether the legal obligations in the Universities and University Colleges Act are clear enough that Article 6(1)(c) (“compliance with a legal obligation to which the controller is subject”) may be used, (4) considerations when reusing data collected for another purpose, (5) requirements for privacy by design in systems with learning analytics functionality, and (6) requirements for data protection impact assessments.

Data subjects’ rights and participation concern (1) the right to information, (2) the right to rectification, (3) the right not to be subject to automated decision-making, including profiling, and (4) the right to object.

Student participation and teacher autonomy concern (1) the right of student bodies to be heard, (2) students’ right to information and reservations, (3) the implications of systematic quality assurance, including learning analytics, for instructors’ autonomy, and (4) how confidence in teachers should not be undermined by the results of the learning analytics.

Institutionalisation concerns (1) establishing a coordinated and comprehensive system for quality assurance and control of statutory requirements, ethical considerations and participation in learning analytics, (2) guidance to relevant actors on legal bases, (3) openness regarding what data are used, and (4) ensuring purposeful competence enhancement.

15.4 The Expert Group’s proposal for points in the guidelines

In this section, we will provide a brief, general description of five action points that we believe are particularly important to include in guidelines for learning analytics in higher education and tertiary vocational education:

data protection

participation

openness

free choice

procurements

15.4.1 Data protection

Learning analytics in higher education and tertiary vocational education will in most cases require the processing of personal data. In the context of quality assurance work, where data are often aggregated and processed in pseudonymised form, the data protection consequences are less significant. The right to privacy is regulated by the GDPR. However, procedures and measures must be established to ensure that this right is observed. We have identified the following four data protection principles that have proven to be particularly challenged in learning analytics and the use of artificial intelligence:

fairness

transparency

data minimisation

accuracy

In particular, the guidelines should set requirements that reduce the risk of breaches of these four principles and that are suitable for safeguarding the rights and freedoms of students.

Fairness

One of the prerequisites for ensuring fairness for students is that they are familiar with their rights to access, to rectify incorrect data and to erase data.

Transparency

The educational institution must facilitate openness and transparency regarding the processing of personal data. Such transparency implies

that there are comprehensible and adequate descriptions of what the individual resources actually do

that outlines of data flows and processing protocols are available[52]

that an explanation is given of how the algorithms in learning analytics weight different variables, how accurately the algorithm processes data and how reliable the result is

that it makes visible what and whence the information is collected, and how it is interpreted in the analysis

that an overview is provided of who has access to the collected data and can view the results of the analysis, and for which decisions the results are used.

Data minimisation

This principle concerns limiting the data that are collected and processed to what is necessary for the purpose of the learning analytics. A current challenge is that myriad data are collected about students’ digital activities the pedagogical value of which is unclear, e.g., regarding the time of day they log on to learning platforms and administrative systems. Technically, data minimisation can be ensured by, among other things, procedures for extracting data, filtering, various ways of anonymising data once the analysis has been performed, and barriers to linking to the collected data for other purposes.

Accuracy

To ensure accuracy in analyses, data sources may be required to undergo quality assurance and validation of relevance and validity prior to use in learning analytics. This increases the chance that the data included in the analyses are accurate. Without quality assurance and validation, there is a risk of bias in the analyses, which can be amplified if resources are used uncritically or fed flawed training data. Therefore, it will be relevant to require built-in regular testing for biases in the data material, the models or in the use of the algorithms. In addition, there may be a requirement to re-train the algorithms if the accuracy falls below a predetermined threshold.

15.4.2 Participation

Pursuant to Section 4-1 of the Universities and University Colleges Act, student bodies have the right to be heard in all matters concerning the students. This includes the use of students’ personal data in learning analytics. Regarding learning analytics, it is crucial that students can always trust that all analyses are conducted securely and responsibly, and that data are never used in other ways than as described in the purpose and legal basis.

Student participation is also important in order for students to be active participants in their own learning. Participation in learning analytics presupposes that students gain as thorough insight as possible into what data and analytical methods are used, and how they are used, so that they can benefit from the insight the analyses provide into their own learning and academic progress.

In their comments to the Expert Group, both the student unions and Universities Norway are specifically concerned with students’ involvement in learning analytics. The National Union of Students in Higher Vocational Education and Training in Norway (2022) is “very concerned with using the tool on the students’ terms, and actively facilitating good student participation in the processes associated with its use’ (p. 3). Universities Norway (2023) notes that “students must also be included when assessing the types of learning analytics that are needed and when, and therefore what types of data should be collected (p. 1).

The Expert Group notes that the guidelines for learning analytics must ensure that educational institutions can ensure students’ participation and information needs.

15.4.3 Openness

In order for students to be confident that the processing of personal data takes place in accordance with the stated purposes and according to the legal basis, they must have access to how it takes place. All this information should be public and easily accessible to students. The guidelines should therefore require educational institutions to state which data are collected from which sources, how they may be combined with other data, and what the data are specifically used for. It must be clear to students to what extent individual students can be identified based on the collected data, and who has access to these data. Students must be ensured access to all collected data about themselves if personal data and identifiable data are stored at the individual level. This follows from Chapter 3 of the GDPR on the rights of the data subject.

It should also be clear to students when collection takes place, and when they can use digital resources without being tracked at the individual level. There should be clear rules for when data about students can be collected. In conversations with the Expert Group, the National Union of Students in Norway and the Organisation of Norwegian Vocational Students have voiced students’ need to be able to distinguish between the student role and the role of private individual. This role is challenged by platformisation in higher education and tertiary vocational education as students and instructors have continuous access to each other and subject matter via the learning platforms. The student unions want both clear limitations on when data about them are collected and limitations on when they can receive notifications from the educational institutions’ systems. This can be done, e.g., by giving students the opportunity to regulate the times themselves or by placing restrictions on the system.

The Expert Group believes that it must be made clear in the guidelines what the students’ rights are, how they proceed to safeguard them, and how confidence can be instilled that the use of learning analytics takes place in accordance with the stated purpose and regulatory legislation.

15.4.4 Free choice

Educators have the freedom and responsibility to prepare the content and structure of instruction within the frameworks established by the institution. A key part of the role of an instructor is to assess which working and teaching methods are best suited for different courses. However, the decision as to which resources with learning analytics functionality should be available to all instructors at, for example, a vocational college, a department or a university, is subject to the institutional frameworks. The Expert Group believes that the guidelines drawn up for learning analytics must clarify how the trade-off between the two considerations is to be assessed at the educational institution. It is important to ensure that instructors have access to different resources, but also to safeguard their freedom and responsibility to independently prepare the content of the instruction.

It is also relevant to include in the guidelines what the scope of students’ free choice should be in terms of learning analytics. In this context, it is important to distinguish between different forms of learning analytics, as there is a difference between learning analytics with aggregated and pseudonymised data as a basis for quality assurance work on the one hand and individual follow-up of the learning process in the individual student on the other. The Expert Group believes that students’ free choice must be more comprehensively linked to the individual-oriented form of learning analytics, in that it may, e.g., be possible to opt out of the analysis of certain types of personal data. The degree of students’ free choice should also be linked to whether information about them is actually anonymised. This will contribute to confidence in learning analytics for students, without affecting the institution’s long-term quality assurance work.

15.4.5 Procurements

In input meetings with the Expert Group, representatives of the sectors have confirmed that the possibilities for learning analytics have not been specifically considered when purchasing resources that allow for such analyses. The foremost example of this is the learning platform Canvas, but it also applies to video platforms and other services. The learning analytics guidelines should support the sectors in developing learning analytics requirements for tender processes, where relevant. The requirements must be based on local academic discussions at the educational institutions regarding what kind of needs the instructors and educational institutions have, what types of analyses they want, and how learning analytics is intended to support learning processes and quality assurance work. There should also be requirements for privacy by design and information security in procurement processes.

15.5 The Expert Group’s proposal on the administration of the guidelines

Although the guidelines for good and justifiable learning analytics should have a common national path, they must correspond to local needs and the local academic profile. The Expert Group believes that the best solution is for the institutions to develop local guidelines based on their national counterparts. As technology changes rapidly, the national guidelines must be revised continuously, e.g., every five years. New statutory requirements may also necessitate a revision of the guidelines.

Relevant national actors to manage the development and administrative responsibilities are the Norwegian Agency for Shared Services in Education and Research (Sikt), the Norwegian Directorate for Higher Education and Skills, the interest group Universities Norway and the National Council for Tertiary Vocational Education:

The Norwegian Directorate for Higher Education and Skills has the overall national responsibility for administrative tasks in higher education and tertiary vocational education. The Norwegian Directorate for Higher Education and Skills shall have a strong legal professional environment in accordance with its assignment from the Norwegian Ministry of Education and Research and has the professional responsibility for information security and data protection. The Directorate is also responsible for implementing and following up the strategy for digital transformation in the university and university college sector, a strategy that encompasses many aspects relevant to learning analytics.

Sikt offers a range of services to the Norwegian knowledge sector with functionality and potential for learning analytics. Sikt is a resource environment in the areas of procurement, operations, data analysis and development of learning technology.

Universities Norway is a member organisation for Norwegian universities and university colleges. In its comments to the Expert Group, Universities Norway (2023) states that “if common guidelines or guides are to be prepared, the sector must be involved, e.g., via Universities Norway’s units” (p. 2). The strategic units[53] are national coordination arenas for its member institutions.

The National Council for Tertiary Vocational Education is an advisory body appointed by the Norwegian Ministry of Education and Research. The Council is tasked with working on the further development of the tertiary vocational education sector and promoting cooperation between the sector and the working life. The Norwegian Directorate for Higher Education and Skills is the secretariat for the National Council for Tertiary Vocational Education.

15.6 The Expert Group’s proposal on competence development and guidance services

Learning Analytics uses different analysis and calculation methods to provide insight into student learning. In order to determine possibilities and limitations in specific contexts, there is a need for knowledge regarding how different methods and algorithms process data, and to be able to judge the results of such processing.

15.6.1 Competence development

In order to perform good and justifiable learning analytics, instructors must have sufficient insight into and understand the academic, pedagogical, ethical and technical aspects of the digital resources that facilitate learning analytics. This is part of the competence many refer to as analytical competence, which in short is the ability to explore, understand and use data in meaningful ways. This competence is highlighted as a core competency for instructors (Sampson et al., 2022). In the Expert Group’s first interim report, we describe how instructors must have sufficient analytical competence to interpret student data and analysis representations. They must be able to make assessments about ethics and practical data protection and have the competence to support students when interpreting analyses of their own learning. Developing students’ competence in learning analytics is part of the relevant subjects and courses at the educational institutions.

In reference to section 2-3 of the Academic Supervision Regulations, we find that pedagogical competence includes competence in utilising digital technology to enhance learning (NOKUT, 2020). Our assessment is that this also includes being able to actively and critically utilise the potential of learning analytics, including the necessary competence on data protection and ethical use of personal data.

Section 1-4, third paragraph of the Regulations concerning appointment and promotion to teaching and research posts[54] refers to pedagogical competence requirements for a permanent position as førsteamanuensis (associate professor): “Completion of a separate programme (minimum 200 hours)/relevant courses and individual practical teaching, and acquired basic skills in planning, implementing, evaluating and developing teaching and supervision (basic competence for teaching and supervision at the university and university college level)”.

As a result of this competence requirement, institutions in higher education offer 200-hour courses in basic pedagogical competence. The Expert Group believes that competence in learning analytics should be systematically included in such training programmes and emphasise the possibility of better follow-up of students by means of learning analytics. We also believe that Universities Norway can contribute to coordinating this work nationally, e.g., through its guidelines for basic pedagogical competence in universities and university colleges[55] .

The Expert Group believes that it is necessary to investigate more closely how competence development in learning analytics can be ensured for instructors in tertiary vocational education.

In addition to incorporating learning analytics into general pedagogical competence, learning analytics should be included in various courses offered under the auspices of the educational institutions’ learning support units. The courses offered should be aimed at both instructors and different types of education administrators and support staff who assist instructors and who participate in quality assurance work.

15.6.2 Guidance Services

The Expert Group wishes to highlight relevant areas of learning analytics where there may be a need for various guidance services for the sectors:

good and user-friendly overviews of different types of supporting data available for learning analytics at the country’s educational institutions in different types of shared digital services, and of the types of analysis the services can perform. This should also include guidance on how educational institutions can legally share data with each other for quality assurance work.

guidance in drawing up guidelines at one’s own institution. An example of a similar guide can be found in Ireland.[56]

support system to assist educational institutions in preparing data protection impact assessments (DPIAs). This includes guidance for educational institutions, templates for data processor agreements and risk analyses, examples and information material.

Relevant actors for developing and administering such guidance services may be the Norwegian Directorate for Higher Education and Skills, Sikt, Universities Norway and the Norwegian Tertiary Vocational Education Council.

15.7 The Expert Group’s assessments

The Expert Group believes there is a considerable need to prepare broad guidelines for good and justifiable learning analytics in higher education and tertiary vocational education. Currently, learning analytics is mainly carried out for administrative purposes because uncertainties in the sectors regarding what is legal and justifiable stands in the way of learning analytics for more pedagogical purposes.

The Expert Group believes that it is crucial that questions regarding learning analytics are addressed on the basis of a comprehensive assessment of pedagogical, ethical, technological and legal considerations. Currently, all use of digital resources is often centrally regulated by institutions solely on legal grounds, and there are few guidelines for good pedagogical use. It is of course a basic premise that all processing of personal data should take place in accordance with the legislation, however, we believe that guidelines should also be drawn up that elaborate on what constitutes good learning analytics in pedagogical practice, so that technology can benefit students to a greater extent than is currently the case.

The Expert Group emphasises that the national, broad guidelines must be drawn up in close cooperation with the sectors, and that it is the responsibility of the institutions to make adaptations and develop local guidelines based on their national counterparts.

The Expert Group finds that there is a need to develop competence development programmes for instructors on learning analytics. It is natural to view this in the context of the training offered on basic pedagogical competence. The teacher and graduate teacher programmes have a special responsibility to ensure that their instructors have competence in learning analytics, since it is important that the students in these programmes acquire this competence through their education.

The Expert Group considers that there is a need for centralised guidance services that can support educational institutions in implementing risk analyses, data protection impact assessments (DPIAs) and data processor agreements in connection with procurement processes and system development projects.

15.8 The Expert Group’s recommendations

The Expert Group recommends that, in cooperation with the sectors, broad national guidelines for good and justifiable learning analytics be developed. It must be possible to adapt the national guidelines to local conditions. At a minimum, the guidelines should include the following action points:

data protection

participation

openness

free choice

procurements

The Expert Group recommends that a government agency develop and administer the broad guidelines for good and justifiable learning analytics in close cooperation with sectoral actors such as Universities Norway and the National Council for Tertiary Vocational Education. The Expert Group emphasises that the responsibility for good and justifiable learning analytics lies with the institutions.

The Expert Group recommends that the broad guidelines be revised regularly in light of rapid technological developments and at least every five years.

The Expert Group recommends that the guidelines include common solutions, local resources and resources that are openly available online.

The Expert Group recommends that a government agency develop a support system to help educational institutions prepare risk analyses, data protection impact assessments (DPIAs) and data processor agreements. The government agency shall also assist educational institutions in connection with procurement processes and system development projects.

The Expert Group recommends that the guidelines account for what constitutes good learning analytics that promote student learning.

The Expert Group recommends that competence in learning analytics be included in training programmes for basic pedagogical competence in higher education and tertiary vocational education. In addition, the Expert Group recommends that learning analytics be included in various courses aimed at instructors, managers and support staff who assist instructors and who participate in quality assurance work.

The Expert Group recommends that teacher training ensures that newly qualified teachers possess the necessary competence in learning analytics and knowledge of artificial intelligence. The institutions must consider how they can ensure such competence in teaching and in learning outcome descriptions.

The Expert Group recommends that funding be announced for innovation, research and development of digital learning resources that have functionality for learning analytics and adaptivity, and funding for research on the use of such resources in authentic learning situations.

The Expert Group recommends that the institutions ensure that students receive adapted and comprehensible information so that they can consider issues relating to learning analytics. Furthermore, it is recommended that the institutions regularly evaluate whether students feel that the school is safeguarding their right to participation.

16 Financial and administrative consequences

The Expert Group recommends several measures that will have varying degrees of financial and administrative consequences. The recommendations pertain to the following action areas:

legal basis for learning analytics

code of conduct for data protection in primary and secondary education and training

frameworks for good learning analytics in primary and secondary education and training

guidelines for good and justifiable learning analytics in higher education and tertiary vocational education

The financial and administrative consequences of the Expert Group’s recommendations will depend on the design and scope of the measures that are decided to be implemented. Some measures entail changing administrative processes without significant additional financial costs, while other measures are assumed to be implemented through reprioritisation within current budgets.

Legal basis for learning analytics

The Expert Group recommends clarifying the legal basis for learning analytics in primary and secondary education and training, higher education and tertiary vocational education. The proposed amendments to the Education Act, the Universities and University Colleges Act and the Vocational Education Act, with regulations, are essentially a clarification of the current legislation and will not have significant financial or administrative consequences. The Expert Group believes that the Norwegian Ministry of Education and Research should be able to cover these costs under its current financial frameworks.

Code of conduct for data protection in primary and secondary education and training

One of the Expert Group’s recommendations is that a code of conduct be drawn up in cooperation with the sector to safeguard data protection and information security in primary and secondary education and training. Establishing this School Code of Conduct will entail developing and administering data protection requirements and national data impact assessments for resources with learning analytics functionality and preparing guidance materials for school owners, school administrators, teachers, pupils, parents, developers and suppliers. We also propose establishing a network for competence development and exchange of experience.

The financial and administrative consequences of this measure will depend on how the School Code of Conduct is designed and administered. The code of conduct for information security and data protection in the health and care sector is managed by a steering group with representatives from the health and care sector. The Norwegian Directorate of eHealth is the secretariat for the work of the steering group. A similar administrative arrangement and scope for the School Code of Conduct will probably require a secretariat composed of five full-time equivalent positions. Developing and regularly updating guidance materials for users in the school sector will require resources. So too will the outreach work and activities in the sector required through conferences and networks to establish and further develop the School Code of Conduct.

The Expert Group believes that, depending on the governance model (steering group, administration by a key actor or the establishment of a new body), the financial consequences will be in the range of NOK 10 million annually.

In the establishment phase, additional resources will have to be expected, as the development of national data protection impact assessments is groundbreaking work, and will require personnel resources and thorough processes in the early years.

Frameworks for good learning analytics in primary and secondary education and training

The Expert Group recommends that centrally defined quality criteria be developed for resources with functionality for learning analytics aimed at teachers, school administrators, school owners and developers. Measures for competence development in learning analytics aimed at student teachers, teachers, school administrators and school owners are recommended. The Expert Group recommends linking competence measures to ordinary schemes and instruments for basic education and supplementary and continuing education. Our assessment is therefore that the administrative and financial consequences of this undertaking will be small in relation to the overall scope of the current system.

The Expert Group recommends that national authorities establish a grant scheme to purchase and develop digital teaching aids with functionality for learning analytics. If this grant scheme is to stimulate innovative and responsible learning analytics and artificial intelligence, it must be of a certain size. We refer to the initiative The Technological Backpack where the goal was, among other things, to grant pupils access to good digital teaching aids. These measures were part of the digitalisation strategy Fremtid, fornyelse og digitalisering 2017–2021 [Future, renewal and digitalisation 2017–2021] for primary and secondary education and training][57] . The Norwegian Government allocated NOK 450 million to the 5-year initiative.

The Expert Group believes the initiative involving a grant scheme to purchase and develop digital teaching aids needs to continue and be strengthened with functionality for learning analytics. We propose that such a grant scheme has a framework of NOK 100 million annually.

Guidelines for good and justifiable learning analytics in higher education and tertiary vocational education

The Expert Group recommends the development of broad national guidelines for good and justifiable learning analytics in higher education and tertiary vocational education. To ensure that the guidelines are implemented, a support system must be established to assist the educational institutions. Training must be offered to educators, managers and support staff who assist educators in the quality assurance work.

The support system shall also ensure that newly qualified teachers possess the necessary competence in learning analytics and knowledge of artificial intelligence. Developing such a support system will require financial and administrative resources and must be investigated further.

Research and development funding

The Expert Group recommends that innovation and R&D funding be announced for digital learning resources with functionality for learning analytics and adaptivity in primary and secondary education and training, higher education and tertiary vocational education. Funding shall encompass research on the use of such resources in authentic learning situations. We propose increasing allocations to existing research programmes (e.g., FINNUT) at the Research Council of Norway, where funding is earmarked for research on learning analytics. We propose an allocation of NOK 30 million per year for this measure.

References

Akhtar, H., Silfiasari, H.A., Vekety, B. and Kovacs, K. (2022). The Effect of Computerized Adaptive Testing on Motivation and Anxiety: A Systematic Review and Meta-Analysis. Assessment. https://doi.org/10.1177/10731911221100995

Andersen, E., Johnson, J.C., Kolbjørnsrud, V. and Sannes, R. (2018). The data-driven organization: Intelligence at SCALE. In A. Sasson (ed.), At the Forefront, Looking Ahead: Research-Based Answers to Contemporary Uncertainties of Management, 23–42. Universitetsforlaget [Scandinavian University Press]. https://www.idunn.no/doi/10.18261/9788215031583-2018-03

The Article 29 Working Party. (2017). Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is “likely to result in a high risk” for the purposes of Regulation 2016/679. https://ec.europa.eu/newsroom/article29/items/611236

The Article 29 Working Party. (2018). Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 (wp251rev.01). https://ec.europa.eu/newsroom/article29/items/612053

Askeland, N., Maagerø, E. and Aamotsbakken, B. (eds.). (2013). Læreboka: studier av ulike læreboktekster [The Textbook: Studies of different textbook texts] Akademika forlag.

ASSS. (2022). Grunnleggende om ASSS-samarbeidet. KS-nettverket Aggregerte styringsdata for samarbeidende storkommuner [Basics of the ASSS collaboration. The KS Network Aggregate Management Data for Cooperating Large Municipalities]. https://www.ks.no/asss-hjem/om-asss-samarbeidet/

Baker, D.L., Basaraba, D.L., Smolkowski, K., Conry, J., Hautala, J., Richardson, U., English, S. and Cole, R. (2017). Exploring the cross-linguistic transfer of reading skills in Spanish to English in the context of a computer adaptive reading intervention. Bilingual Research Journal, 40(2), 222–239. https://doi.org/10.1080/15235882.2017.1309719

The Norwegian Ombudsperson for Children. (2021). Medvirkningshåndboka [Participation Handbook]: Metoder for å involvere barn og unge i viktige saker [Methods for involving children and young people in important matters]. https://www.barneombudet.no/uploads/documents/Publikasjoner/Andre-publikasjoner/Medvirkningshandboka_2021.pdf

Berg, R.S., Haugen, A., Lanestedt, J., Lie, N.-E.K., Pedersen, S.H., Pedersen, V., Røst, T.H., Wiborg, A. and Ørnes, H. (2020). Tilstandsrapport for høyere utdanning [Status report for higher education 2020]. The Norwegian Agency for International Cooperation and Quality Enhancement in Higher Education. https://diku.no/rapporter/diku-rapportserie-03-2020-tilstandsrapport-for-hoeyere-utdanning-2020

Blikstad-Balas, M. and Klette, K. (2020). Still a long way to go: Narrow and transmissive use of technology in the classroom. Nordic Journal of Digital Literacy, 15(1), 55–68. https://www.idunn.no/doi/10.18261/issn.1891-943x-2020-01-05

Botnevik, S., Khalil, M. and Wasson, B. (2020). Student awareness and privacy perception of learning analytics in higher education. Addressing Global Challenges and Quality Education. Proceedings of the 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, 374–379. https://doi.org/10.1007/978-3-030-57717-9_30

Bouvet. (2021). Digitalisering i skolen. Har vi glemt personvernet? [Digitalisation in schools. Have we forgotten about privacy?] https://cdn.vev.design/private/hNjwDIts16ZNNuwMVdzd56NnMyz1/38XFoNiOFC_Digitalisering%20i%20skolen%20-%20Har%20vi%20glemt%20personvernet.pdf.pdf

Broomfield, H. and Reutter, L. (2022). In search of the citizen in the datafication of public administration. Big Data & Society, 9(1), 1–14. https://journals.sagepub.com/doi/pdf/10.1177/20539517221089302

Norwegian Directorate for Children, Youth and Family Affairs (Bufdir). (2021). Prinsipper og råd: Barn og unges medvirkning på systemnivå. [Principles and advice: Children and young people’s involvement at the system level]. Bufdir. https://www.bufdir.no/fagstotte/produkter/barn_og_unges_medvirkning___veileder/

Bygrave, L.A. (2014). Core Principles of Data Privacy Law. In L.A. Bygrave (ed.), Data Privacy Law: An International Perspective, 145–168. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199675555.003.0005

Bygrave, L.A. (2020). Article 22 Automated individual decision-making, including profiling. In C. Kuner, L.A. Bygrave and C.A. Docksey (ed.), The EU General Data Protection Regulation (GDPR): A Commentary, 522-542. Oxford University Press.

C-434/16 (Nowak). (2017). ECLI:EU:C:2017:994. European Court of Justice. https://curia.europa.eu/juris/document/document.jsf;jsessionid=8719C4088E44CC4774756CD60C0569CC?text=&docid=198059&pageIndex=0&doclang=en&mode=lst&dir=&occ=first&part=1&cid=2820784

C-439/19 Latvijas Republikas Saeima [Grand Chamber]. (2021). ECLI:EU:C:2021:504. European Court of Justice. https://curia.europa.eu/juris/document/document.jsf?text=&docid=243244&pageIndex=0&doclang=en&mode=lst&dir=&occ=first&part=1&cid=4688976

C-524/06 (Huber). (2008). ECLI:EU:C:2008:724. European Court of Justice. https://curia.europa.eu/juris/document/document.jsf?text=&docid=76077&pageIndex=0&doclang=en&mode=lst&dir=&occ=first&part=1&cid=3383997

Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. The MIT Press.

Craxi (No. 2) v Italy No. 25337/94. (2003). European Court of Human Rights. https://hudoc.echr.coe.int/eng#{%22itemid%22:[%22001-61229%22]}

Cuban, L. (2001). Oversold and underused: computers in the classroom. Harvard University Press.

The Norwegian Data Protection Authority. (2015). Anonymisering av personopplysninger: Veileder [Anonymisation of personal data: A guide]. https://www.datatilsynet.no/globalassets/global/dokumenter-pdfer-skjema-ol/regelverk/veiledere/anonymisering-veileder-041115.pdf

The Norwegian Data Protection Authority. (2019). Vurdering av personvernkonsekvenser (DPIA) [Data Protection Impact Assessment (DPIA)] https://www.datatilsynet.no/rettigheter-og-plikter/virksomhetenes-plikter/vurdere-personvernkonsekvenser/vurdering-av-personvernkonsekvenser/

The Norwegian Data Protection Authority. (2020a). Nedkjølingseffekt [Chilling effect]. https://www.datatilsynet.no/regelverk-og-verktoy/rapporter-og-utredninger/personvernundersokelser/personvernundersokelsen-20192020/nedkjolingseffekt/

The Norwegian Data Protection Authority. (2020b). Personvernundersøkelsen 2019/2020: Om befolkningens holdninger til personvern og kjennskap til det nye personvernregelverket [2019/2020 Data protection survey: On the population’s attitudes regarding data protection and knowledge of the new data protection legislation]. https://www.datatilsynet.no/regelverk-og-verktoy/rapporter-og-utredninger/personvernundersokelser/personvernundersokelsen-20192020/

The Norwegian Data Protection Authority. (2021a). Høringsinnspill vedrørende endringer i opplæringsloven, friskoleloven og barnehageloven [Consultation comments regarding amendments to the Education Act, the Independent Schools Act and the Kindergarten Act]. https://www.regjeringen.no/contentassets/b50f6b063a884ffba1899c819a16d2b3/datatilsynet.pdf?uid=Datatilsynet

The Norwegian Data Protection Authority. (2021b). Årsrapport for 2020: Tall og tendenser fra Datatilsynets virksomhet [Annual Report for 2020: Figures and trends from the Norwegian Data Protection Authority’s activities]. https://www.datatilsynet.no/om-datatilsynet/arsmeldinger/arsrapport-for-2020

The Norwegian Data Protection Authority. (2022a). AVT: Sluttrapport fra sandkasseprosjektet med KS, SLATE ved UiB og Utdanningsetaten i Oslo kommune [Exit report from the Sandbox Project with KS, the Centre for the Science of Learning & Technology (SLATE) at the University of Bergen (UiB) and the City of Oslo Agency of Education: Activity Data for Assessment and Adaptivity (AVT)]. https://www.datatilsynet.no/regelverk-og-verktoy/sandkasse-for-kunstig-intelligens/ferdige-prosjekter-og-rapporter/avt-sluttrapport

The Norwegian Data Protection Authority. (2022b). Sluttrapport fra sandkasseprosjektet med NAV. Temaer: rettslig grunnlag, rettferdighet og forklarbarhet [Exit report from the Sandbox Project with the Norwegian Labour and Welfare Administration (NAV). Topics: legal basis, fairness and explainability]. https://www.datatilsynet.no/regelverk-og-verktoy/sandkasse-for-kunstig-intelligens/ferdige-prosjekter-og-rapporter/nav-sluttrapport

The Norwegian Data Protection Authority. (2022c). Å lykkes med åpenhet [Succeeding with Transparency]. https://www.datatilsynet.no/regelverk-og-verktoy/sandkasse-for-kunstig-intelligens/ferdige-prosjekter-og-rapporter/a-lykkes-med-apenhet-hvordan-informere-om-bruk-av-kunstig-intelligens/

The Norwegian Data Protection Authority. (2023). Føresette sin access to learning platforms [Parents’ and guardians’ access to learning platforms] https://www.datatilsynet.no/personvern-pa-ulike-omrader/skole-barn-unge/laringsplattformer/

de Oliveira, C.F., Sobral, S.R., Ferreira, M.J. and Moreira, F. (2021). How does learning analytics contribute to prevent students’ dropout in higher education: a systematic literature review. Big Data and Cognitive Computing, 5(4), 64 (33 pages). https://doi.org/10.3390/bdcc5040064

European Convention on Human Rights of 4 November 1950 (ECHR), Convention for the Protection of Human Rights and Fundamental Freedoms. https://lovdata.no/dokument/NL/lov/1999-05-21-30/KAPITTEL_2#KAPITTEL_2

European Court of Human Rights. (2022). Guide to the Case-Law of the European Court of Human Rights. https://www.echr.coe.int/Documents/Guide_Data_protection_ENG.pdf

European Data Protection Board (EDPB). (2022). Guidelines 01/2022 on data subject rights – Right of access. https://uk.practicallaw.thomsonreuters.com/w-034-3575?transitionType=Default&contextData=(sc. Default)

Di Mitri, D., Scheffel, M., Drachsler, H., Börner, D., Ternier, S. and Specht, M. (2017). Learning pulse: A machine learning approach for predicting performance in self-regulated learning using multimodal data. LAK ‘17: Proceedings of the seventh international learning analytics & knowledge conference, 188–197. https://doi.org/10.1145/3027385.3027447

Norwegian Directorate for Higher Education and Skills. (2022). Action Plan for Digitalisation in Higher Education and Research https://hkdir.no/vaare-tenester/handlingsplan-for-digital-omstilling-i-hoeyere-utdanning-og-forskning

Document 3:5 (2019–2020). Riksrevisjonens undersøkelse av samiske elevers rett til opplæring i og på samisk [The Office of the Auditor General’s investigation of Sámi pupils’ right to receive education in Sámi]. https://www.stortinget.no/globalassets/pdf/dokumentserien/2019-2020/dok3-201920-005.pdf

Document 16 (2011–2012). Rapport til Stortingets presidentskap fra Menneskerettighetsutvalget om menneskerettigheter i Grunnloven. Rapport fra Menneskerettighetsutvalget om menneskerettigheter i Grunnloven, avgitt 19. desember 2011 [Report by the Human Rights Commission to the Presidium of the Storting on human rights in the Constitution of Norway. Report of the Human Rights Commission on Human Rights in the Constitution of Norway, submitted on 19 December 2011]. https://www.stortinget.no/Global/pdf/Dokumentserien/2011-2012/dok16-201112.pdf

Drugli, M.B. and Nordahl, T. (2016). Forskningsartikkel: Samarbeidet mellom hjem og skole [Research article: Cooperation between the home and school]. https://www.udir.no/kvalitet-og-kompetanse/samarbeid/samarbeid-mellom-hjem-og-skole/samarbeidet-mellom-hjem-og-skole/betydningen-av-skole-hjem-samarbeid/

Egeberg, G., Hultin, H. og Berge, O. (2017). Monitor skole 2016: Skolens digitale tilstand (2. utgave). Norwegian Centre for ICT in Education [School Monitor 2016: The Digital State of the School (2nd edition)]. https://www.udir.no/globalassets/filer/tall-og-forskning/rapporter/2016/monitor_2016_bm_-_2._utgave.pdf

Egelandsdal, K., Smith, M., Hansen, C.J.S., Ness, I.J. and Wasson, B. (2019). Adaptiv læring i matematikk: empirisk rapport om Multi Smart Øving i grunnskolen [Adaptive learning in mathematics: empirical report on Multi Smart Exercises in primary and lower secondary schools]. SLATE Research Report 2019-4. Centre for the Science of Learning & Technology (SLATE). https://bora.uib.no/bora-xmlui/bitstream/handle/1956/21354/ALMAT_Egelandsdal%2520et%2520al_2019.pdf?sequence=1&isAllowed=y

Ekspertgruppen for skolebidrag. (2021). En skole for vår tid: Sluttrapport [Expert Group on School Performance Indicators. A school for our time: Final report]. https://files.nettsteder.regjeringen.no/wpuploads01/blogs.dir/415/files/2021/05/En-skole-for-v%C3%A5r-tid.pdf

The Expert Group on Learning Analytics. (2022). Læringsanalyse – noen sentrale dilemmaer. Delrapport fra ekspertgruppen for digital læringsanalyse [Learning Analytics – some key dilemmas. Interim report by the Expert Group on Learning Analytics]. https://www.regjeringen.no/contentassets/e1d6eec14bf34762ba01230e81cdc191/no/pdfs/f-4476-b-rapport-digital-laeringsanalyse.pdf

Erstad, O. (2022). Pedagogikkens relevans i den digitale tilstand [The relevance of pedagogy in the digital situation]. Nordic Journal of Pedagogy & Critique, 8, 314–327. https://doi.org/10.23865/ntpk.v8.4070

The European Commission. (2021). The Commission’s proposal, COM/2021/206 final. Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts. CELEX 52021PC0206. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52021PC0206

Council of Europe. (1981). Council of Europe Convention of 28 January 1981 No. 108 for the Protection of Individuals with regard to Automatic Processing of Personal Data Convention for the Protection of Individuals with regard to the Automatic Processing for Personal Data

Council of Europe. (2023). Committee on Artificial Intelligence (CAU): Revised zero draft [framework] Convention on Artificial Intelligence, human rights, democracy and the rule of law. https://rm.coe.int/cai-2023-01-revised-zero-draft-framework-convention-public/1680aa193f

Research, Innovation and Competence Development in Schools (FIKS). (2023). Comments to the Expert Group.

UNITED NATIONS. (1966). International Covenant on Civil and Political Rights (ICCPR) of 16 December 1966, International Covenant on Civil and Political Rights. https://lovdata.no/dokument/NL/lov/1999-05-21-30/KAPITTEL_6#KAPITTEL_6

UNITED NATIONS. (1966). International Covenant on Economic, Social and Cultural Rights (ICESCR) of 16 December 1966, International Covenant on Economic, Social and Cultural Rights. https://lovdata.no/dokument/NL/lov/1999-05-21-30/KAPITTEL_4#KAPITTEL_4

UNITED NATIONS. (1989). United Nations Convention on the Rights of the Child of 20 November 1989, Convention on the Rights of the Child. https://lovdata.no/dokument/NL/lov/1999-05-21-30/bkn#bkn

United Nations Committee on the Rights of the Child. (2013). General Comment No. 14 (2013) on the right of the child to have his or her best interests taken as a primary consideration (Art. 3, para. 1). https://www.regjeringen.no/contentassets/1376fac2fe2a427389f9f94b52acdefc/crc_c_gc_14_eng_nor.pdf

United Nations Committee on the Rights of the Child. (2021). General Comment No. 25 (2021) on children’s rights in relation to the digital environment. https://www.regjeringen.no/contentassets/1376fac2fe2a427389f9f94b52acdefc/gc-no-25_eng.pdf

Parents’ Committee for Primary and Secondary Education. (2022a). Comments to the Expert Group on Learning Analytics (28 February 2022)

Parents’ Committee for Primary and Secondary Education. (2022b). Comments to the Expert Group on Learning Analytics (28 October 2022)

Forskrift om ansettelse og opprykk i undervisnings- og forskerstillinger [Regulations concerning appointment and promotion to teaching and research posts] (2006). (FOR-2006-02-09-129). https://lovdata.no/dokument/SF/forskrift/2006-02-09-129

Forskrift til opplæringslova [Regulations pursuant to the Education Act]. (2006). (FOR-2006-06-23-724). https://lovdata.no/dokument/SF/forskrift/2006-06-23-724?q=forskrift%20til%20oppl%C3%A6ringslov

Forskrift til privatskolelova [Regulations pursuant to the Independent Schools Act]. (2006). (FOR-2006-07-14-932). https://lovdata.no/dokument/SF/forskrift/2006-07-14-932?q=forskrift%20privatskolelov

Forskrift om rammeplan for lektorutdanning for trinn 8–13 [Regulations relating to the framework plan for graduate teacher training for grades 8-13]. (2013). (FOR-2013-03-18-288). https://lovdata.no/dokument/SF/forskrift/2013-03-18-288

Forskrift om rammeplan for grunnskolelærerutdanning [Regulations relating to the framework plan for primary and lower secondary teacher training for grades 1-7]. (2016). (FOR-2016-06-07-860). https://lovdata.no/dokument/SF/forskrift/2016-06-07-860

Forskrift om rammeplan for grunnskolelærerutdanning for trinn 5–10. [Regulations relating to the framework plan for primary and lower secondary teacher training for grades 5-10]. (2016). (FOR-2016-06-07-861). https://lovdata.no/dokument/SF/forskrift/2016-06-07-861

Forskrift om tilsyn med utdanningskvaliteten i høyere utdanning (studietilsynsforskriften) [Regulations relating to the supervision of the quality of education in higher education (Academic Supervision Regulations)]. (2017). (FOR-2017-02-07-137). https://lovdata.no/dokument/SF/forskrift/2017-02-07-137?q=studietilsyn

Forskrift om endring i forskrift om kvalitetssikring og kvalitetsutvikling i høyere utdanning og fagskoleutdanning (studiekvalitetsforskriften) [Regulations relating to quality assurance and quality development in higher education and vocational education (Regulations on the quality of programmes of study)]. (2018). (FOR-2018-01-03-6). https://lovdata.no/dokument/SF/forskrift/2018-01-03-6?q=studiekvalitet

Forskrift om høyere yrkesfaglig utdanning (fagskoleforskriften) [Regulations relating to tertiary vocational education (Vocational Education Regulations) (2019). (FOR-2019-07-11-1005). https://lovdata.no/dokument/SF/forskrift/2019-07-11-1005?q=fagskole

Forskrift om akkreditering av og tilsyn med høyere yrkesfaglig utdanning (fagskoletilsynsforskriften) [Regulations relating to accreditation and supervision of tertiary vocational education (Vocational Education Academic Supervision Regulations)]. (2020). (FOR-2020-04-23-853). https://lovdata.no/dokument/SF/forskrift/2020-04-23-853?q=fagskole

Faannessen, M., Meltevik, S., Burkeland, I., Simonsen, M.F., Nordahl, B. and Stiberg-Jamt, R. (2022). Elevmedvirkning: FoU om hvordan elevmedvirkning kan styrkes i norske skoler i tråd med nytt læreplanverk [Pupil participation: R&D on how pupil participation can be strengthened in Norwegian schools in line with the new national curriculum]. KS Consultant AS / Oxford Research. https://www.ks.no/contentassets/a493533676044e4c804ddd0637e3112f/FoU-elevmedvirkning.pdf

Giannakos, M., Spikol, D., Di Mitri, D., Sharma, K., Ochoa, X. and Hammad, R. (eds.). (2022). The Multimodal Learning Analytics Handbook. Springer Nature

Giannakos, M.N., Sharma, K., Pappas, I.O., Kostakos, V. and Velloso, E. (2019). Multimodal data as a means to understand the learning experience. International Journal of Information Management, 48, 108–119. https://doi.org/10.1016/j.ijinfomgt.2019.02.003

Gilje, Ø., Bjerke, Å. and Thuen, F. (2020). Gode eksempler på praksis. Undervisning i én-til-én-klasserommet [Good examples of practice. One-to-one classroom teaching]. University of Oslo: Research, Innovation and Competence Development in Schools (FIKS). https://www.uv.uio.no/forskning/satsinger/fiks/kunnskapsbase/digitalisering-i-skolen%20%28tidligere%20versjon%29/gepp-rapport--undervisning-i-en-til-en-klasseromme/gepp-rapport_15.05.20_fiks.pdf

Gudmundsdottir, G.B. and Hatlevik, O.E. (2018). Newly qualified teachers’ professional digital competence: implications for teacher education. European Journal of Teacher Education, 41(2), 214–231. https://doi.org/10.1080/02619768.2017.1416085

BI Norwegian Business School. (2022). Comments to the Expert Group on Learning Analytics (29 March 2022)

BI Norwegian Business School. (2023). Comments to the Expert Group on Learning Analytics (24 February 2022)

Hauge, M.S., Bakken, P. and Guajardo, G. (2023). The Student Survey 2022 – Main trends. Norwegian Agency for Quality Assurance in Education (NOKUT), report 2-2023. https://www.nokut.no/globalassets/studiebarometeret/2023/hoyere-utdanning/studiebarometeret-2022_hovedtendenser_2-2023.pdf

Høiseth-Gilje, K., Vikøren, S., Foseid, H.M., Gierløff, C.W., Ofstad, B. and Overøye, T.K. (2022). Samfunnsøkonomisk analyse av anskaffelse av digitale læremidler [Socioeconomic analysis of the procurement of digital teaching aids]. Menon Economics. Publication 58/2022. https://www.menon.no/wp-content/uploads/2022-58-Samfunnsokonomisk-analyse-digitale-laeremidler.pdf

ICT-Norway. (2023). ICT-Norway’s comments to the Expert Group on Learning Analytics (05 March 2023)

Innst. 186 S (2013–2014) [Recoomendation to the Storting No. 186 (2013–2014)]. Innstilling fra kontroll- og konstitusjonskomiteen om grunnlovsforslag fra Per-Kristian Foss, Martin Kolberg, Marit Nybakk, Jette F. Christensen, Anders Anundsen, Hallgeir H. Langeland, Per Olaf Lundteigen, Geir Jørgen Bekkevold og Trine Skei Grande om grunnlovfesting av sivile og politiske menneskerettigheter, med unntak av romertall X og romertall XXIV [Recommendation by the Standing Committee on Scrutiny and Constitutional Affairs on proposed amendment to the Constitution of Norway by Per-Kristian Foss, Martin Kolberg, Marit Nybakk, Jette F. Christensen, Anders Anundsen, Hallgeir H. Langeland, Per Olaf Lundteigen, Geir Jørgen Bekkevold and Trine Skei Grande on the constitutionalisation of civil and political human rights, with the exception of Roman numeral X and Roman numeral XXIV]. Standing Committee on Scrutiny and Constitutional Affairs. https://www.stortinget.no/globalassets/pdf/innstillinger/stortinget/2013-2014/inns-201314-186.pdf

Instefjord, E. and Munthe, E. (2016). Preparing pre-service teachers to integrate technology: an analysis of the emphasis on digital competence in teacher education curricula. European Journal of Teacher Education, 39(1), 77–93. https://doi.org/10.1080/02619768.2015.1100602

Johnson, W.G. (2018). Data Mining and Machine Learning in Education with Focus in Undergraduate CS Student Success. In L. Malmi and A. Korhonen (ed.), Proceedings of the 2018 ACM Conference on International Computing Education Research, 270–271. https://doi.org/10.1145/3230977.3231012

Norwegian Association of Lawyers. (2021). Innbyggernes rettssikkerhet er truet uten jurister i kommunene [The due process guarantees of citizens is threatened without lawyers in the municipalities]. https://www.juristforbundet.no/nyheter/2021/innbyggernes-rettssikkerhet-er-truet-uten-jurister-i-kommunene/

Norwegian Ministry of Justice and Public Security. (2022). Tolkningsuttalelse fra lovavdelingen: § 2 – Artikkel 6 og 9 – Tolkningsuttalelse – DSOP Kontrollinformasjon [Interpretation of Conventions and Recommendations of the Legislation Department: Section 2 – Articles 6 and 9 – Interpretation of Conventions and Recommendations – DSOP Control Information]. https://www.regjeringen.no/no/dokumenter/tolkningsuttalelse-dsop-kontrollinformasjon/id2937056/

Kapur, M. (2008). Productive failure. Cognition and instruction, 26(3), 379–424. https://www.tandfonline.com/doi/abs/10.1080/07370000802212669

Kay, D. and Harmelen, M. Activity data – delivering benefits from the data deluge. http://www.jisc.ac.uk/publications/reports/2012/activity-data-delivering-benefits.aspx

Kluge, A. (2021). Læring med digital teknologi: teorier og utviklingstrekk [Learning with digital technology: theories and development trends]. Cappelen Damm Akademisk.

Norwegian Ministry of Local Government and Modernisation. (2019). One digital public sector: Digitalisation strategy for the public sector 2019–2025. https://www.regjeringen.no/contentassets/db9bf2bf10594ab88a470db40da0d10f/no/pdfs/digitaliseringsstrategi_for_offentlig_sektor_rettet.pdf

Norwegian Ministry of Local Government and Modernisation. (2020). National Strategy for Artificial Intelligence. https://www.regjeringen.no/contentassets/1febbbb2c4fd4b7d92c67ddd353b6ae8/no/pdfs/ki-strategi.pdf

Constitution of the Kingdom of Norway of 17 May 1814. https://lovdata.no/dokument/NL/lov/1814-05-17?q=grunnlov

Kotschy, W. (2020). Article 6 Lawfulness of processing. In C. Kuner, L.A. Bygrave and C.A. Docksey (ed.), The EU General Data Protection Regulation (GDPR): A Commentary, 321–344. Oxford University Press.

The Norwegian Association of Local and Regional Authorities (KS) (2021). Betalingsmodell [Payment model]. https://www.ks.no/fagomrader/digitalisering/utviklingsprosjekter/laringsanalyse-i-skolen/betalingsmodell/

Norwegian Ministry of Education and Research (2017). Core curriculum – values and principles for primary and secondary education. Issued as Regulations by Royal Decree. The Curriculum for Knowledge Promotion in Primary and Secondary Education and Training 2020. https://www.udir.no/lk20/overordnet-del/

Norwegian Ministry of Education and Research. (2019). Høringsnotat om endringer i forskrift om høyere yrkesfaglig utdanning og forskrift om kvalitetssikring og kvalitetsutvikling i høyere utdanning og fagskoleutdanning [Consultation paper on amendments to the Regulations relating to tertiary vocational education and the Regulations relating to quality assurance and quality development in higher education and tertiary vocational education]. https://www.regjeringen.no/contentassets/edee44943e2c43aab6d1da13b2e7b5ba/horingsnotat-om-endringer-i-forskrift-om-hoyere-yrkesfaglig-utdanning-og-forskrift-om-kvalitetssikring-og-kvalitetsutvikling-i-hoyere-utdanning-og-fagskoleutdanning.pdf

Norwegian Ministry of Education and Research. (2020). Handlingsplan for digitalisering i grunnopplæringen (2020–2021) [Action Plan for digitalisation in primary and secondary education and training (2020–2021)]. https://www.regjeringen.no/contentassets/44b8b3234a124bb28f0a5a22e2ac197a/handlingsplan-for-digitalisering-i-grunnopplaringen-2020-2021.pdf

Norwegian Ministry of Education and Research. (2021a). Strategy for digital transformation in the higher education sector 2021–2025. https://www.regjeringen.no/contentassets/c151afba427f446b8aa44aa1a673e6d6/no/pdfs/kd-strategi-digital-omstilling.pdf

Norwegian Ministry of Education and Research. (2021b). Tildelingsbrev til Utdanningsdirektoratet for 2021 [Letter of allocation to the Norwegian Directorate for Education and Training for 2021]. https://www.regjeringen.no/contentassets/d7ff3b0447e446a0865cdc409d857825/tildelingsbrev-til-utdanningsdirektoratet-for-2021.pdf

Norwegian Ministry of Education and Research. (2023). Strategi for digital kompetanse og infrastruktur i barnehage og skole 2023–2030 [Strategy for digital competence and infrastructure in kindergartens and schools 2023–2030]. https://www.regjeringen.no/no/dokumenter/strategi-for-digital-kompetanse-og-infrastruktur-i-barnehage-og-skole/id2972254/

L.H. v. Latvia No. 52019/07. (2014). European Court of Human Rights. https://hudoc.echr.coe.int/eng#{%22itemid%22:[%22001-142673%22]}

Langford, M., Blikstad-Balas, M., Bull-Berg, J.A.B., Dolonen, J.A., Markgraf-Bye, R., Evju, A., Nenseth, V., Skogan, J.H., Weitzenboeck, E.-M. and Finseth, L.C. (2022). Læringsanalyse og kvalitet i utdanningene ved UiO: Forslag til personvernpolicy En rapport fra arbeidsgruppen for utvikling av en GDPR-policy for arbeid med kvalitet i utdanningene ved UiO [Learning analytics and quality in education at UiO: Proposal for a privacy policy. A report by the Working Group for the development of a GDPR policy for work on quality assurance in education at UiO]. University of Oslo. https://www.jus.uio.no/cell/publikasjoner/rapporter/pdf/rapporten-gdpr-la-uio-10062022-(1).pdf

Larsen, A. (2020). Maskinlæring i profesjonell fotball: Kan man ved hjelp av multidimensjonale modeller predikere ikke-kontakt skader i et norsk eliteserielag i fotball? [Machine learning in professional football: Can multidimensional models predict non-contact injuries in a Norwegian Eliteserien football club?]. https://nih.brage.unit.no/nih-xmlui/bitstream/handle/11250/2660753/Larsen%20A%20v2020%20.pdf?sequence=1&isAllowed=y

Lester, J., Klein, C., Johri, A. Rangwala, H. (2018). Learning Analytics in Higher Education: Current Innovations, Future Potential, and Practical Applications. Routledge.

Lov om barn og foreldre (barnelova) [Act of 8 April 1981 No. 7 relating to children and parents (Children Act)]. https://lovdata.no/dokument/NL/lov/1981-04-08-7?q=barnelova

Lov om grunnskolen og den vidaregåande opplæringa (opplæringslova) [Act of 17 July 1998 No. 61 relating to primary and secondary education and training (Education Act)]. https://lovdata.no/dokument/NL/lov/1998-07-1761?q=oppl%C3%A6ringslov

Lov om styrking av menneskerettighetenes stilling i norsk rett (menneskerettsloven) [Act of 21 May 1999 No. Article 30 relating to the strengthening of the status of human rights in Norwegian law (The Human Rights Act)]. https://lovdata.no/dokument/NL/lov/1999-05-21-30?q=styrking%20av%20menneskerettighetenes%20stilling%20i

Lov om private skolar med rett til statstilskot (privatskolelova) [Act of 4 July 2003 No. 84 relating to independent schools (The Independent Schools Act)]. https://lovdata.no/dokument/NL/lov/2003-07-04-84?q=privatskole

Lov om universiteter og høyskoler (universitets- og høyskoleloven) [Act of 1 April 2005 No. 15 relating to universities and university colleges (the Universities and University Colleges Act)]. https://lovdata.no/dokument/NL/lov/2005-04-01-15?q=uhl

Lov om høyere yrkesfaglig utdanning (fagskoleloven) [Act of 8 June 2018 No. 28 relating to tertiary vocational education (The Vocational Education Act)]. https://lovdata.no/dokument/NL/lov/2018-06-08-28?q=fagskolelov

Lov om behandling av personopplysninger (personopplysningsloven) [Act of 15 June 2018 No. 38 relating to the processing of personal data (The Personal Data Act)]. https://lovdata.no/dokument/NL/lov/2018-06-15-38?q=personopplysningslov

Lund, A. (2021). The Norwegian Ministry of Education and Research’s action plan for digitalization in primary and secondary education and training: appraisal and critique. Nordic Journal of Digital Literacy, 16(1), 34–42. https://doi.org/10.18261/issn.1891-943x-2021-01-04

Mahler, T. (2022). Between Risk Management and Proportionality: The Risk-Based Approach in the EU’s Artificial Intelligence Act Proposal. In L. Colonna and S. Greenstein (eds.), Nordic Yearbook of Law and Informatics: Law in the Era of Artificial Intelligence (Swedish Law and Informatics Research Institute (IRI)), 245–267.

Marper v. United Kingdom [Grand Chamber] No. 30562/04 and 30566/04. (2008). European Court of Human Rights. https://rm.coe.int/168067d216

Martinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G. Buckingham Shum, S. (2020). From data to insights: A layered storytelling approach for multimodal learning analytics. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–15. https://doi.org/10.1145/3313831.3376148

Mausethagen, S. and Mølstad, C.E. (2015). Shifts in curriculum control: contesting ideas of teacher autonomy. Nordic Journal of Studies in Educational Policy, 2015(2), 30–41. https://doi.org/10.3402/nstep.v1.28520

McTigue, E.M., Solheim, O.J., Zimmer, W.K. Uppstad, P.H. (2020). Critically reviewing GraphoGame across the world: Recommendations and cautions for research and implementation of computer-assisted instruction for word-reading acquisition. Reading Research Quarterly, 55(1), 45–73. https://doi.org/10.1002/rrq.256

McTigue, E.M. and Uppstad, P.H. (2019). Getting serious about serious games: Best practices for computer games in reading classrooms. The Reading Teacher, 72(4), 453–461. https://doi.org/10.1002/trtr.1737

Report to the Storting No. 5 (2022–2023) Long-term plan for research and higher education 2023–2032. Norwegian Ministry of Education and Research. https://www.regjeringen.no/no/dokumenter/meld.-st.-5-20222023/id2931400/

Report to the Storting No. 8 (2022–2023) Human rights for persons with intellectual disabilities – It is about being seen and heard. Norwegian Ministry of Culture and Equality. https://www.regjeringen.no/no/dokumenter/meld.-st.-8-20222023/id2945431/

Report to the Storting No. 9 (2016–2017) Skilled Workers for the Future — Vocational college education. Norwegian Ministry of Education and Research. https://www.regjeringen.no/no/dokumenter/meld.-st.-9-20162017/id2522412/?ch=1

Report to the Storting No. 16 (2016–2017) Quality Culture in Higher Education. Norwegian Ministry of Education and Research. https://www.regjeringen.no/no/dokumenter/meld.-st.-16-20162017/id2536007/

Report to the Storting No. 22 (2020–2021) Data as a resource – The data-driven economy and innovation. Norwegian Ministry of Local Government and Modernisation. https://www.regjeringen.no/contentassets/4f357e18bd314dc08c8e1b447b71b700/no/pdfs/stm202020210022000dddpdfs.pdf

Report to the Storting No. 27 (2015–2016) Digital agenda for Norge – IKT for en enklere hverdag og økt produktivitet [Digital agenda for Norway – ICT for an easier everyday life and increased productivity]. Norwegian Ministry of Local Government and Modernisation. https://www.regjeringen.no/contentassets/fe3e34b866034b82b9c623c5cec39823/no/pdfs/stm201520160027000dddpdfs.pdf

Misiejuk, K., Ness, I.J., Gray, R. and Wasson, B. (2023). Changes in online course designs: Before, during, and after the pandemic. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.996006

Misiejuk, K. and Wasson, B. (2017). State of the Field report on Learning Analytics. SLATE Research Report 2017-2. Centre for the Science of Learning & Technology (SLATE). https://bora.uib.no/bora-xmlui/handle/1956/17740

Moltudal, S., Høydal, K. and Krumsvik, R.J. (2020). Glimpses Into Real-Life Introduction of Adaptive Learning Technology: A Mixed Methods Research Approach to Personalized Pupil Learning. Designs for learning, 12(1), 13–28. https://doi.org/10.16993/dfl.138

Munthe, E., Erstad, O., Njå, M.B., Forsström, S., Gilje, Ø., Amdam, S., Moltudal, S. and Hagen, S.B. (2022). Digitalisering i grunnopplæring; kunnskap, trender og framtidig forskningsbehov [Digitalisation in primary and secondary education and training; knowledge, trends and future research needs]. Knowledge Centre for Education: University of Stavanger. https://www.uis.no/sites/default/files/2022-12/13767200%20Rapport%20GrunDig_0.pdf

Mølstad, CE and Karseth, B. (2016). National curricula in Norway and Finland: The role of learning outcomes. European Educational Research Journal, 15(3), 329–344. https://journals.sagepub.com/doi/pdf/10.1177/1474904116639311

Møre and Romsdal County Authority. (2022). Comments to the Norwegian Ministry of Education and Research’s National Expert Group on Learning Analytics in Primary and Secondary Education and Training (24 February 2022).

Neddy. (2023). Comments to the Expert Group on Learning Analytics (02 March 2023)

Norwegian Agency for Quality Assurance in Education (NOKUT) (2017). Tilsyn med nettbaserte tekniske fagskoleutdanninger [Supervision of online technical vocational college educations]. NOKUT’s supervision reports. Norwegian Agency for Quality Assurance in Education (NOKUT). https://www.nokut.no/globalassets/nokut/rapporter/tilsyn-fagskoletilbud/2017/tilsyn_nettbaserte_tekniske_fagskoleutdanninger_2017.pdf

Norwegian Agency for Quality Assurance in Education (NOKUT) (2019). Consultation paper – forslag til ny forskrift om tilsyn med kvaliteten i høyere yrkesfaglig utdanning (fagskoletilsynsforskrift) [proposal for new regulations on supervision of quality in tertiary vocational education (Vocational Education Academic Supervision Regulations). Norwegian Agency for Quality Assurance in Education (NOKUT). https://www.nokut.no/globalassets/nokut/artikkelbibliotek/norsk_utdanning/horing/horingsnotat--proslog-til-ny-forskript-om-tilsyn-med-kvaliteten-i-hoyere-yrkesfaglig-utdanning-fagskoletilsynsforscriptionen.pdf

Norwegian Agency for Quality Assurance in Education (NOKUT) (2020). Merknader til studietilsynsforskriften (forskrift om tilsyn med utdanningskvaliteten i høyere utdanning [Comments to the Academic Supervision Regulations (Regulations relating to the supervision of the quality of education in higher education)], 7 February 2017 No. 137). NOKUT – National body for quality in education. https://www.nokut.no/contentassets/13569b66043545fa95102e966004cb2b/merknader-til-studietilsynsforskriften_nokut_250220.pdf

Norwegian Agency for Quality Assurance in Education (NOKUT) (2022). Systematisk kvalitetsarbeid i høyere utdanning: Veiledning til kravene i universitets- og høyskoleloven, studiekvalitetsforskriften og studietilsynsforskriften [Systematic quality assurance work in higher education: Guide to the requirements in the Universities and University Colleges Act, the Academic Supervision Regulations and the Regulations relating to the supervision of the quality of education in higher education]. Norwegian Agency for Quality Assurance in Education (NOKUT). https://www.nokut.no/siteassets/tilsyn-og-kvalitetsarbeid/nokuts-veileder-til-institusjonenes-systematiske-kvalitetsarbeid_august-2022.pdf

Norwegian Association of Graduate Teachers. (2022). Comments to the Expert Group on Learning Analytics (30 May 2022)

Norwegian Association of Graduate Teachers. (2023). Comments to the Expert Group on Learning Analytics (28 February 2023).

NOU 2015: 8 The School of the Future — Renewal of subjects and competences. https://www.regjeringen.no/no/dokumenter/nou-2015-8/id2417001/

NOU 2019: 23 Ny opplæringslov [New Education Act]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/no/dokumenter/nou-2019-23/id2682434/

NOU 2020: 3 Ny lov om universiteter og høyskoler [New Act on universities and university colleges]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/65a475004027495a8009b359e253b19e/no/pdfs/nou202020200003000dddpdfs.pdf

NOU 2020: 14 Ny barnelov – Til barnets beste [New Children’s Act – In the best interests of the child]. Norwegian Ministry of Children and Families. https://www.regjeringen.no/no/dokumenter/nou-2020-14/id2788399/

NOU 2022: 11 Your privacy – our shared responsibility. Time for a privacy policy. Norwegian Ministry of Education and Research. https://www.regjeringen.no/no/dokumenter/nou-2022-11/id2928543/

NOU 2023: 1 Kvalitetsvurdering og kvalitetsutvikling i skolen: Et kunnskapsgrunnlag [Quality Assessment and Quality Development in Schools: A knowledge base]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/8758c1a973ae4f94ab67e3830c96b9c9/no/pdfs/nou202320230001000dddpdfs.pdf

Organisation of Norwegian Vocational Students (2022). Comments to the Expert Group on Learning Analytics (23 May 2022)

Oslo Economics. (2022). Markedet for digitale læremidler og læringsressurser i grunnskolen og videregående opplæring [The market for digital teaching aids and learning resources in primary, lower secondary and upper secondary education]. OE report 2022-27. https://osloeconomics.no/publication/markedet-for-digitale-laeremidler-og-laeringsressurser-i-grunnskolen-og-videregaende-opplaering/

Ot.prp. nr. 40 (2007-2008). Om lov om endringar i opplæringslova og privatskolelova [On the Act relating to amendments to the Education Act and the Independent Schools Act. Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/6b94ab11f51c4f57b9b34bb53201c0bd/nn-no/pdfs/otp200720080040000dddpdfs.pdf

General Data Protection Regulations. (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of individuals with regard to the processing of personal data and on the free movement of such data and repealing Directive 95/46/EC (27-04-2016) (2016). https://lovdata.no/static/NLX3/32016r0679.pdf

PG and JH v Great Britain No. 44787/98. (2001). European Court of Human Rights

Pintrich, P.R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into practice, 41(4), 219–225. https://www.tandfonline.com/doi/abs/10.1207/s15430421tip4104_3?journalCode=htip20

Pipino, L.L., Lee, Y.W. Wang, R.Y. (2002). Data quality assessment. Communications of the ACM, 45(4), 211–218. https://dl.acm.org/doi/abs/10.1145/505248.506010

Prop. 52 L (2017–2018) Endringar i opplæringslova, friskolelova og folkehøgskolelova (plikt til å tilby intensiv opplæring og plikt til fleirfagleg samarbeid m.m.) [Amendments to the Education Act, the Independent Schools Act and the Folk High School Act (duty to offer intensive training and duty to engage in multidisciplinary cooperation etc.)]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/68819e1ca994428db2219eb459d56ecc/nn-no/pdfs/prp201720180052000dddpdfs.pdf

Prop. 56 LS (2017–2018) Act relating to the processing of personal data (Personal Data Act) and consent to participate in a decision in the EEA Joint Committee on the incorporation of Regulation (EU) No. 2016/679 (General Data Protection Regulation) in the EEA Agreement. Norwegian Ministry of Justice and Public Security. https://www.regjeringen.no/no/dokumenter/prop.-56-ls-20172018/id2594627/

Prop. 57 L (2022–2023) Act relating to primary and secondary education and training (the Education Act). Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/f50a3e82602c4dd9b7a6e8d558fb328f/nn-no/pdfs/prp202220230057000dddpdfs.pdf

Prop. 59 L (2017–2018) Endringer i utlendingsloven (behandling av personopplysninger) [Amendments to the Immigration Act (processing of personal data)]. Norwegian Ministry of Justice and Public Security. https://www.regjeringen.no/contentassets/30916adb5e794f4bbee0acb00fde6553/no/pdfs/prp201720180059000dddpdfs.pdf

Prop. 64 L (2017–2018) Endringer i universitets- og høyskoleloven (NOKUTs oppgaver, eksamen og personvern mv.) [Amendments in the Universities and University Colleges Act (NOKUT’s tasks, examinations and data protection etc.)]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/5a367f93bd864a4f92bd7669f5ea993f/no/pdfs/prp201720180064000dddpdfs.pdf

Prop. 74 L (2021–2022) Endringer i opplæringsloven, universitets og høyskoleloven og fagskoleloven (endring av NOKUTs oppgaver, studentrettigheter mv.) [Amendments to the Education Act, the Universities and University Colleges Act and the Vocational Education Act (changes to NOKUT’s tasks, student rights etc.)]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/contentassets/0601f156a7b14330b7d5856f4f7109a6/no/pdfs/prp202120220074000dddpdfs.pdf

Prop. 81 L (2019–2020) Endringer av internkontrollregler i sektorlovgivningen (tilpasning til ny kommunelov) [Amendments to internal control rules in sectoral legislation (adaptation to new Local Government Act)]. Norwegian Ministry of Local Government and Modernisation. https://www.regjeringen.no/contentassets/cab55c241ccb413abaf61c33eebba795/no/pdfs/prp201920200081000dddpdfs.pdf

Prop. 111 L (2020–2021) Endringer i universitets- og høyskoleloven, utdanningsstøtteloven, fagskoleloven og yrkeskvalifikasjonsloven mv. (samleproposisjon) [Amendments to the Universities and University Colleges Act, Student Financial Aid Act, Vocational Education Act and the Professional Qualifications Act, etc. (joint bill)]. Norwegian Ministry of Education and Research https://www.regjeringen.no/contentassets/747245434bb04cd5a378c14330f173fa/no/pdfs/prp202020210111000dddpdfs.pdf

Prop. 145 L (2020–2021) Endringar i opplæringslova, friskulelova og barnehagelova (behandling av personopplysningar, fjernundervisning o.a.) [Amendments in the Education Act, the Independent Schools Act and the Kindergarten Act (processing of personal data, distance education etc.)]. Norwegian Ministry of Education and Research. https://www.regjeringen.no/no/dokumenter/prop.-145-l-20202021/id2842962/

Rambøll. (2023). Digital læringsanalyse i norsk utdanning: omfang, pedagogisk praksis og holdninger [Learning analytics in Norwegian education: scope, pedagogical practice and attitudes]. Rambøll Management Consulting AS.

Save the Children Norway. (2023). Comments to the Expert Group on Learning Analytics (14 February 2023)

Supreme Court Reports (Rt.) 2014 p. 1105. Supreme Court of Norway

Supreme Court Reports (Rt.) 2015 p. 93. Supreme Court of Norway

Salomon, G. (2016). It’s not just the tool but the educational rationale that counts. In E. Elstad (ed.) Educational technology and polycontextual bridging, 147–161. Springer Link

Sampson, D., Papamitsiou, Z., Ifenthaler, D., Giannakos, M., Mougiakou, S. and Vinatsella, D. (2022). Educational Data Literacy. Springer Nature. https://hdl.handle.net/11250/3047111

Samuelsen, J., Chen, W. and Wasson, B. (2019). Integrating multiple data sources for learning analytics – review of literature. Research and Practice in Technology Enhanced Learning (RPTEL), 14(11), 1–20. https://doi.org/10.1186/s41039-019-0105-4

Samuelsen, J., Chen, W. and Wasson, B. (2021). Enriching context descriptions for enhanced LA scalability: a case study. Research and Practice in Technology Enhanced Learning (RPTEL), 16(6). https://doi.org/10.1186/s41039-021-00150-2

Satakunnan Markkinapörssi Oy and Satamedia Oy v. Finland [Grand Chamber] No. 931/13. (2017). European Court of Human Rights. https://hudoc.echr.coe.int/eng#{%22itemid%22:[%22001-175121%22]}

Sclater, N. and Bailey, P. (2018). Code of practice for learning analytics: Setting out the responsibilities of educational institutions to ensure that learning analytics is carried out responsibly, appropriately and effectively. Jisc. https://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics

Selwyn, N. (2022). The future of AI and education: Some cautionary notes. European Journal of Education, 57(4), 620–631. https://doi.org/10.1111/ejed.12532

Sikt. (2022). Comments to the Expert Group on Learning Analytics (01 April 2022)

Skedsmo, G. (2022). Ulike aktørers behov for informasjon for å vurdere og utvikle kvaliteten i og av skolen. Kunnskapsoppsummering til det offentlige utvalget for kvalitetsutvikling i skolen. [Various actors’ need for information to assess and develop the quality in and of schools. Knowledge summary for the Committee for Quality Development in Schools]. https://files.nettsteder.regjeringen.no/wpuploads01/sites/508/2023/02/Forskningsoppsumering-Skedsmo-22-12-12.pdf

Norwegian School Heads Association. (2022). Comments to the Expert Group on Learning Analytics (24 March 2022)

Norwegian Union of School Employees. (2023). Comments to the Expert Group on Learning Analytics (01 March 2022)

Norwegian National Support System for Special Needs Education (Statped). (2022). Comments to the Expert Group on Learning Analytics (01 April 2022)

SURF. (2019). Learning analytics in 5 steps: A guide to the GDPR. https://www.surf.nl/files/2020-06/learning-analytics-in-5-steps.pdf

Throndsen, I., Carlsten, T.C. and Björnsson, J.K. (2019). TALIS 2018: Første hovedfunn fra ungdomstrinnet [Initial main findings from the lower secondary level]. Department of Teacher Education and School Research, University of Oslo / Nordic Institute for Studies of Innovation, Research and Education (NIFU). http://hdl.handle.net/11250/2601320

Tønnesen, E.S. (2013). Læreboka som kunnskapsdesign [The textbook as knowledge design]. In N. Askeland, E. Maagerø and B. Aamotsbakken (eds.), Læreboka: studier i ulike læreboktekster [The textbook: studies in various textbooks], 147–163. Akademika forlag.

Udsen, H. (2022). Læreboka: studier i ulike læreboktekster [Data protection law]. https://databestelltskeitsret.dk/wp-content/uploads/Databestelltskeetsret-2022-11-263.pdf

Ungdomspanelet i Møre og Romsdal Youth Panel of Møre og Romsdal]. (2023). Comments by the Youth Panel in Møre og Romsdal (14 March 2023)

Unit. (2020). Foreløpig rapport: Arbeidsgruppe Canvas Data [Preliminary report: Canvas Data Working Group]. Norwegian Directorate for ICT and joint services in higher education and research (Unit). https://unit.atlassian.net/wiki/spaces/LMS/pages/2319286277/Canvas+Data

Universities Norway. (2023). Comments to the Expert Group on Learning Analytics (27 February 2022)

Norwegian Directorate for Education and Training. (2019). Rapport, Felles nasjonalt tilsyn – 2014-2017 [Report, Joint national supervision – 2014-2017]. https://www.udir.no/regelverk-og-tilsyn/tilsyn/felles-nasjonalt-tilsyn/felles-nasjonalt-tilsyn-2014-2017-/skolebasert-vurdering/

Norwegian Directorate for Education and Training. (2021a). Individuell vurdering Udir-2-2020 (Udir-2-2020) [Rundskriv]. [Individual assessment Udir-2-2020 (Udir-2-2020) [Circular]]. Norwegian Directorate for Education and Training. https://www.udir.no/regelverkstolkninger/opplaring/Vurdering/udir-2-2020-individuell-vurdering/

Norwegian Directorate for Education and Training. (2021b). Kunnskapsgrunnlag for kvalitetskriterium for læremiddel i norsk [Platform of knowledge for quality criteria for learning materials in Norwegian language]. https://www.udir.no/kvalitet-og-kompetanse/laremidelr/kvalitetskriterier-for-laremidelr/kunnskapsgrunnlag-kvalitetskriterium-norsk/

Norwegian Directorate for Education and Training. (2022a). Innspill til strategi for digital kompetanse og infrastruktur i barnehage, grunnskole og videregående opplæring 2023–2030 [Comments to the strategy for digital competence and infrastructure in kindergartens, primary and lower secondary school and upper secondary education 2023–2030]. https://www.udir.no/om-udir/udirs-svar-pa-oppdrag-fra-kunnskapsdepartementet/inspill-til-strategi-for-digital-kompetanse-og-infrastruktur-i-barnehage-grunnskole-og- advanced-education-20232030/

Norwegian Directorate for Education and Training. (2022b). Tilskudd for kjøp av digitale læremidler 2022. [Grant for the purchase of digital teaching aids 2022]. https://www.udir.no/om-udir/tilskudd-og-prosjektmidler/tilskudd-for-skoler/tilskudd-for-kjop-av-digitale-laremidelr/

Norwegian Directorate for Education and Training. (2023). Consultation response from the Norwegian Directorate for Education and Training. https://www.regjeringen.no/no/dokumenter/horing-nou-202211-ditt-personvern-vart-felles-ansvar.-personvernkommissionens-rapport/id2947128/?uid=0b6495f7-6c80-47ad-ac8f-dacdb70571a6

Union of Education Norway. (2022). Comments to the Expert Group’s work (22 April 2022)

Veale, M. and Borgesius, F.Z. (2021). Demystifying the Draft EU Artificial Intelligence Act – Analyzing the good, the bad, and the unclear elements of the proposed approach. Computer Law Review International, 22.4, 97-112

Vestfold and Telemark County Authority. (2022). Comments to the Expert Group on Learning Analytics (18 March 2022)

Vestland ungdomsutval [Youth Council of Vestland County]. (2023). Proceedings in the Youth Council of Vestland County – 06 February 2023. ICT-Norway’s comments to the Expert Group on Learning Analytics (06 February 2023)

Viken ungdomsråd [Viken Youth Council]. (2023). Comments to the Expert Group on Learning Analytics (28 February 2023).

Vukota-Bojić v Switzerland No. 61838/10. (2016). European Court of Human Rights. https://hudoc.echr.coe.int/fre#{%22itemid%22:[%22001-167490%22]}

Winne, P.H. (2015). Self-Regulated Learning. In J.D. Wright (ed.), International Encyclopedia of the Social & Behavioral Sciences, 2nd ed., 535–540. https://www.sciencedirect.com/science/article/pii/B9780080970868250915

Worsley, M., Martinez-Maldonado, R. and D’Angelo, C. (2021). A New Era in Multimodal Learning Analytics: Twelve Core Commitments to Ground and Grow MMLA. Journal of Learning Analytics, 8(3), 10–27. https://doi.org/10.18608/jla.2021.7361

Ørnes, H., Oboza, A., Refsnes, S. and Landø, A. (2021). Digital tilstand 2021: Støttemiljøers arbeid med digitalisering og utdanningskvalitet ved universiteter og høyskoler. [Support environments’ work on digitalisation and quality of education at universities and university colleges]. Directorate for Higher Education and Skills., Report No. 06/2021. https://diku.no/rapporter/digital-tilstand-2021-stoettemiljoeers-arbeid-med-digitalisering-og-utdanningskvalitet-ved-universiteter-og-hoeyskoler

Øygarden, K.F. and Stensby, B.R. (2022). Studiebarometeret for fagskolestudenter 2022 – Hovedtendenser. [Student survey for vocational college students 2022 – Main trends]. Norwegian Agency for Quality Assurance in Education (NOKUT), Report 16-2022. https://www.nokut.no/globalassets/studiebarometeret/2022/fagskole/studiebarometeret-for-fagskoler_2022_hovedtendenser_16-2022.pdf

Aas, C.V. (2023). Comments to the Expert Group on Learning Analytics in primary and secondary education and training (01 March 2023)

Fotnoter

1)

https://www.ssb.no/utdanning

2)

Sjöberg, F. (2011) Lyckad skövling i ny natur [Successful Destruction in New Nature]. Den utbrände kronofogden som fann lyckan [The Burned Out Debt Collector Who Found Happiness]. Nye Doxa Förlag

3)

https://laringsanalyse.no/

4)

https://kvalitetsutviklingsutvalget.no/mandat/

5)

https://www.uis.no/nb/skole/grunndig-digitalisering-i-grunnopplaering-kunnskaper-trender-og-framtidig-forskningsbehov

6)

https://www.hiof.no/lusp/pil/english/research/projects/ai4afl/index.html

7) https://www.uis.no/en/research/gameplay

8)

https://www.uv.uio.no/ils/english/research/projects/lat/index.html

9)

https://www.hiof.no/lusp/om/aktuelt/aktuelle-saker/2022/stort-forskningsprosjekt-skal-forhindre-uetisk-bru.html

10)

https://www.uib.no/ai/161820/stort-l%C3%B8ft-ai-forskning-ved-uib

11)

Regulations of 23 June 2006 No. 724 to the Education Act

12)

https://ndla.no/

13)

https://snl.no/data

14)

https://www.feide.no/

15)

https://learningpool.com/solutions/learning-locker-community-overview/

16)

https://xapi.com/

17)

https://www.imsglobal.org/activity/caliper

18)

Inter alia PVN-2022-02 (Erasure of internet search engine results) and PVN-2020-05 (Erasure of personal data in pupil folder)

19)

Regulations of 1 February 2010 No. 96 relating to quality assurance and quality development in higher education and tertiary vocational education (Regulations on the quality of programmes of study)

20)

Regulations of 7 February 2017 No. 137 relating to the supervision of the quality of education in higher education (Academic Supervision Regulations)

21)

Regulations of 11 July 2019 No. 1005 relating to tertiary vocational education (Vocational Education Regulations)

22)

Regulations of 23 April 2020 No. 853 relating to accreditation and supervision of tertiary vocational education (Vocational Education Academic Supervision Regulations)

23)

Self-regulation involves the ability to plan, implement and monitor one’s own learning and assess the extent to which one must change something to achieve a goal (Hopfenbeck, 2011; Pintrich, 2002; Winne, 2015).

24)

https://www.regjeringen.no/no/tema/utenrikssaker/utviklingssamarbeid/bkm_agenda2030/id2510974/

25)

https://www.fellesstudentsystem.no/

26)

https://www.studiebarometeret.no/

27)

https://ndla.no/

28)

cf. Regulations to the Education Act, section 3-15, second paragraph.

29)

https://www.udir.no/tall-og-forskning/brukerundersokelser/elevundersokelsen/

30)

https://www.udir.no/tall-og-forskning/statistikk/elevundersokelsen/

31)

https://www.studiebarometeret.no/no/artikkel/2

32)

https://www.studiebarometeret.no/no/artikkel/6

33)

https://www.datatilsynet.no/rettigheter-og-plikter/personvernprinsippene/

34)

See, e.g., section 32 of the Act relating to medical and health research (Health Research Act) and section 5-11 of the Act relating to tax administration (Tax Administration Act).

35)

https://www.ehelse.no/normen

36)

https://www.ks.no/fagomrader/digitalisering/felleslosninger/skolesec/

37)

https://www.udir.no/regelverk-og-tilsyn/personvern-for-barnehage-og-skole/

38)

https://www.datatilsynet.no/rettigheter-og-plikter/virksomhetenes-plikter/protokoll-over-behandlingsaktiviteter/

39)

https://www.udir.no/kvalitet-og-kompetanse/laremidler/kvalitetskriterier-for-laremidler/https://www.udir.no/regelverk-og-tilsyn/personvern-for-barnehage-og-skole/veiledere/

40)

https://www.ehelse.no/normen

41)

https://www.udir.no/kvalitet-og-kompetanse/nasjonale-satsinger/den-teknologiske-skolesekken/

42)

https://www.udir.no/kvalitet-og-kompetanse/laremidler/kvalitetskriterier-for-laremidler/

43)

Explainable AI.

44)

https://www.udir.no/kvalitet-og-kompetanse/profesjonsfaglig-digital-kompetanse/rammeverk-larerens-profesjonsfaglige-digitale-komp/

45)

Data Literacy.

46)

Regulations of 7 June 2016 No. 860 relating to the Framework Plan for primary and lower secondary teacher training for grades 1-7 and the Regulations of 7 June 2016 No. 861 relating to the Framework Plan for primary and lower secondary teacher training for grades 5-10

47)

Regulations of 18 March 2013 No. 288 relating to the Framework Plan for graduate teacher training for grades 8-13

48)

https://www.udir.no/kvalitet-og-kompetanse/kompetansepakker/

49)

https://www.tudublin.ie/media/website/explore/about-the-university/academic-affairs/quality-framework/blanch-qa/2MP46-Learning-Analytics-Policy-and-Strategy.pdf

50)

https://www.ed.ac.uk/files/atoms/files/learninganalyticspolicy.pdf

51)

https://www.aalto.fi/en/aalto-university/learning-analytics-policy-in-aalto-university

52)

https://www.datatilsynet.no/rettigheter-og-plikter/virksomhetenes-plikter/protokoll-over-behandlingsaktiviteter/

53)

https://www.uhr.no/strategiske-enheter/

54)

Regulations of 9 February 2006 No. 129 concerning appointment and promotion to teaching and research posts.

55)

https://www.uhr.no/temasider/karrierepolitikk-og-merittering/nasjonale-veiledende-retningslinjer-for-uh-pedagogisk-basiskompetanse/

56)

https://hub.teachingandlearning.ie/resource/developing-learning-analytics-policies-to-support-student-success/

57)

https://www.regjeringen.no/no/dokumenter/framtid-fornyelse-og-digitalisering/id2568347/