School of Social and Political Science

The University of Edinburgh

Blogs

The use of evaluation in six Norwegian directories

Posted on by

Øyunn Syrstad Høydal, PhD candidate, Oslo and Akershus University College of Applied Sciences (HIOA)

This blog post informs a talk at the SKAPE seminar on 13 December 2017

10 years ago, I started work in the communication department of a Norwegian directorate. My background was from the private sector and one of the first things that caught my attention was all the fuss about evaluations. Evaluations were presented by the leadership as some kind of new magic medicine: providing knowledge, educating the organization and its partners, changing policies and making the world a better place. All this got me interested in the phenomenon of evaluation.

At present, I am doing a PhD project in the field of evaluation research at Oslo and Akershus University College of Applied . Allthough evaluation is considered an important source of knowledge in the Norwegian political administrative system, hardly any research is done on the public use of evaluations and the consequences of an increase in evaluation activity(Halvorsen & Lier Madsen 2013:21; Askim, Døving & Johnsen 2013:73; Sverdrup 2014:20). This general lack of research led me to do a survey about the use of evaluation in six Norwegian directorates, as well as interviews with civil servants. My PhD is paper based, and when finished it will consist of three papers and a theoretical document binding it all together.

First, a few words about the Norwegian field of evaluation: the evaluation agenda made significant impact on the Norwegian public sector from the late 1980´s onward and was part of an overall shift towards a New Public Management inspired change in the bureaucracy, including  decentralization of public tasks and responsibilities and an associated system of performance measurement, evaluation and control. Even though evaluations of public policy have been performed in impressive numbers ever since, there is nothing like a Norwegian profession of evaluators. Researchers from research institutes or universities mainly produce evaluations used by the bureaucracy (Dahler-Larsen 2013). Some of the research institutes have produced knowledge for public sector bureaucracy since the 196os, and it is often hard to find major differences between what used to be called research reports and the evaluations of today (Kvitastein 2002). In my project, I am therefore not making any clear distinctions between evaluation and applied research – looking at the influence on policy from this kind of knowledge.

My first paper is theoretical. Using perspectives from the field of evaluation research, I argue for a dual perspective in the discussion concerning evaluation influence. Theories of influence (Kirkhart 2000, Henry & Mark 2003) are combined with the idea of The Evaluation Society (Peter Dahler-Larsen, 2012) which together create a broad perspective on the use of evaluations and their possible impact. From this dual perspective, evaluation is recognized both as an instrument for planning and development and as a societal phenomenon. The Norwegian participation in The Programme for International Student Assessment (PISA) is used to illustrate the essence of this double perspective and how it can yield a more complete understanding of the complexity of evaluation influence. The analytical framework from the theories of influence makes the extensive influence from the PISA tests on different levels of the Norwegian society (individual, interpersonal and collective) visible and more easy to pinpoint. The idea about the evaluation society reveals how the values and norms of society leave their marks on the use of evaluation. Thus, Norway’s school reforms could be seen as an expression of the evaluation society, rather than an attempt to lift low-performing students.

My second paper addresses the well-establised debate about whether knowledge from research is used instrumentally by the political administrative system and has any direct consequences for decisions taken. The paper is based on the data from my survey, and the findings indicate that the evaluation results do have direct consequences in the decision-making process. This finding is explained both by the role of evaluation as an incorporated administrative tool (Dahler-Larsen 2000) and the co-production (Jasanoff 2004) of knowledge that takes place between the political-administrative system and the external knowledge producers. I also refer to Weiss (1998:23-24) and the factors she claims contribute to the instrumental use of knowledge. According to Weiss, instrumental use is common if the implications of the findings from evaluations are relatively non-controversial, small-scale and within the programs’ existing repertoire. I believe both the role of evaluation as an everyday tool for public sector bureaucracy and the process of co-production is leading to this kind of non-controversial and manageable evaluation results. From this perspective, the evaluations do seem to have direct implications for the political decision process.

The third and final paper is still work in progress and explores the enlightenment use of research, or more precisely the process of enlightenment. The paper will be based on data from the survey I designed and conducted and will draw relevant theoretical perspectives. I am exploring  literature in social theories of learning as well as perspectives on discourses and empirical research on the use of knowledge in organizations.

Visiting the University of Edinburgh and the School of Social and Political Science this term and having the opportunity to attend and speak at the SKAPE seminars is a great experience. I look forward to learning and being inspired to bring new insights into my PhD project.

Comments are closed.

Categories