Evaluation for equity and justice in higher education The Political, the Relational, and the Transformational
Main Article Content
Abstract
Universities have a long history of “widening participation”, attempting to advance social equity and social mobility through access and success within higher education. In England, evaluation has been central to the widening participation agenda for over twenty years, so much so that it has become systematised within higher education providers and is intertwined with the development and delivery of programmes and activities. In this system, normalised evaluation of practice valuing technical expertise, linear thinking, and rationality is dominant. These systems typically focus on questions such as “Did this intervention work?” However, socio-political influences prevail, particularly in systems driven by policy agendas, which can contribute to symbolic and non-use of evaluation that maintains the status quo, inhibiting our ability to advance social equity. In these contexts, it is important to make explicit the value judgments decision-makers hold that affect how higher education providers enact WP policy and its evaluation in practice, so that we can begin to consider alternative perspectives and strategies for change. This paper builds a conceptual framework drawing from political science, organisational theory, and systems thinking to develop our understanding of the widening participation system, including evaluation within the system...
Article Details
References
Alkin MC & King J 2017 ‘Definitions of Evaluation Use and Misuse, Evaluation Influence, and Factors Affecting Use.’ American Journal of Evaluation 38(3) pp. 434–450. DOI: 10.1177/1098214017717015
Andersen NA & Pattyn V 2025 ‘The organisation of evaluations: the influence of the ministry of finance on evaluation systems.’ Evidence & Policy pp. 1–23. DOI: https://doi.org/10.1332/17442648Y2025D000000045
Anjum RL, Price C & Rocca E 2025 ‘When Patient Voices Get Lost in Evidence Hierarchies: A Testimony of Rare Adverse Events and Participatory Epistemic Injustice in Drug Safety Monitoring.’ Social Epistemology 39(2) pp. 187–201. DOI: https://doi.org/10.1080/02691728.2024.2400079
Anthony A 2022 Is attainment raising about moral obligation or regulatory requirement? Available at: https://wonkhe.com/blogs/is-attainment-raising-about-moral-obligation-or-regulatory-requirement/ (accessed 7 August 2025).
Arieli S, Sagiv L & Roccas S 2020 ‘Values at Work: The Impact of Personal Values in Organisations.’ Applied Psychology: An International Review 69(2) pp. 230–275. DOI: 10.1111/apps.12181
Austen L 2020 ‘The amplification of student voices via institutional research and evaluation.’ In Lowe T and El Hakim Y (Eds.) A Handbook for Student Engagement in Higher Education: Theory into Practice. London: Routledge, pp. 164–176. Available at: https://doi-org.libproxy.mit.edu/10.4324/9780429023033 (accessed 7 August 2025).
Baker S, Brown B & Fazey JA 2006 ‘Individualization in the Widening Participation Debate.’ London Review of Education 4(2) pp. 169–182. DOI: 10.1080/14748460600855302
Befani B, Barnett C and Stern E 2014 Introduction - Rethinking Impact Evaluation for Development. IDS Bulletin 44(6). IDS. Available at: https://doi.org/10.1111/1759-5436.12108 (accessed 7 August 2025).
Boliver V, Banerjee P, Gorard S & Powell, M 2022 ‘Reconceptualising fair access to highly academically selective universities.’ Higher Education 84 pp. 85–100. DOI: https://doi.org/10.1007/s10734-021-00755-y
Burke PJ 2016 ‘Access to and Widening Participation in Higher Education.’ Encyclopedia of International Higher Education Systems and Institutions. DOI: 10.1007/978-94-017-9553-1_47-1
Burke PJ 2018 ‘Re/imagining widening participation: A praxis-based framework.’ International Studies in Widening Participation 5(1) pp. 10–20. ISSN: 2203-8841
Cartwright N 2007 ‘Are RCTs the Gold Standard? BioSocieties.’ 2(1). Special Issue: The Construction and Governance of Randomised Control Trials:, pp. 11–20. DOI: https://doi.org/10.1017/S1745855207005029
Clements N & Short M 2020 ‘Becoming not Being: using the rhizome in widening participation evaluation.’ Widening Participation and Lifelong Learning 22(2): 246–254. DOI: 10.5456/WPLL.22.2.246
Cousins JB & Whitmore E 1998 ‘Framing participatory evaluation.’ New Directions for Evaluation 1998(80) pp. 5–23. DOI: 10.1002/ev.1114
Crawford C, Dytham S & Naylor R 2017 The Evaluation of the Impact of Outreach: Proposed Standards of Evaluation Practice and Associated Guidance. Office for Fair Access. Available at: https://pure.northampton.ac.uk/ws/files/6246443/Crawford_Claire_UoN_2017_The_Evaluation_of_the_Impact_of_Outreach.pdf.
Cunningham SW & Hermans LM 2018 Actor and Strategy Models: Practical Applications and Step-Wise Approaches. First. Hoboken, New Jersey: John Wiley & Sons, Inc.
Dahler-Larsen P 2012 The Evaluation Society. Stanford, CA.: Stanford University Press.
Dahler-Larsen P 2015 The Evaluation Society: Critique, Contestability and Skepticism. In: Guglielminetti E and Regina L (Eds.) Spaziofilosofico.
Gates EF 2017 ‘Toward Valuing with Critical Systems Heuristics.’ American Journal of Evaluation 39(2) pp. 201–220. DOI: 10.1177/1098214017703703
Harrison N & Waller R 2017 ‘Evaluating outreach activities: overcoming challenges through a realist ‘small steps’ approach.’ Perspectives: Policy and Practice in Higher Education 21(2–3) pp. 81–87. DOI: 10.1080/3603108.2016.1256353
Hayton A & Bengry-Howell A 2016 ‘Theory, evaluation, and practice in widening participation: A framework approach to assessing impact.’ London Review of Education 14(3) pp. 41–53. DOI: 10.18546/LRE.14.3.04
House ER & Howe KR 2000 ‘Deliberative democratic evaluation in practice.’ In Evaluation Models: Viewpoints on Education and Human Services Evaluation. Dordrecht, Netherlands: Springer, pp. 409–421.
Ingram N & Gamsu S 2022 ‘Talking the Talk of Social Mobility: The Political Performance of a Misguided Agenda.’ Sociological Research Online 00(0) pp. 1–18. DOI: 10.1177/13607804211055493
Kelly C 2024 ‘Revisiting values in evaluation: exploring the role of values in shaping evaluation practices and their influences on decision-making within English higher education providers.’ Higher Education. Epub ahead of print 2024. DOI: https://doi.org/10.4324/9781003163954-25.
Leeuw FL & Furubo JE 2008 ‘Evaluation Systems: What Are They and Why Study Them?’ Evaluation 14(2) pp. 157–169. DOI: 10.1177/1356389007087537
Leeuw FL & Pleger LE 2023 ‘Evaluation Capture, Evaluator Resilience, and the Need for Competencies of Evaluators.’ Journal of MultiDisciplinary Evaluation 19(46) pp. 46–53. ISSN: 1556-8180
Maguire M, Braun A and Ball SJ 2015 ‘Where you stand depends on where you sit’: the social construction of policy enactments in the (English) secondary school. Discourse 36(4) pp. 485–499. https://discovery.ucl.ac.uk/id/eprint/1475199/8/Braun_1475199_FIN%20Discourse%20-
Maguire%2C%20Braun%20%26%20Ball%202015.pdf (accessed 7 December 2025)
McCaig C 2011 ‘Access agreements, widening participation and market positionality: enabling student choice?’ In: Molesworth M, Nixon E, and Scullion R (Eds.) The Marketisation of Higher Education and the Student as Consumer. Routledge, pp. 115–129. ISBN: 978-0-415-58445-6
Midgley G 1997 ‘Dealing with coercion: Critical Systems Heuristics and beyond.’ Systems Practice 10 pp. 37–57. https://doi.org/10.1007/BF02557850
Moores E, Summers RJ, Horton M, Woodfield, L, Austen, L & Crockford, J 2023 ‘Evaluation of access and participation plans: Understanding what works.’ Frontiers in Education 8(2023). DOI: 10.3389/feduc.2023.1002934
OfS 2019 Using standards of evidence to evaluate impact of outreach. Office for Students. Available at: https://www.officeforstudents.org.uk/media/f2424bc6-38d5-446c-881e-f4f54b73c2bc/using-standards-of-evidence-to-evaluate-impact-of-outreach.pdf (accessed 23 March 2019).
OfS 2023a Regulatory advice 6: How to prepare an access and participation plan. OfS 2023.66. Available at: https://www.officeforstudents.org.uk/media/mesjucep/regulatory-advice-6-how-to-prepare-an-access-and-participation-plan-may-2025.pdf (accessed 7 August 2025).
OfS 2023b Regulatory notice 1: Access and participation plan guidance. OfS 2023.15. Available at: https://www.officeforstudents.org.uk/media/12221897-d0d7-4f37-9c6d-4197db178cfd/regulatory-notice-1-access-and-participation-plan-guidance-march-2023.pdf.
OfS 2024a Key performance measure 5: Access to higher education. Available at: https://www.officeforstudents.org.uk/about/how-we-are-run/key-performance-measures/kpm-5-access-to-higher-education/ (accessed 7 July 2025).
OfS 2024b Key performance measure 6: Success and progression. Available at: https://www.officeforstudents.org.uk/about/how-we-are-run/key-performance-measures/kpm-6-success-and-progression/ (accessed 7 July 2025).
Raimondo E & Leeuw FL 2021 ‘Evaluation Systems and Bureaucratic Capture: Locked in the System and Potential Avenues for Change.’ In
Perrin B and Tyrrell T (Eds.) Changing Bureaucracies: Adapting to Uncertainty, and How Evaluation Can Help. New York, NY: Routledge. ISBN: 978-1-003-10058-4
Rainford J 2021 ‘Working with/in institutions: how policy enactment in widening participation is shaped through practitioners’ experience.’ British Journal of Sociology of Education: 1–17. DOI: 10.1080/01425692.2020.1865130
Reynolds M 2007 ‘Evaluation based on critical systems heuristics.’ In Williams B and Imam I (eds) Using Systems Concepts in Evaluation: An Expert Anthology. Point Reyes, CA.: EdgePress, pp. 101–122. ISBN: 978-0-918528-22-3
Sagiv L, Roccas S, Cieciuch J, Schwartz, S.H 2017 ‘Personal values in human life.’ Nature Human Behaviour 1 pp. 630–639. DOI: https://doi.org/10.1111/gove.12677
Schulte J & Aston T 2025 ‘Towards principled adequacy for purpose in choosing evaluation methods.’ 10.21953/lse.go7zknl0dmdw. LSE Research Online. Available at: https://eprints.lse.ac.uk/128322/ (accessed 7 August 2025).
Schwandt TA 2015 Evaluation Foundations Revisited: Cultivating the Life of the Mind for Practice. Stanford, CA.: Stanford University Press. ISBN: 978-0-8047-8655-3
Schwandt TA & Gates EF 2021 Evaluating and Valuing in Social Research. New York, NY: The Guildford Press. ISBN: 9781462547326
Spohrer K, Stahl G & Bowers-Brown T 2018 ‘Constituting neoliberal subjects? ‘Aspiration’ as technology of government in UK policy discourse.’ Journal of Education Policy 33(3) pp. 327–342. DOI: 10.1080/02680939.2017.1336573
Stevenson J, Clegg S & Lefever R 2010 ‘The discourse of widening participation and its critics: an institutional case study.’ London Review of Education 8(2) pp. 105–115. DOI: 10.1080/14748460.2010.487328
Stickl Haugen JS & Chouinard JA 2019 ‘Transparent, Translucent, Opaque: Exploring the Dimensions of Power in Culturally Responsive Evaluation Contexts.’ American Journal of Evaluation 40(3) pp. 376–394. DOI: 10.1177/1098214018796342
TASO 2022 ‘Typology of attainment-raising activities conducted by HEPs: Rapid Evidence Review.’ Transforming Access and Student Outcomes in Higher Education. Available at: https://cdn.taso.org.uk/wp-content/uploads/2022-06_Attainment-raising-typology-and-rapid-evidence-review_TASO.pdf (accessed 7 August 2025).
TASO Annual Conference 2022 - Part 1 2022 YouTube. Available at: https://www.youtube.com/watch?v=PS7imCzzKZQ (accessed 7 August 2025).
Ulrich W 1983 Critical Heuristics of Social Planning: A New Approach to Practical Philosophy. Bern, Switzerland: Haupt. ISBN: 978-0-471-95345-6
Ulrich W 2005 A brief introduction to critical systems heuristics (CSH). Available at: http://projects.kmi.open.ac.uk/ecosensus/publications/ulrich_csh_intro.pdf (accessed 7 August 2025).
Ulrich W & Reynolds M 2020 ‘ Critical Systems Heuristics: The Idea and Practice of Boundary Critique.’ In Reynolds M and Holwell S (eds) Systems Approaches to Making Change: A Practice Guide. 2nd edn. London: Open University and Springer, pp. 255–305. Available at: https://oro.open.ac.uk/70336/.
Weick KE, Sutcliffe KM & Obstfeld D 2005 ‘Organizing and the Process of Sensemaking.’ Organization Science 16(4) pp. 409–421. DOI: 10.1287/orsc.1050.0133
Weiss CH 1993 ‘Where politics and evaluation research meet.’ Evaluation 1(2) pp. 37–45. DOI: 10.1177/109821409301400119
Wisker G & Masika R 2017 ‘Creating a positive environment for widening participation: a taxonomy for socially just higher education policy and practice.’ Higher Education Review 49(2) pp. 56–84. ISSN: 0018-1609
Younger K, Gascoine L, Menzies V & Togerson, C 2018 ‘A systematic review of evidence on the effectiveness of interventions and strategies for widening participation in higher education.’ Journal of Further and Higher Education. Epub ahead of print 2018. DOI: 10.1080/0309877X.2017.1404558.