Projects and Evaluations

A Theory of Change is an explanation of how outcomes are expected to happen. The process of developing a Theory of Change involves thinking about the causes of a specific problem, issue or situation and what needs to change, followed by an explanation of how any planned action or intervention is expected to lead to that change. We have a briefing document INKC Briefing - Developing theories of change for evaluation: An introduction (PDF: 887.9 KB) and an informative booklet Theory-based Methodology: Using ToC for development, research and evaluation (PDF: 2.2 MB) which outline how developing a Theory of Change works and why we believe it is one of the most effective ways of designing an intervention. Once a Theory of Change has been developed, it can be used to create an evaluation plan.

The INKC team has undertaken interviews, focus groups and workshops with staff and students across the University, and integrated learning from evaluations and research in order to co-produce an explanation of what issues Newcastle University faces in understanding what needs to change to help us widen access and strengthen equity in students' participation in the full entitlement we provide for them. More information about this work is available in our booklet A Framework for Action (theory of change) for an Inclusive University (PDF: 1.5MB).

This process is ongoing as we learn more and it is helping us to understand what kinds of intervention we need in our new 4 year Access and Participation Plan from 2025-2026. The Office for Students has stressed that each intervention proposed in our submission for the new APP (Spring 2024) needs its own Theory of Change, which will identify the aspects which need to be evaluated, and why; and provide a framework for an evaluation plan. The INKC is supporting teams across the University in using Theory of Change in the design and planning of the strategic interventions in the new APP.

To find out more, please contact Karen Laing.

Sense of belonging and community is a priority research area for INKC. We have embedded an increased focus on students' sense of belonging and community in our university Access and Participation Plan 2020-21 to 2024-25 (PDF: 607.9 KB). 

We have produced a literature review exploring sense of belonging definitions and interventions. This work has informed multiple presentations at conferences and sparked conversations in University Senate and across our faculties. The full literature review Sense of Belonging and Community in Higher Education Literature Review (0.4 MB) is published here.

We also have a summary report of our key findings: Sense of Belonging and Community - Key Insights (PDF: 218.7 KB) and a briefing document INKC Briefing - Key insights on sense of belonging (PDF: 209 KB). These key findings inform our interventions and underpin our support for other projects. Our work on evaluating the use of funding to support Participation Bursaries and the new Academic Societies Funding Scheme is part of our wider focus on facilitating a greater sense of belonging for students. 

Key insights about social and academic fit

Students' sense of belonging in the higher education world is made up of ‘social fit’ and ‘academic/ability fit’.  

Students gauge their own sense of belonging in terms of ‘social fit’ based on how well they fit in socially by considering: 

• Social integration 
• A feeling of fitting in and being part of an academic community  
• Feeling accepted and valued socially, as part of a cohort or identity group  
• Connection with peers   
• Connection with staff and mentors  
• Positive experience of engagement with the community, campus and University  
• Low levels of concern about negative stereotypes and biases about their ability to fit in socially.

 ‘Academic fit’ goes beyond social interaction and considers how a student feels they fit into the academic environment: 

• A feeling of fitting in and being part of an academic community  
• Feeling accepted and valued academically, as part of their course cohort  
• Positive experience of engagement with academia (the community, facilities and university)  
• Feeling capable and comparable in their abilities to their peers  
• Low levels of concern about negative stereotypes and biases about their academic abilities 
  

It is important that students feel they fit in both socially and academically, and both types of ‘fit’ should be regarded as being of equal importance to fostering a greater sense of belonging in the university; to some extent they are mutually dependent in the academic environment for students to be able to thrive and to feel comfortable that they can do so 'as theselves'. 

Academic and social fit can be understood as a Venn diagram where the overlap between academic and social fit is important for developing a strong sense of belonging. The academic school (programme, course or discipline) combines opportunities for social and academic fit, so is an important area to focus interventions. Insights from students confirm that their sense of belonging is rooted and nurtured first and foremost at programme/school level.

The concept of 'Community' is closely linked to Belonging, and the academic community is viewed as a group with shared interests and attitudes within the overall University environment. Feeling a lack of belonging to an academic community can cause students to 'opt out' and change disciplines to a community that feels more welcoming. 

References: Lewis and Hodges (2014).

Key insights about belonging interventions and studies

'Belonging interventions’ are used to test a planned impact on students' experiences and academic outcomes. ​ 

For example, a common early 'belonging intervention' involves a reading and writing exercise framed around older students describing their initial struggle to make the transition to university as being a normal, transient and surmountable experience. ​ 

There remains a limited number of UK-based literature on this topic, but studies in the USA strongly suggest that interventions have: 

• Increased the likelihood that racial-ethnic minority and first-generation students maintain continuous enrolment^3​ 
• Reduced the GPA attainment gap between African American and European American students by 79%^1​ 
• Improved self-reported health and wellbeing for African American students^1​ 
• Increased students' social and academic integration on campus^2​ 
• Led to a 9% increase in continuation and 31% reduction in 'disadvantaged' students' first year raw achievement gap^2​ 
• Improved overall university experience – reported more social networks and use of support services²
• Promoted greater feelings of social and academic fit one-year post-intervention^3​ 
• Created strong, sustained social networks for mature students^4​ 

References 

1. Walton & Cohen (2011). 2. Yeager et al. (2016). 3. Murphy (2020). 4. Mannay & Ward (2020)​.

To find out more, please contact Charlotte Boulton.

The widening access approach of using Contextual Admissions involves utilising information about an applicant’s context in considering their application (Mountford-Zimdars, Moore and Graham, 2016; WonkHE, 2017; Sutton Trust, 2019; UCAS, 2020).

At Newcastle University, contextual admissions is used as part of a university-wide, holistic approach to fair admissions. This approach aims to recognise the challenges that some applicants may have - and may continue - to face in their educational journey.

Contextual offers (CO) are one of several other possible contextual admissions pathways at Newcastle University. A CO is based on postcode (living in an area identified as having historically low numbers of people attending university) and care-experienced status.

Undertaken in 2022, this project involved a mixed-method evaluation of Contextual Offers as part of the evaluation of the Newcastle University Access and Participation Plan 2020-21 to 2024-25. Using a Theory of Change approach, aligned with the Office for Students' commitment to evaluation, we explored how contextual admissions work to:

• eliminate Newcastle University’s entry rate gaps between the most and least represented applicant groups to the University;
• develop positive interventions and produce positive outcomes for applicants/students.

We interviewed staff, analysed survey data, ran workshops with students and carried out statistical analysis of contextual offers. We examined applicant cohort data to understand which student groups were more likely:

• to be made and to accept offers;
• to continue to progress on their course;
• to complete their course with a good degree award (First or 2(i)).

Key research findings

This evaluation project presented multiple insights and questions for us to seek to learn more about:

• Our systematic data collection through the student lifecycle;
• The complexities in our offer/place allocation processes;
• Ways to develop more strategic, tailored and targeted information, advice and guidance that consistently aligns with [potential] applicants’ sense of belonging to university, to Newcastle and to their department and potential course;
• Opportunities to further broaden our outreach and partner with underrepresented and underserved communities.

We developed a Contextual Offer Making Infographic to visually demonstrate the impact of contextual offers on the student cohort and highlight some of our key findings and recommendations.

This work was led by Dr Krystal Douglas-Dodd.

The degree awarding gap is a persistent, sector-wide issue with white students, on average more likely to leave university with a first or upper-second class degree compared to Black and other Global Majority students. Taking a students’ background or prior attainment into account still doesn’t explain the awarding gap.

Here at Newcastle University (NU), there is also an unexplained gap for Black home undergraduate students. Reducing the degree awarding gap is a strategic priority in our Access and Participation Plan 2020-21 to 2024-25. Our target is to reduce the gap to 10 percentage points by 2024-25. In order to take measures to reduce this gap, work is needed better to understand its causes.

Working with the School of Engineering (SoE), the INKC therefore embarked on a participatory action research pilot project, centralising student voice and co-creation to understand the causes of the awarding gap and how it can potentially be narrowed.

A Student Advisory Board (SAB) consisting of students from the SoE were recruited who oversaw and co-created the research with INKC. Members of the SAB past and present who shared their expertise and knowledge include Kudzaishe Chiwara, Anna Christy, Mihir Dhonsale, Jonathan Lee, Micha Newman, Andy Oroke, Reaha Sanghera, and Kirijana Yogaratnam.

The SAB was trained in Community Organising and listening methodology by Tyne & Wear Citizens UK. NU is a founding and strategic partner of Tyne & Wear Citizens UK. The training supported the SAB to reflect on complex issues of power and influence and identify the power of their own voice. Regular workshops and cycles of dialogic and reflective interactions underpinned the project.

The pilot project involved a mixed methods approach to understand student experiences in the SoE and what could potentially be driving the awarding gap. The methods included peer-to-peer listening, a focus group with current Black home students, semi-structured interviews with current Black and other Global Majority students and Alumni, and a survey distributed to all SoE students in stages 2-4.

Key research findings

• Experiences of racism and microaggressions

Black and other Global Majority students in Engineering are experiencing incidents of racism and microaggressions. These happen both on and off campus. These experiences lead students to disengage from learning and avoid spaces and people that may expose them to further such incidents. Some students also experienced misidentification and mispronunciation of their names.

Students spoke about the ongoing trauma of these incidents and the effect on their health and wellbeing. Students are not aware of how to report their experiences and who to report them to. Staff are also largely unaware of reporting processes, meaning that they are not able to advise students appropriately.

• Experiences of gendered and racial discrimination

Black and other Global Majority female students experience both gendered and racial discrimination. They experience differential treatment, such as being ignored, excluded, and feeling bullied. These experiences lead these female students to doubt their own capabilities and have a negative impact on their sense of belonging. They also expressed concerns about their personal safety on campus.

• A lack of sense of belonging

There is a lack of representation at NU for Black home students. This results in students feeling hyper-visible and feeling isolated. A lack of sense of belonging can impact on confidence and how or if students engage socially or on the course. It can also impact on how students feel about their ethnic and cultural identity. In the survey Black home students were less likely to report feeling part of a community of staff and students. Students told us that societies can help to support a sense of belonging.

• The impact of cultural and intersectional lived experiences

Cultural and familial expectations can put an additional strain on Black home students. Some Black home students feel pressure from their families to succeed and feel these additional pressures are not well understood by NU. Some students were carers for family members, placing additional demands on them. Other Global Majority students also shared similar experiences.

• Mental health and wellbeing and student support

Black home students experience mental health and wellbeing challenges linked to the pressures of their course, cultural expectations, racism, microaggressions and a lack of sense of belonging. Black home students are unaware of mechanisms to access culturally competent support and often turn to family and friends. Some Black home students have accessed NU Student Health and Wellbeing Services and suggested a need to be culturally sensitive and understand cultural and familial expectations. Other Global Majority students also shared similar experiences.

• Interpersonal relationships and dialogue

Building supportive peer networks was noted as an important source of support. However, developing such networks can be challenging. It was felt that these networks are often more easily accessed by white students. Group work was suggested as a useful tool in helping students to build networks.  

• Curriculum, teaching and learning

Black and other Global Majority students value their relationships with academic staff and personal tutors but support can be inconsistent, varied and untimely. Cultural norms of respect can influence students interactions with academics and personal tutors. Not all students were aware they could access feedback on their assessments. In the survey Black home students were less likely than other students to feel they were treated equally on their course and be satisfied with their course. Students may not always be aware of the support and guidance available to them.

Recommendations

Recommendations for the SoE and the wider institution were co-created with the SAB. These include suggestions on how to:

• increase diversity and representation
• establish an anti-racist culture
• support transition and sense of belonging
• strengthen academic and personal tutor support

The research highlighted the value and benefits of student co-creation both for students and NU. Specific recommendations on supporting student co-creation include:

• investing staff time and resources to meaningful engagement
• building trusting relationships
• raising awareness of student co-creation projects and resulting actions

The Black Engineering Network

One of the key recommendations from the research was to establish a Black Engineering Network to support sense of belonging, peer networking and access, success and progression of Black home students.

The Network is open to membership from:

• SoE Black and other Global Majority current undergraduate and postgraduates
• Black and other Global Majority Alumni
• SoE current student allies
• SoE staff

The Network will be co-created with students and be responsive to student need.

The learning from this project and the Network itself is intended to inform wider work across the University to ensure greater equity for Black and Global Majority students.

The Network was launched on 12^th October 2023. 

To find out more about this project please contact Dr Amina Razak.

Participation Bursaries are funded by the Access and Participation Plan and delivered by NUSU. This bursary aims to help remove some of the barriers to extracurricular participation for under-represented students by covering some of the costs of getting involved in NUSU and University extracurricular activities.   

INKC have supported the 2022/23 evaluation of the Participation Bursaries by providing evaluation mentoring for the NUSU staff lead. The evaluation has used a Theory of Change approach and includes data from administration and surveys. The evaluation report NUSU Participation Bursary Evaluation Report 2022-23 (PDF: 2.1MB) is being used to make improvements to the bursaries for the academic year 2023/24. Evaluation, and support from INKC, will continue to assess the effectiveness of the changes made over 2023/24.

Key insights from the evaluation

• 96% of respondents agreed that their Participation Bursary was necessary to engage with NUSU opportunities while 83% of respondents agreed that the amount was sufficient.  
• 93% of respondents agreed that the Participation Bursary helps them to feel included   
• 87% of respondents agreed that the Participation Bursary helps them gain skills and knowledge  
• 96% of respondents agreed that the Participation Bursary helps them afford to participate   
• 95% of respondents agreed that the Participation Bursary helps improve their health or wellbeing with 52.3% of respondents using the bursary for sports-related activities  
• Students who were Postgraduate and/or International appear to be more likely to report that the PB had a positive impact on their feelings of inclusion at Newcastle University. 

To find out more, please contact Charlotte Boulton. 

The Academic Societies Funding Scheme is a new intervention delivered by NUSU. The scheme secured University funding in academic year 2022/23 to give academic societies the opportunity to carry out welcome events for new students, and to enable 6-weeks of free membership to these for new students. The scheme continued for the 2023/24 academic year following the same model.

INKC have supported both the 2022/23 and 2023/24 evaluation of the Academic Societies Funding Scheme by providing evaluation mentoring for a student evaluation intern. The evaluation has used a Theory of Change approach and includes data from focus groups and surveys.

The NUSU Academic Societies Funding Scheme Evaluation Report 2023-24 (PDF: 1.8MB) is the latest report detailing the experiences and opinions of current students and committee members. An article for higher education sector blog Wonkhe has also been released sharing some of these evaluation findings: Academic societies can make students’ time on campus more magical.

Evaluation, and support from INKC, will continue to assess the effectiveness of any changes made over 2024/25.

Key insights from the 2023/24 evaluation

• 64% of students (n=123) agreed with the statement ‘I feel a sense of belonging to my academic society’ (Strongly Agree and Agree). 
• 74% of students (n=123) agreed with the statement ‘I feel like attending welcome activities helped improve my sense of belonging to my academic society’ (Strongly Agree and Agree). 
• 68% of students (n=123) had already purchased their full society membership after the end of the free 6-week membership trial at the time of the survey one week later.  
• 76% of students (n=123) said the opportunity to try the society for free had affected their choice to join or not.  
• The evaluation suggests the scheme, and particularly the welcome events, are associated with students reporting a sense of belonging to their academic society. There is more to be done to understand how feelings of social and academic fit within the school environment could be influenced by this scheme and how sense of belonging may be felt to the school and to the university more broadly.

The NUSU Academic Societies Funding Evaluation Report 2022-23 (PDF: 1MB) was used to make improvements to the funding scheme for academic year 2023/24. 

Key insights from the 2022/23 evaluation

• The evaluation suggests that the funding received has been associated with the creation of a sense of belonging within academic societies, particularly from the perspective of academic society committees. Student committees commented that the funding helped to “connect members from across the year groups” and allowed members to “integrate and feel involved, therefore developing a sense of community within the society”. 
• 92% of committees that responded to the question “I would apply/encourage the application to this fund again for the 2023/24 academic year”, agreed or strongly agreed with the statement. 
• The committee survey shows that 70% of committees who responded (24 out of 41) believe their membership numbers have increased as a result of the funding. 

To find out more, please contact Charlotte Boulton.