DSC3005 : Reinforcement Learning and Autonomous Systems
- Inactive for Year: 2026/27
- Module Leader(s): Professor Nick Wright
- Lecturer: Dr Chris Holder, Professor Wei Pan
- Owning School: Engineering
- Teaching Location: Newcastle City Campus
Semesters
Your programme is made up of credits, the total differs on programme to programme.
| Semester 1 Credit Value: | 20 |
| ECTS Credits: | 10.0 |
| European Credit Transfer System | |
Aims
This module develops your understanding of how intelligent systems make decisions and act within dynamic and interactive environments. You will study approaches to sequential decision making, including reinforcement learning, deep reinforcement learning, imitation learning, and world models, alongside methods through which systems learn from interaction, feedback, and experience. The emphasis is on understanding how intelligent behaviour emerges over time, evaluating the performance and limitations of decision-making systems, and applying these approaches within practical and real-world contexts.
Outline Of Syllabus
Topics covered by this module include:
Principles of reinforcement learning and sequential decision making
Value-based reinforcement learning methods, including Q-learning
Policy-based reinforcement learning approaches
Deep reinforcement learning architectures and methods
Soft actor–critic and related optimisation approaches
World models and model-based reinforcement learning
Imitation learning and learning from demonstrations
Applications of reinforcement learning within robotics and autonomous systems
Humanoid robot control and embodied learning systems
Principles and operational characteristics of large language models
Applications and emerging uses of large language models within intelligent systems
Teaching Methods
Teaching Activities
| Category | Activity | Number | Length | Student Hours | Comment |
|---|---|---|---|---|---|
| Guided Independent Study | Assessment preparation and completion | 1 | 36:30 | 36:30 | Preparation and completion of summative coursework |
| Structured Guided Learning | Lecture materials | 1 | 20:00 | 20:00 | Reading through lecture materials – preparing / follow up |
| Guided Independent Study | Assessment preparation and completion | 1 | 1:30 | 1:30 | Completion of summative exam |
| Guided Independent Study | Assessment preparation and completion | 1 | 60:00 | 60:00 | Preparation for summative exam |
| Scheduled Learning And Teaching Activities | Lecture | 21 | 1:00 | 21:00 | Lectures |
| Guided Independent Study | Assessment preparation and completion | 1 | 1:00 | 1:00 | Formative assessment (Canvas quiz) completion |
| Guided Independent Study | Assessment preparation and completion | 1 | 20:00 | 20:00 | Preparation for formative assessment |
| Guided Independent Study | Directed research and reading | 1 | 20:00 | 20:00 | Wider reading as per provided reading list |
| Scheduled Learning And Teaching Activities | Practical | 10 | 1:00 | 10:00 | Supervised practical lab sessions focused on hands on implementation |
| Scheduled Learning And Teaching Activities | Practical | 10 | 1:00 | 10:00 | Supervised tutorial classes on lecture material |
| Total | 200:00 |
Teaching Rationale And Relationship
This module includes an encounter with the leading edge of research and practice.
Assessment Methods
The format of resits will be determined by the Board of Examiners
Exams
| Description | Length | Semester | When Set | Percentage | Comment |
|---|---|---|---|---|---|
| Written Examination | 90 | 1 | A | 80 | N/A |
Other Assessment
| Description | Semester | When Set | Percentage | Comment |
|---|---|---|---|---|
| Report | 1 | M | 20 | Students to complete an assessed technical task (model development using python) and provide a short report (1000 words) |
Zero Weighted Pass/Fail Assessments
| Description | When Set | Comment |
|---|---|---|
| Written exercise | M | Canvas quiz-based assessment of one hour |
Assessment Rationale And Relationship
The written examination assesses students’ understanding of the principles, methods, and behaviour of reinforcement learning, sequential decision making, and language-model-based intelligent systems within controlled conditions. The examination evaluates students’ ability to reason about learning strategies, model behaviour, optimisation approaches, performance, stability, and the suitability of different approaches for practical and real-world applications. The examination assesses all module learning outcomes. (MLO1, MLO2, MLO3, MLO4, MLO5)
The report provides students with the opportunity to apply reinforcement learning or language-model-based approaches within a practical computational setting. Students will complete a technical implementation task involving model development and evaluation using appropriate computational tools and communicate their methods, results, and analysis through a structured technical report. The assessment supports the development of practical skills in model implementation, evaluation, and critical interpretation of system behaviour and performance. (MLO1, MLO2, MLO3, MLO4, MLO5)
The formative written exercise provides students with opportunities to practise conceptual understanding and applied problem solving associated with reinforcement learning and intelligent systems while receiving feedback in advance of the summative assessments.
The assessment structure supports the development of both conceptual understanding and applied technical skills, enabling students to analyse, evaluate, and implement intelligent decision-making systems within dynamic and interactive environments.
Alternative assessment:
For students with a Student Support Plan (SSP) who have alternative assessment as a reasonable adjustment and/or students with supported Personal Extenuating Circumstances (PEC), it may be necessary to set an alternative assessment as presented below. Due to the professional requirements of some programmes, alternative assessments may not be available even where students are eligible.
Where appropriate, alternative arrangements for the written examination may include an oral examination comprising a 15-minute prepared topic followed by 15 minutes of questions from the overall module syllabus.
Reading Lists
Timetable
- Timetable Website: www.ncl.ac.uk/timetable/
- DSC3005's Timetable