Newcastle University

Pre Requisite Comment

This module extends the area of Machine Learning in the specific area of Deep Learning. Much of the pre-requisite module will be required to follow this module.

Co Requisite Comment

N/A

Aims

Deep Learning is a sub-field within the area of Machine Learning which has attracted significant interest during recent years most notably because of its ability to outperform humans on many tasks which previously were considered to be too hard to perform on a computer. Deep Learning is all around us and used on a daily basis. If you talk to your phone or a home assistant, it’s Deep Learning which is converting your speech into text. If you search for friends on social media by giving a picture of them, then this is using Deep Learning. Even the adverts you see when surfing the web are likely to have been chosen for you by Deep Learning. Deep Learning is also going to be one of the key technologies for future developments such as self-driving cars and autonomous robots.
This module will introduce students to the area of Deep Learning, the different technologies available along with the myriad of application areas where it can be applied. The aim is to make students fluent in the approach of Deep Learning which will allow them to build up skills that will enable them to apply these skills for applications in companies, academia or the third sector.
Deep Learning is a powerful tool which, like all powerful tools, be used for good or bad. Hence a running theme within this module will be the ethical use of Deep Learning along with ensuring that students are aware of the biases which can be present and approaches to minimising such bias. Students will be made aware of good practices for Deep Learning and how to report their results in a fair and honest manner.

Outline Of Syllabus

Material will cover such topics as:
•       Multi-Level Perceptrons (MLP)
•       Backpropagation, gradient decent and optimisation
•       Convolutional Neural Networks (CNN) and their various applications in Computer Vision.
•       Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU)
•       Natural Language Processing (NLP)
•       Generative Models, e.g. Generative Adversarial Networks (GAN), and Variational Autoencoders (VAE)
•       Better training of neural networks, e.g. Preventing Overfitting, Dropout, Transfer Learning, Data
Augmentation
•       Ethics and Challenges of Deep Learning, Adversarial Examples, and good practices in Deep Learning
•       Interoperability and Explainability of Deep Learning
•       Deep Reinforcement Learning
•       Embedding
•       Deep Learning on Graphs, GraphCNN

Intended Knowledge Outcomes

The student should:
•       Be aware of the commonly used techniques in Deep Learning
•       Have a working knowledge of which techniques are useful for a particular problem
•       Possess the knowledge of how the different techniques are implemented in common frameworks
•       Have an appreciation of the ethical and moral obligations required when performing Deep Learning.

Intended Skill Outcomes

The student should:
•       Be able to understand the concepts behind deep learning
•       Implement a Deep Learning network within a common framework
•       Discuss and communicate the ability and limitations of a deep learning approach
•       Identify the most appropriate deep learning solution for a given problem
•       Be able to critically evaluate and synthesise research findings in DL and translate into own context
•       Be able to investigate and devise efficient and effective DL architectures, to maximise the impact of DL
solutions
•       Be able to assess risks/limitations associated with applications of DL
•       Be able to identify and quantify sources of bias and use appropriate mechanisms to reduce bias.

Teaching Rationale And Relationship

Lectures explain the underpinning principles for the module and technologies that support deep learning and exploratory data analysis. Lectures are complemented by supervised practical sessions to guide the application of these principles using suitable computational tools. The practical work builds up experience working with a computational toolset that is used to complete a substantive project working with data from a real-world context.

Assessment Rationale And Relationship

The report tests the students’ ability to apply deep learning techniques in a reproducible manner, using effective tools and methods to solve a real-world challenge. The presentation assesses the students’ ability to communicate their findings and approach. The formative assessment facilitates a reflective learning.

General Notes

N/A