Module Catalogue 2024/25

Pre Requisite Comment

N/A

Co Requisite Comment

N/A

Aims

The aim of the module is to introduce students to the complex combination of data engineering technology and data science that makes it possible to extract valuable knowledge from “Big Data”. A number of technical challenges are derived from the high volume and high diversity (heterogeneity of meaning and format) and variable quality of the data, and a distinction is made based on whether the data is stationary (resides in a data repository) or it is in motion (data streaming, as it would be produced for instance by sensors), with further emphasis on graph data structures.

The module will focus on the following aspects:
- Distribution of data processing over a cluster of computing nodes, hosted in a cloud environment, as a way to
scale out computing resources as the size of the data to be processed increases. This includes current
frameworks for massively parallel data processing, notably Spark which is the most successful example of
cloud-based distributed programming platform, and possibly Dask, its direct competitor.
- Examples of algorithms that can be successfully parallelised and thus are able to take advantage of
distributed data architectures
- Models of computation that enable near- real time analytics on data streams
- Specialised data structures, specifically graphs. The module covers basics of graph databases (Neo 4J) but
also massively parallel graph algorithms, i.e., implemented using the Pregel framework.
- Examples of data science applications, including Machine Learning algorithms that are enabled by Big Data
technology.

Emphasis is also placed on the rapid pace of technology advances in this area, and cutting-edge further reading material is offered for in-depth learning and deep-dives into specific topics.

Outline Of Syllabus

1.       Introduction to Data Science and Data Analytics. Scalability, efficiency of parallel processing.
2.       Batch Big Data Processing (MapReduce)
3.       Computing environments for Big Data Analytics and Machine Learning:
•       Big Data platforms (Hortonworks, Cloudera), Spark
4.       Data Stream processing: Overview of real time Event Processing and querying
5.       Graph data processing: Example of algorithms for graph analytics, graph databases and query languages (GDBMS), massively parallel graph processing model

Intended Knowledge Outcomes

You will:
- Learn fundamental notions of parallel data processing and scalability.
- Understand the challenges associated with processing different types of Big Data (batch, streaming, graph-structured).
- Learn fundamental concepts in data analytics: Exploratory (EDA) and Predictive (Machine Learning) with case studies in different application domains.
- Learn to take “stay on top” of cutting edge algorithms and architectures for Scalable Data Engineering, by discovering and reading selected research literature and providing a critical analysis.

Degree Apprenticeship standard:
K17: How key algorithms and models are applied in developing analytical solutions and how analytical solutions can deliver benefits to organisations.
K20:The properties of different data storage solutions, and the transmission, processing and analytics of data from an enterprise system perspective. Including the platform choices available for designing and implementing solutions for data storage, processing and analytics in different data scenarios.

Intended Skill Outcomes

- Learn to use practical computation environments for Big Data: Spark (massively parallel data processing) on the Cloud, and analytics workflows, with applications to specific analysis goals in diverse application domains.
- Develop problem-solving skills that are specific to Big Data Analytics.

Degree Apprenticeship standard:

S1: Identify and select the business data that needs to be collected and transitioned from a range of data systems; acquire, manage and process complex data sets, including large-scale and real-time data.
S22: Formulate analysis questions and hypotheses which are answerable given the data available and come to statistically sound conclusions.
S23: Conduct high-quality complex investigations, employing a range of analytical software, statistical modelling & machine learning techniques to make data driven decisions solve live commercial problems.

Teaching Rationale And Relationship

The learning experience is organized into two parts with roughly equal weight:
1.       Theory (50 hours). In turn this follows the paradigm: watch-study-engage. Lectures will be used to introduce the learning material and for demonstrating the key concepts by example. Selected lectures will be pre-recorded to enable the class to be “flipped” during scheduled lecture time. For these lectures, students will be expected to follow the recording ahead of time (Structured Guided Learning) and then engage in Q&A during online / PIP class time. Students are also expected to address specific topics in depth and independently (Directed research and reading) as part of this
2.       Practical programming. Workshops are offered to introduce the computational environment(s), as well as weekly Drop- in/Surgery hours to help solve practical problems. The bulk of the time for this part is for independent study and programming.

Assessment Rationale And Relationship

The assessment structure is designed to
-       promote a deep understanding of the lecture material through assessed exercises
-       encourage students to engage with one or more programming environments, which may be new to them, and develop practical problem-solving skills to address specific programming challenges

General Notes

N/A