BigData is now a fundamental part of all decision making processes but it is far more complicated than the public perception.
IEEE Cloud Computing is a global initiative launched by IEEE to promote cloud computing, big data and related technologies, and to provide expertise and resources to individuals and enterprises involved in cloud computing.
A recent paper by Dr Rajiv Ranjan (Newcastle University School of Computing) and Philip James (Newcastle University School of Engineering) discussing how BigData is now a fundamental part of all decision making processes but that it is far more complicated than the public perception was selected by IEEE Cloud Computing as their feature article for October.
Ranjan and James write that “One of the myths is that BigData analysis is driven purely by the innovation of new data mining and machine learning algorithms. While innovation of new data mining and machine learning algorithms is critical, this is only one aspect of producing BigData analysis solutions”.
In the paper, Ranjan and James argue that, like many other software solutions, BigData analysis solutions are not monolithic pieces of software that are developed specifically for every application: “They often combine and reuse existing trusted software components that perform necessary data analysis steps, and take advantage of the elasticity of datacenter computation and storage resources to meet the requirements of their owners.”
The authors used Real-Time Flood Modelling, work coming from recent NERC-funded projects, as an example of a workflow that uses existing components including tweets and sensor data.
The workflow (below) is triggered from long-range forecasting - for example from UK Met Office DataPoint - and radar scans at multiple scales are initiated and passed to statistical processing models, updating probability based forecasts.
Ranjan, R., Garg, S., Khoskbar, A.R., Solaiman, E., James, P. and Georgakopoulos, D., 2017. Orchestrating BigData Analysis Workflows. IEEE Cloud Computing, 4(3), pp.20-28.
published on: 1 November 2017