Jiawei Yan
Maintaining privacy in human behaviour analysis.
Email: j.yan11@ncl.ac.uk
Project title
Contextual information for action recognition and behaviour analysis
Supervisors
Project description
We are investigating methods of human behaviour analysis. We are targeting:
- detection and localisation
- contextual information
- 2D pose features fusion
- multimodal processing
Methodology and objectives
To recognise human action, there are two main aspects to consider, which are target data and contextual data. Target data refers to those frame pixels which describe the human body. In contrast, contextual data refers to every other pixel which describes the background and the visible objects.
We have implemented Mask-RCNN to provide semantic segmentation between background and human targets. This provides high-quality segmentation masks of detected objects.
We use Mask-RCNN to pre-process the data. We then inject frames into the state-of-the-art Inflated 3D ConvNet (I3D) network for human action recognition.
Result
We propose a method to preserve the target privacy for those applications where privacy-protection is an issue. We have based the method on an image segmentation mask to occlude the target-related data.
We have demonstrated that the resulting network performs human action recognition with similar performance when compared to the baseline. Our research has highlighted the importance played by contextual information. The information is crucial in providing high human action recognition performance.
Interests
Football.
Qualifications
- MSc in Automation and Control from Newcastle University