Statistical Methods for Determining the Relationship Between Team Communication Dynamics & Task Performance

Location: Fishermans Bend, VIC

Duration: 5 months

Proposed start date: 1 July 2018

Keywords: Statistical Methods, Bayesian Modelling, Multidiscipline, Machine Learning, Change Point Detection

Please note: Due to the sensitivity and security of this project, students must have Australian Citizenship to apply. Any applicants not meeting this requirement will automatically be deemed ineligible for this project.

Project Background

In many Australian Defence Force (ADF) settings, teams of two or more operators, with distinct, but interdependent roles, are required to work together to achieve common goals under conditions characterised by significant time pressure and high cost for error.While, the effectiveness of these teams is dependent on a number of factors, effective communication between team members is recognised as vital for team coordination, planning, and execution of mission goals. Recent research has shown that the complexity of team communication (e.g., as operationalised by sample entropy) varies as a function of team workload and practice, and changes in the complexity of communication dynamics are associated with changes in task performance.

The Defence Science and Technology Group (DST) have applied the results of the above research to the training of air-battle management crew in the Royal Australian Air Force (RAAF) in the context of simulated exercises. Teams of eight or more trainee operators are exposed to high-fidelity simulated missions and assessed by instructors on their mission performance. DST has recorded sequences of communication events throughout these training missions, and is currently investigating whether the complexity of the communication sequences can assist assessors in identifying changes in team performance. If the temporal dynamics of team communication can be shown to be a robust and sensitive measure of team performance, then there is potential for these measures to assist instructors in identifying and assessing changes in team performance, both in real-time and during post-training feedback. Furthermore, there is potential for communication complexity measures to be used to facilitate teaming between humans and synthetic agents.

DST is seeking candidates with experience in statistical modelling to assist our researchers in identifying methods for assessing the relationship between communication complexity measures and task performance. Specifically, we require a PhD to (i) use Bayesian analyses of change points in time series of communication complexity (e.g., Sample Entropy) to identify changes in complexity, (ii) use machine learning to tune the hyper-parameters of the Bayesian Change Point model using training data, and (iii) measure the test error rates of the Bayesian models.

Research to be Conducted

Data from a simulation training exercise has already been recorded. Data-frames include time-stamped team communication sequences, communication complexity values, and performance measures.

The successful candidate will carry out the following research on the above data using the R statistical programming language:
1. Analyse communication complexity time-series to detect change points using Bayesian change point (BCP) methods.
2. Tune the hyper-parameters of the BCP model using statistical learning such that the model minimises the training errors of the BCP model for identifying changes in team performance.
3. Measure the test error rates of the candidate BCP models with existing data sets, and investigate signal detection as a method to assess the sensitivity and specificity of the model in detecting changes in team performance.
4. Potential to port R-scripts to Python.

Skills Required

ESSENTIAL
• Highly experienced with the R statistical programing language
• Able to identify and use R packages appropriate for the analyses
• Experience with, or be capable of learning about Bayesian modelling, change point detection, and machine learning
• Able to work in a team environment
• Willing to share code and communicate different statistical models and analyses to other researchers

DESIRABLE
• Able to port R-scripts to python would be advantageous but not strictly necessary

Expected Outcomes

The expected outcomes from the intern project are:
1. R-scripts that include all analyses carried out for the research, with detailed commenting.
2. A brief report on the change point analyses, tuning of parameters, and test/training error rates as well as measures of sensitivity and specificity.
3. Reporting of analysis for peer-review publication.

Additional Details

The intern will receive $3,000 per month of the internship, usually in the form of stipend payments.
It is expected that the intern will primarily undertake this research project during regular business hours, spending at least 80% of their time on-site with the industry partner. The intern will be expected to maintain contact with their academic mentor throughout the internship either through face-to-face or phone meetings as appropriate.
The intern and their academic mentor will have the opportunity to negotiate the project’s scope, milestones and timeline during the project planning stage.

To participate in the APR.Intern program, all applicants must satisfy the following criteria:
• Be a PhD student currently enrolled at an Australian university
• PhD candidature must be confirmed
• Applicants must have the written approval of their Principal Supervisor to undertake the internship. This approval must be submitted at the time of application.
• Internships are also subject to any requirements stipulated by the student’s and the academic mentor’s university

Applications Close

25 June 2018

Reference

INT – 0415