Statistical Methods for Determining the Relationship Between Team Communication Dynamics & Task Performance
Location: Fishermans Bend, VIC
Duration: 5 months
Proposed start date: 1 July 2018
Keywords: Statistical Methods, Bayesian Modelling, Multidiscipline, Machine Learning, Change Point Detection
Please note: Due to the sensitivity and security of this project, students must have Australian Citizenship to apply. Any applicants not meeting this requirement will automatically be deemed ineligible for this project.
In many Australian Defence Force (ADF) settings, teams of two or more operators, with distinct, but interdependent roles, are required to work together to achieve common goals under conditions characterised by significant time pressure and high cost for error.While, the effectiveness of these teams is dependent on a number of factors, effective communication between team members is recognised as vital for team coordination, planning, and execution of mission goals. Recent research has shown that the complexity of team communication (e.g., as operationalised by sample entropy) varies as a function of team workload and practice, and changes in the complexity of communication dynamics are associated with changes in task performance.
The Defence Science and Technology Group (DST) have applied the results of the above research to the training of air-battle management crew in the Royal Australian Air Force (RAAF) in the context of simulated exercises. Teams of eight or more trainee operators are exposed to high-fidelity simulated missions and assessed by instructors on their mission performance. DST has recorded sequences of communication events throughout these training missions, and is currently investigating whether the complexity of the communication sequences can assist assessors in identifying changes in team performance. If the temporal dynamics of team communication can be shown to be a robust and sensitive measure of team performance, then there is potential for these measures to assist instructors in identifying and assessing changes in team performance, both in real-time and during post-training feedback. Furthermore, there is potential for communication complexity measures to be used to facilitate teaming between humans and synthetic agents.
DST is seeking candidates with experience in statistical modelling to assist our researchers in identifying methods for assessing the relationship between communication complexity measures and task performance. Specifically, we require a PhD to (i) use Bayesian analyses of change points in time series of communication complexity (e.g., Sample Entropy) to identify changes in complexity, (ii) use machine learning to tune the hyper-parameters of the Bayesian Change Point model using training data, and (iii) measure the test error rates of the Bayesian models.
Research to be Conducted
Data from a simulation training exercise has already been recorded. Data-frames include time-stamped team communication sequences, communication complexity values, and performance measures.
The successful candidate will carry out the following research on the above data using the R statistical programming language:
1. Analyse communication complexity time-series to detect change points using Bayesian change point (BCP) methods.
2. Tune the hyper-parameters of the BCP model using statistical learning such that the model minimises the training errors of the BCP model for identifying changes in team performance.
3. Measure the test error rates of the candidate BCP models with existing data sets, and investigate signal detection as a method to assess the sensitivity and specificity of the model in detecting changes in team performance.
4. Potential to port R-scripts to Python.
• Highly experienced with the R statistical programing language
• Able to identify and use R packages appropriate for the analyses
• Experience with, or be capable of learning about Bayesian modelling, change point detection, and machine learning
• Able to work in a team environment
• Willing to share code and communicate different statistical models and analyses to other researchers
• Able to port R-scripts to python would be advantageous but not strictly necessary
The expected outcomes from the intern project are:
1. R-scripts that include all analyses carried out for the research, with detailed commenting.
2. A brief report on the change point analyses, tuning of parameters, and test/training error rates as well as measures of sensitivity and specificity.
3. Reporting of analysis for peer-review publication.
The intern will receive $3,000 per month of the internship, usually in the form of stipend payments.
It is expected that the intern will primarily undertake this research project during regular business hours, spending at least 80% of their time on-site with the industry partner. The intern will be expected to maintain contact with their academic mentor throughout the internship either through face-to-face or phone meetings as appropriate.
The intern and their academic mentor will have the opportunity to negotiate the project’s scope, milestones and timeline during the project planning stage.
To participate in the APR.Intern program, all applicants must satisfy the following criteria:
• Be a PhD student currently enrolled at an Australian university
• PhD candidature must be confirmed
• Applicants must have the written approval of their Principal Supervisor to undertake the internship. This approval must be submitted at the time of application.
• Internships are also subject to any requirements stipulated by the student’s and the academic mentor’s university
25 June 2018
INT – 0415