Real-time Object Detection and Pattern Recognition in Images and Video

Location: Sydney, NSW

Duration: 5 months

Proposed start date: March 2019

Project Background

WallSync is a start-up that has just received investment from Telstra through their muru-d accelerator program to develop a mobile application that analyses images taken from a mobile device camera.

WallSync take the output of physical collaborative workshops; such as handwritten sticky notes, diagrams and writing on whiteboards and create a digital replica.

The proposed project involves taking the image data that represents the workshop output and;

  • Object detection – for example detecting the sticky notes in a range of lighting conditions.
  • Correction – for each detected object, performing image correction so that we can get the best results from handwriting detection.
  • Pattern detection – determining groups of items based on how they are organised in columns or clusters.
  • Content detection – for example determining what is handwriting and what is a sketch on a note.
  • Performance tuning – getting the above results on a range of mobile devices in a timely manner, leading to a greater user experience.

A 20 second video of our current prototype is here https://makemebeta.wallsync.net/assets/img/hero-video.mp4 and will give a good flavour of the problem space.

Research to be Conducted

The project seeks to get a deep as possible understanding of the content of the workshop output and the intent from analysing the image data.

  • Review potential computer vision algorithms and machine learning approaches that best solve the problem within the project constraints.
  • Develop the model to handle a wide range of real-world scenarios and lighting conditions.
  • Iteratively tune the performance of the model.

Skills Required

We are looking for a PhD student with the following:

ESSENTIAL

  • Computer vision
  • Machine learning

DESIRABLE

  • Mobile app programming (especially iOS Swift)

Expected Outcomes

This project seeks to build up the solution iteratively. To get to a working “Minimum Viable Product” that solves some of the problem in some of the scenarios as quickly as possible and then continually refine and enhance for the duration of the engagement.

As such, WallSync expect;

  • A working model that runs on iOS devices at a minimum, and Android as well if possible.
  • Well documented code, along with instructions on how to maintain and make minor enhancements.
  • If machine learning is involved, the training data along with documented instructions for conducting further training and continual improvement.
  • The solution to be integrated with existing services that interact with external digital tools.

Additional Details

The intern will receive $3,000 per month of the internship, usually in the form of stipend payments.

It is expected that the intern will primarily undertake this research project during regular business hours, spending at least 80% of their time on-site with the industry partner.  The intern will be expected to maintain contact with their academic mentor throughout the internship either through face-to-face or phone meetings as appropriate.

The intern and their academic mentor will have the opportunity to negotiate the project’s scope, milestones and timeline during the project planning stage.

Applications Close

 27 February 2019

Reference

APR – 0823