Rafael Papallas

PhD Student Software Engineer

Cashier Robot

Human-Robot Interaction For Cashier Robot


Project Information

© 2016/2017 The University of Leeds and Rafael Papallas

Date 2016/2017
Categories Robotics
Programming Language Python
Tags , ,
GitHub Repository https://github.com/rpapallas/baxter_cashier
Website https://rpapallas.github.io/baxter_cashier/

Project Description

"Human-Robot Interaction for Cashier Robot" is my final-year project submitted at the University of Leeds. My individual project was awarded the "Buckley Prize" as the best 60-credit project in the School of Computing for the academic year 2016-2017.

We used Baxter Robot by Rethink Robotics and Python as the programming language.

As can be seen from the diagram below, the project has three main functionalities among others:

  • Skeleton Tracking: The system detects people and the positioning of their body parts (head, hands, torso etc) using a depth sensor. We are interested in tracking the left and right hand of the user to pick banknotes. Using a custom filter algorithm called "Two-Way Pose Elimination" the system is able to distinguish false positive hand-over signals (that is, the user did not raise his/her hand to hand over a banknote) from actual hand-over signals (that is the user actually is handing over banknotes to the robot).
  • Money Recognition: Different approaches for money recognition has been evaluated including colour detection, template matching and AR-code recognition. Because of time constraints, the AR-code recognition has been selected mainly because (1) was very accurate with no false positives and (2) was well tested from other people (third-party library).
  • Change Handling: The robot is able to handle change. This involves calculating the change based on the user's order, picking up change from the table and returning the change to the user's hands.

Furthermore, the robot is aware of obstacles (table and walls) and the robot is planning accordingly to avoid those obstacles throughout the interaction.

The development of the project was split into three phases each of which brought a major functionality or improvement to the system. The final solution was evaluated against ten novice participants.


Click magnifier to enlarge