The aim of this blog is to document our project work in the course Intelligent Interactive Systems at Uppsala University. Our project is about creating software for a bartender robot that can recognize hand gestures. We will familiarize ourselves with state-of-the-art technology and will therefore work with  (among other things):

• (hardware) a Leap Motion controller for recognizing user input
• (software) an artificial neural network library, such as Caffe or Google's TensorFlow, in order for our robot to distinguish between different gestures.

Today the weather in Uppsala was beautiful, so me, Pablo and Ismael met at Café Storken to create a timetable for our project. We realized that we have just about three weeks left until the final deadline, so we decided to split the work at hand into more manageable subtasks. The first order of business was to book a time slot with the Leap Motion controller so that we can get a better feeling for the format of the data that we will need to work with. We agreed that Pablo and Ismael will take a look at the controller tomorrow and on Saturday, the three of us will meet and experiment with the TensorFlow library. At this time, we believe that the most challenging part of the project will be to train the artificial neural network, so it is important to start as early as possible.

Other things left to do include the text-to-audio module (see the System Architecture diagram) and to generate data, in the form of video clips of people showing different hand gestures. However, we believe that those parts of the project will take relatively short time to implement, so we will put them on hold for the times being.