Introduction to Processing and Arduino

The purpose of this project is to combine the physical and the digital world…

Have you ever wanted to be like iron man and to be able to control the screen of your computers just by using gestures, to be able to interact with the digital world from the physical one?! Here we are doing exactly this, by showing you a fun new way of playing an all-time favourite game Whack-A-Mole. You have the chance to meet all your favourite looney tUNES in the virtual game called Whack-A-Toons.

This is a very interesting project in which we are going to learn how to implement virtual reality using Arduino and Processing. So, in this project we have tried to create one in which we can simply wave our hand in front of the computer and move a hammer to the desired location and ‘whack’ the looney toon, it’s simple and a lot of fun.

What you will need:

  1. Arduino uno
  2. Hall sensor, bipolar A3144
  3. A small piece of magnet x 2
  4. Bluetooth module HC-05
  5. 9V Battery
  6. Connecting wires
  7. Tape
  8. Arduino software
  9. Processing software
  10. Unique color object
  11. 1 Resistor 10K
  12. Bread board
  13. A Computer with webcam and Bluetooth


How does it work?

First you run the Arduino code to power the Bluetooth in order to connect to the computer and also power the hall sensor. If everything goes fine the LED on your Bluetooth module will light up and your webcam light will go on. Then the application will automatically launch on your screen, a calibration needs to be done, you have to select the object to be tracked. This tracing can be simply done by clicking on the object. In this case the object is the yellow fabric, anything in a bright colour can be used. The camera of your computer detects the yellow colour of your object and can track it on the screen. Now you can move your object and notice that the hammer is following the yellow object. Use a unique colour object and a bright room for best results.

The Arduino is powered by a 9V battery, the bipolar hall sensor uses the negative and the positive magnetic field of the two magnets to create a pulse that gives back value according to the polarity of the sensor. Therefor the two magnets need to be of an opposite polarity. Each of them is attached to the index and the middle finger and the hall sensor is attached to the thumb. That’s when you switch fingers that’s when the hall sensor detects the impulse.

The Whack-A-Toon is a project of IaaC, Institute for Advanced Architecture of Catalonia developed at Master in Advanced Architecture in 2017/2018 by:


Nikoleta Nikolova
Milivoje Momcilovic
Ipsita Datta


Angelos Chronis
Angel Muñoz