Hi // Hard Ware III
‘ Hi ‘ is a project by MRAC 01 student Tomas Quijano , Abanoub Nagy ,Christopher Booth and
Alberto Martínez during the Hardware III seminar of the third term. Hardware was taught by IAAC faculty Starsk Lara ,Daniil Koshelyuk
During the Hardware III course, students had to explore ::
Using Unity and use it as a tool to explore ways of human robot interactions
ABSTRACT
In this project is foucasing on using Unity as a broker or middle man to communicate with human interaction using a leap motion camera. The communication from the robot side is using ROS to control the robot.
Introduction
As new tools are growing every to help designer to be more close to their design and more interactive. construction site also have its emerging technologies and application in sites.What we saw is lagging and opportunity to contribute is the amount of set up you need and tools to do specific thing on site. This kind of set ups and installation needed is maybe good for laps but is not convenient to construction site with its harsh environment that is changing every day. So in our project we wanted to eliminate or reduce the the equipment needed for a human interact with robotic arm. Taking in consideration that the workers would be working with robot, so our system allow easy usage and less installation.
The Project
The project was divided into two teams for the two main tasks :
A) Front end
This team of the project which responsible to designing the user interaction . Build a virtual box in a virtual space. , The pattern of the user’s intersection with the surfaces is logged by the interface and sent to the project back end via an ROS Bridge web-socket.
We used a leap motion camera package and from this package there is a model for the hands . Another hierarchy object surrounding the right hand object, that was the collusion object.A bounding box created virtually 0.80 m * 0.80 m and its center was locked in a proportional to the camera distance so can be move the camera around and the hand movement will be always in relation without a fixed places and without a certain calibration.
After setting up our box and the collusion object.We created a script collusion object that will be activate the box when the the hand (Colluding object) will trigger it.While the box is triggered or the right hand is inside the box will get the position of the hands and collect this data coordinate .
The coordinate data collected:
- Pose
The initial idea was to collect and send the the Pose of the hands and this would be sent Pose target position for the robot End factor would be the target position to go to for the robot.Sooner we faced problems from the back-end team with the move it simulation of not receiving or not reading the pose position.
2. Twist
Reading more in the documentation of ‘move it’ the type of messages that could be read was a Twist message. The different between Pose and Twist. If Pose is the end factor target point that the robot should goes too . The twist is like a joy stick controller that move the robot with liner and angular direction.
3. Twist Stamp
After getting the twist position and proven that it is working and read by the ‘Moveit servo’. It was needed to add the update it with other type of messages which is Twist stamp message. The diffrenece between the twist messages and twist stamp, the twist stamp is the same data with added header.
Communication :
To communicate this date from the unity side or from the Human iteration to the robot. We used ROS bridge to send the Twist Stamp message from unity to ROS (Moveit Servo) to control the robot through web-socket protocol.
- Units:
In the script there is a function used to convert the unity coordinate to normal world coordinate. Since Unity Z axes and the Y axes are switched. So we used the function
—- Unity2Ros() — to convert and prepare it before publishing this message.
2. Scalier:
The movement of a human arm is not the same like the robot movement. So a Scalier number added to fine tune the movement of the robot. Scale up or scale down this movement of the robot if someone witth shorter hand or moving to fast or to slow. There is two Scalier one for the liner motion and one for the angular motion. The recommended for the angular motion is to have very small Scalier since it was creating a lot of singularity.
B) Back end
The twist transformation of hands done by the user is sent through ROSBridge .
GitHub
Our project repo
for source files:
- Repo
https://github.com/Albertomartinezlopez/Ros-Ur-Roca-Frontend
Resources + Documentation
The leap motion
- Package
https://developer.leapmotion.com/unity
The unity
- Octocopler style
https://askubuntu.com/questions/320121/simple-port-forwarding
- ROSSharp Docmentation
https://github.com/siemens/ros-sharp/blob/master/Libraries/RosBridgeClient/MessageTypes/Geometry/msg/Twist.cs
- ROS real time
https://ros-planning.github.io/moveit_tutorials/doc/realtime_servo/realtime_servo_tutorial.html