Speech recognition market is prospected to grow 19.8% from 2018 until 2024. Market grown can be attributed to high potential in every area, specially in healthcare, in-car systems, military, smartphones, etc. Speech recognition is less used in construction sites. So this type of research can open up new opportunities, like it allows people in construction site to control a robot by voice.
Process
This project is completed in 5 phases. In first phase the speech is converted to text for commands. In the second phase robot is moved with ros commands from robot operating system. Then robot movement is again translated in rhino environment for augmented feedback through fologram which can help on construction site to visualize the path ways of robots. It will increase the efficiency of logistics when workers are able to avoid the robot paths. Link to the git-hub repository for this project:
https://github.com/MRAC-IAAC/Human-Machine-Interaction
Arduino Voice Controlled Robot
Controlled the robot with voice commands using Arduino. Below are the hardware specs used for building the voice operated Arduino mobile robot.
Speech to Text (Flow Chart)
Movement pattern (Workflow)
Used Yolo Darknet frame to detect the robot and person and then used homography matrix to bring the cordinates of robot and person from 3d space to 2d space for showing in real time with mixed reality.
Movement pattern ( Live Tranformation)
From 3D Space to 2D Space with UDP protocol.
Augmented Feedback in (Real Environment)
Site patterns of robot and workers can be seen in real environment on construction site with mixed reality headset or mobile phones which can help in better decision making and safety.
Real Time Voice Command Movement and Augmented feed back
In this case ROS and Windows Platform are parallel sending and receiving data with ROS Bridge & Benguesht.
Real Time Voice Command Movement and Augmented feed back (Demo)
Custom made Command used in this demo are :
Forward ? Go | Backward ? Back | Right ? Positive | Left ? Negative
</p>
Augmented Feedback in (Real Environment)
This is the demo for real time augmented feedback of movement pattern, similarly augmented feedback on construction site can help in increasing the efficiency of logistics when workers are able to avoid the robot paths by having visual access to the invisible paths.
Title: Voice Control and Augmented Feedback of Robots
Seminar: MRAC
Faculty: Angel Monuz, Agustina Palazzo