Design details
Mechatronics was a big part of our design and we have carefully chosen all the mechatronics components we want to use. This is a list of the most important mechatronics components we have in our design and some reasons why we chose to use these parts instead of their alternatives.
​
​
Algorithm
We put in a lot of thought in our algorithms. In order for the robot to play music, we implemented two algorithms, one algorithm to generate the robot commands and one Arduino algorithm to execute the commands. The planning algorithm reads a MIDI file which carries event messages that specify notation, pitch and velocity (loudness or softness). By reading the MIDI file, the planning algorithm thus obtains when to press which key and how long each key needs to be pressed. With the assumption that each hand covers half of the keyboard, the algorithm then formulates the task as an optimization problem for each hand.
In the first phase, the code only considers to press the keys in sequence. It iteratively takes a key that needs to be pressed, considers all possible hand positions that can press this key, and then calculate travel distances from current potential positions ( the positions that can press the previous required key). For each possible hand position for the current key, it then keeps one minimum sequence of hand movements.
After obtaining an optimized sequence of hand movement and its corresponding finger movements, the code adds time stamp for each finger press in the sequence to obtain a sequence of control commands for the robot. Because the robot hand does not travel as fast as human hand, the code then considers the possible delays that would be caused by the hand movement. The code would try to release a finger press earlier and move to the next position to minimize the delay.
After all the planning is completed, the code would output a txt file containing all the necessary information and the txt file content would be transferred to the Arduino control code.
The Arduino code would read the output from the previous algorithm, executes the task subsequently. If a delay cannot be eliminated, the Arduino code would handle the actual delay. If one hand is delayed, both hands would be delayed to keep the rhythm of the music. It would wait until both hands are in place and resume the music.
​