Concept:
Finstruments is an interactive educational tool for children between ages of 4 to 8. We wanted to provide a fun introduction to musical instruments and music theory. The goal of Finstruments is to help children develop basic musical motor skills by associating finger positions with the audio output. With three instruments to choose from – vocals, piano and trumpet – Finstruments bring to life the small orchestra of child’s imagination.
Our inspiration for Finstruments came from wanting to explore methods through which wearable technology can provide a fun learning experience for children, by making it more immersive through tactile interaction. Finstruments was inspired by the MiMu glove, which has allows vocal artists turn hand movements into an audiovisual spectacle.
From taking our original inspiration and turning it into an actual concept, we began thinking of different sensors we could use for our product. We originally wanted to use a distance sensor to control the volume of the sound, however, we found that using a rotation sensor would be easier to use for children, and provide better accuracy of output. We also wanted the users to stand on a Dance Dance Revolution-like pad, in which they could choose their instruments by tapping the existing pressure sensors, but we decided that making our own smaller pads would make the project more accessible for children, and would reduce the space taken up in physical setup. We also wanted to allow all 5 fingers to be able to play musical notes to coordinate with real life instruments like flute, however we encountered some calibration errors with one of the flex sensors, which forced us to omit it from the glove, reducing the number of interactive fingers to 4. In this stage, we concluded our list of sensors to include flex sensors for the fingers, force sensors for the instrument choice, and a rotation sensor for the volume.
My Role:
My contribution to the project was doing the arduino code and to help assemble the glove. I had to find
out how to use all three of our sensors, which was easy since I followed tutorials online for the force, flex
and rotation sensors alike. I helped assemble the glove with Sina by piecing the glove together the arduino and binding it to the glove and soldering the wires together as well. I also helped with finding and purchasing the materials we had to used, such as the fabrics and several gloves to use to make Finstruments. Since we wanted to take in account for the glove to fit a child’s hand and also be comfortable for them to use I had to go find an appropriate glove. We eventually chose a gardening glove, since it offered a comforting wrap and stretch over a hand. Next was assembling the sensors onto the glove where they would not obstruct the use and comfort for the user. We decided looking at precedents to base our design off of, and by analyzing them we realized that by putting the rotation and flex sensors on the back of the hand, it allowed for a better experience as the readings would be more accurate from how the way the conductive ink on the flex sensors would stretch. Reflecting back on the work, the hardest part we had struggled with was that we had was that some of our sensors had inconsistent readings due to the voltage that was lost with all the wires we had, however we eventually did manage to get it to work.
Click here to watch a demonstration
How it works:
Finstruments is a glove that uses flex sensors on 4 fingers, as well as a rotation sensor on the back of the glove for volume control. There are also three pads connected to the Finstrument system to choose the instruments that the user wants to play. Both the pads and the glove lead to a box which contains the Arduino and breadboards with the core wiring. The contents of this box is then connected to the computer via USB port.
Three pressure sensors are placed on a mat at the user’s feet. They are used to switch the instrument to the user’s choice. The pressure sensors work with Arduino by sending resistance data. If the value of resistance is over a certain amount, (the threshold is different for each sensor for callibration purpose) the processing software changes the instrument being played.
There are 4 flex sensors that work in similar fashion as the pressure sensors. The conductive ink on the sensors become further apart as the flex sensor is bent, this causes the resistance value to change depending on the angle of the sensor. Based on this value, the processing software plays the note associated with the finger the sensor is placed on.
There is a rotation sensor that works just like a potentiometer and changes its resistance value based on the rotation angle. The rotation sensor is connected directly into the circuit and unlike the other sensors, it’s not used in a voltage divider circuit. The resistance value changes the volume level in the program.
By connecting the Arduino through the USB port of the computer, Finstruments communicate with Processing to run its musical program. By turning the analog inputs of the sensors into digital output, Processing allows us to turn a tactile function into audio output. Processing communicates with the Arduino to receive this information, and uses the information to play musical notes. Our final result has nearly all of the proposed product functionality. Finstruments is a musical interactive glove, turning the tactile movements of a person moving their fingers to produce a musical sound between three instruments of choice. With matching consistent color design throughout, on the glove, interface and the instrument choice mat, Finstruments provides users with an easy visual choice of selecting between vocals, piano or a trumpet to play musical notes with their fingers, with adjustable volume and a fun interactive interface.
Difficulties Faced:
Some of the overall major challenges we faced was finding a design to fit our product, to both appeal to kids as well as engage them, and fine tuning the flex sensors for our glove. The way we accomplished these challenges was by researching precedents to help solve various problems, as well as creating constraints over the inconsistent inputs to make them more stable.