Emodget
Wearable that Augments Your Speech with Emojis
Emojis are great, they make us laugh, they make us empathize. But for all their greatness, they are constrained within digital communication.
As a heavy user and die-hard fan, I find myself deprived of emoji's amazing power when I talk to people face to face and couldn't use them.
That’s why I developed Emodget to help us communicate better in daily conversation.
Independent Work | 2016 Skills: Integrative Product Design, Prototyping, Electrical Engineering Tools: Arduino, LED Matrix, Voice Recognition Unit
Ideation
I gave up the emoji-glass since it could hinder eye-contact in face-to-face communication. People wearing these glasses may not see clearly, and whoever is talking to them may feel offended by these "emoji-glassholes". I also thought about a rabbit-ear with voice recognition that can perform different actions based on what it "hears". Maybe that could be something to try after Emodget.
Technical Solution
DFRobot Voice Recognition Unit was used to identify 50 pre-stored sentences. According to the voice command, Arduino would display various emojis on the 16*16 LED matrix. To make the whole device portable, Aduino was powered by 3.7V lithium batteries.
Making Process
In the 1st version, a group of 12 words and their corresponding emojis was enabled.
The emoji patterns were converted into [1][32] array data so that Arduino could access the LED matrix through the Display(array) function.
To be specific, I first translated them into binary code, and then into hexadecimal code as the array element.
Here is what the Arduino sketch below does:
(1) Voice.addCommand() includes voice commands and assign value labels to them. (Synonymous commands can share the same label.)
(2) In the main loop, Voice.read() gets the value of the detected command, based on which switch() performs different tasks.
(3) The Display() function enables the 16*16 LED matrix to put on various emojis through Send() and Scan_Line();
#include <avr/wdt.h> #include <VoiceRecognition.h> VoiceRecognition Voice; #include <Arduino.h> //IO #define LEDARRAY_D 15 #define LEDARRAY_C 16 #define LEDARRAY_B 17 #define LEDARRAY_A 18 #define LEDARRAY_G 19 #define LEDARRAY_DI 7 #define LEDARRAY_CLK 8 #define LEDARRAY_LAT 9 unsigned char Display_Buffer[2]; const unsigned char Word1[1][32] = { 0xFF,0xFF,0xFF,0xE1,0xC0,0x80,0x8C,0x92,0xC0,0xE0,0xF0,0xF8,0xFC,0xFE,0xFF,0xFF, 0xFF,0xFF,0xFF,0x87,0x03,0x01,0x31,0x49,0x03,0x07,0x0F,0x1F,0x3F,0x7F,0xFF,0xFF, // love you }; const unsigned char Word2[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xB7,0x6A,0x7F,0x7F,0x7F,0x7F,0xA0,0xBD,0xDE,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0x7B,0xBD,0xFD,0xFD,0xFD,0xFD,0x7B,0x7B,0xF7,0xCF,0x3F,0xFF, // interesting }; const unsigned char Word3[1][32] = { 0xF8,0xE6,0xDF,0xBF,0xA4,0x6D,0x7F,0x7F,0x7F,0x7F,0xBC,0x80,0xC0,0xE0,0xF8,0xFF, 0x3F,0x0F,0x07,0x83,0xC3,0xC1,0xC1,0x81,0x01,0x01,0x03,0x03,0x07,0x0F,0x3F,0xFF, //despise }; const unsigned char Word4[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xAF,0x57,0x7F,0x7F,0x7F,0x78,0xB7,0xBB,0xDC,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0xEB,0xD5,0xFD,0xFD,0xFD,0x3D,0xDB,0xBB,0x77,0xCF,0x3F,0xFF,//joyful }; const unsigned char Word5[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xB7,0x6B,0x63,0x77,0x7F,0x7F,0xBD,0xBE,0xDF,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0xDB,0xAD,0x8D,0xDD,0xFD,0xFD,0x7B,0xFB,0xF7,0xCF,0x3F,0xFF, // moved }; const unsigned char Word6[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xA3,0x41,0x01,0x41,0x63,0x7F,0xBF,0xB8,0xDF,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0x8D,0x15,0x11,0x35,0x8D,0xFD,0x7B,0x7B,0xF7,0xCF,0x3F,0xFF, // worship me }; const unsigned char Word7[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xA7,0x5B,0x5B,0x67,0x7F,0x7C,0xBB,0xB8,0xDF,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0xCB,0xB5,0xB5,0xCD,0xFD,0x7D,0xBB,0x3B,0xF7,0xCF,0x3F,0xFF, // horrified }; const unsigned char Word8[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xBF,0x7F,0x43,0x7B,0x7F,0x7F,0xB7,0xB8,0xDF,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0xFB,0xFD,0x05,0xF5,0xFD,0xFD,0xDB,0x3B,0xF7,0xCF,0x3F,0xFF, // evil giggle }; const unsigned char Word9[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xBF,0x77,0x0F,0x7F,0x67,0x7F,0xBE,0xBD,0xDF,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0xFB,0xDD,0xE1,0xFD,0xCD,0xFD,0xFB,0x7B,0xF7,0xCF,0x3F,0xFF, // heartbroken }; const unsigned char Word10[1][32] = { 0xF8,0xE6,0xDE,0xBF,0xBF,0x7F,0x7F,0x41,0x73,0x73,0xBF,0xBF,0xDF,0xE7,0xF8,0xFF, 0x3F,0x8F,0xA7,0xAB,0xAB,0xED,0xFD,0x81,0xCD,0xCD,0xFB,0xFB,0xF7,0xCF,0x3F,0xFF, // embarrassed }; const unsigned char Word11[1][32] = { 0xF8,0xE7,0xCF,0xB7,0xBB,0x4F,0x37,0x4B,0x67,0x7F,0xBC,0xB9,0xDB,0xE7,0xF8,0xFF, 0x3F,0xCF,0xE7,0xDB,0xBB,0xE5,0xD9,0xA5,0xCD,0xFD,0x7B,0x3B,0xB7,0xCF,0x3F,0xFF, // enraged }; const unsigned char Word12[1][32] = { 0xF8,0xE7,0xDF,0xBF,0xBF,0x63,0x5D,0x55,0x5D,0x63,0xBF,0xBF,0xDF,0xE7,0xF8,0xFF, 0x3F,0xCF,0xF7,0xFB,0xFB,0x8D,0x75,0x55,0x75,0x8D,0xFB,0xFB,0xF7,0xCF,0x3F,0xFF, // at a loss }; const unsigned char Init_Display[1][32] = { 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, }; void setup() { Serial.begin(9600); pinMode(LEDARRAY_D, OUTPUT); pinMode(LEDARRAY_C, OUTPUT); pinMode(LEDARRAY_B, OUTPUT); pinMode(LEDARRAY_A, OUTPUT); pinMode(LEDARRAY_G, OUTPUT); pinMode(LEDARRAY_DI, OUTPUT); pinMode(LEDARRAY_CLK, OUTPUT); pinMode(LEDARRAY_LAT, OUTPUT); Display(Init_Display); Voice.init(); Voice.addCommand("ai ni ou",0); Voice.addCommand("he he",1); Voice.addCommand("bi shi",2); Voice.addCommand("chao kai xin",3); oice.addCommand("gan dong",4); Voice.addCommand("diao bu diao",5); Voice.addCommand("xia dai",6); Voice.addCommand("hei hei",7); oice.addCommand("lan shou xiang gu",8); Voice.addCommand("gan ga",9); Voice.addCommand("fen nu",10); Voice.addCommand("mi mang lian",11); Voice.start(); wdt_enable(WDTO_1S); } void loop() { switch(Voice.read()) { case 0: Display(Word1); Serial.println("received'ai ni ou',command flag'0'"); break; case 1: Display(Word2); Serial.println("received'he he',command flag'1'"); break; case 2: Display(Word3); Serial.println("received'bi shi',command flag'2'"); break; case 3: Display(Word4); Serial.println("received'chao kai xin',command flag'3'"); break; case 4: Display(Word5); Serial.println("received'gan dong',command flag'4'"); break; case 5: Display(Word6); Serial.println("received'diao bu diao',command flag'5'"); break; case 6: Display(Word7); Serial.println("received'xia dai',command flag'6'"); break; case 7: Display(Word8); Serial.println("received'hei hei',command flag'7'"); break; case 8: Display(Word9); Serial.println("received'lan shou xiang gu',command flag'8'"); break; case 9: Display(Word10); Serial.println("received'gan ga',command flag'9'"); break; case 10: Display(Word11); Serial.println("received'fen nu',command flag'10'"); break; case 11: Display(Word12); Serial.println("received'mi mang lian',command flag'11'"); break; } wdt_reset(); } void Display(const unsigned char dat[][32]) { unsigned char i; for( i = 0 ; i < 16 ; i++ ) { digitalWrite(LEDARRAY_G, HIGH); Display_Buffer[0] = dat[0][i]; Display_Buffer[1] = dat[0][i+16]; Send(Display_Buffer[1]); Send(Display_Buffer[0]); digitalWrite(LEDARRAY_LAT, HIGH); delayMicroseconds(1); digitalWrite(LEDARRAY_LAT, LOW); delayMicroseconds(1); Scan_Line(i); digitalWrite(LEDARRAY_G, LOW); delayMicroseconds(100);; } } void Scan_Line( unsigned char m) { switch(m) { case 0: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, LOW); break; case 1: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, HIGH); break; case 2: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, LOW); break; case 3: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, HIGH); break; case 4: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, LOW); break; case 5: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, HIGH); break; case 6: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, LOW); break; case 7: digitalWrite(LEDARRAY_D, LOW);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, HIGH); break; case 8: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, LOW); break; case 9: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, HIGH); break; case 10: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, LOW); break; case 11: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, LOW);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, HIGH); break; case 12: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, LOW); break; case 13: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, LOW);digitalWrite(LEDARRAY_A, HIGH); break; case 14: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, LOW); break; case 15: digitalWrite(LEDARRAY_D, HIGH);digitalWrite(LEDARRAY_C, HIGH);digitalWrite(LEDARRAY_B, HIGH);digitalWrite(LEDARRAY_A, HIGH); break; default : break; } } void Send( unsigned char dat) { unsigned char i; digitalWrite(LEDARRAY_CLK, LOW); delayMicroseconds(1);; digitalWrite(LEDARRAY_LAT, LOW); delayMicroseconds(1);; for( i = 0 ; i < 8 ; i++ ) { if( dat&0x01 ) { digitalWrite(LEDARRAY_DI, HIGH); } else { digitalWrite(LEDARRAY_DI, LOW); } delayMicroseconds(1); digitalWrite(LEDARRAY_CLK, HIGH); delayMicroseconds(1); digitalWrite(LEDARRAY_CLK, LOW); delayMicroseconds(1); dat >>= 1; } }
Testing
During the making process, several problems were encountered and solved. To name a few:
(1) At first I connected the LED matrix with Arduino Uno3 and the Voice Recognition module according to their handbooks, but the LED screen wouldn't work. Then I found out the digital pin2,4 on Uno were used by Voice Recognition module for communication, So I rewired the pinD,C,B,A,G of the matrix to analog pin A1-A5 on Uno instead of the suggested digital pin2-6, and the problem was fixed.
(2) Early into the prototype test, whenever I gave out a voice command, the corresponding emoji would flash on the screen and disappear instantly, that led me to fix the timing error in the code.
