A Universal Bluetooth Remote App

About half a year ago I programmed a small app for Android that communicates with an Arduino over Bluetooth (http://coretechrobotics.blogspot.com/2013/12/controlling-arduino-with-android-device.html). It had some very basic features and a simple user interface, which was just enough for the project it was designed for. But no two projects are alike.

Because of that I decided to make a universal remote that would be customizable to all possible applications.

Ideas

The main idea is that the user should be able to create his own remote, without having to learn java programming. This would require the app to have some kind of built in editor.

Another part is Multitouch, which is essential for any touchscreen application. It seems to be simple at first but as I knew from my previous attempt at an android app, this makes things extremely complicated. Imagine you don’t just have an X/Y position of your mouse pointer but a whole array of those.

Implementation

Instead of finally learning how to program in pure Java I went with Processing again. This helped me a lot and reduced the time I had to spend on the project.

At first I needed Multitouch. This was a difficult part, I had to read through a lot of Android documentation and Internet articles until it finally worked. Maybe I will write a separate Post about this in the near future.

The next step were proper UI elements like sliders and buttons. My previous app used the controlP5 library by Andreas Schlegel (http://www.sojamo.de/libraries/controlP5/). But I wanted something of my own. I chose to write a UI library from scratch, with a simple edgy design and full Multitouch integration. The four UI elements are sliders, joysticks (2D-Sliders), buttons and switches.

The editor is the main part of the app, from there the user can drag and drop the elements on the screen. The size and position can be changed by moving its center or corner.

The color can be selected in a popup window with RGB sliders.

Another important thing: If you look at a RC remote for a car, the steering wheel centers itself with a built in spring. This way the car will drive straight if you let go of the wheel. The same applies to the sticks of an airplane remote. For this purpose I made a second window that lets you select which slider or which axis of a joysticks should jump back to the middle position.

Saving those UI layouts is also implemented. This means you can create up to 8 layouts and save or load them. The most recent layout will be loaded automatically with the app’s startup.

Finally, with a press of the back-button on your Android device the editor buttons vanish and you can use the controls you created.

This means that the slider values and button states are transmitted over Bluetooth. Actually I wanted to write the Bluetooth part myself but had to give up after a while. The process of selecting the Arduino from a menu is complicated enough. To save development time I am using the Ketai Bluetooth library again (https://code.google.com/p/ketai/).

Protocol

The protocol is basically the same as in my previous app, with some additions to support switch states and the slider centering. If a UI element changes, its key (a character A-Z) will be transmitted, followed by its value. This way, the Arduino knows what variable to write the value to. If no control is touched, nothing will be sent. If multiple elements are being changed at the same time, all their values are being transmitted, one at a time. This works fine for up to 3-4 touch points, but you can cause some errors if your try to touch too many elements at the same time, which is should be impractical anyway.

Installation Instructions

Unfortunately I don’t have a Google Developer account, so you won’t find my app on the Play Store. You can download the .apk-file from this link:

https://dl.dropboxusercontent.com/u/7740278/CTRC.apk

1.       Download the .apk and transfer it to your android device

2.       Open it with a file explorer and install it

3.       Done! You can now create your own Bluetooth Arduino remote

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Example Arduino Sketch

I also wrote a simple example sketch you can upload to an Arduino board of your choice. It handles the serial communication with a Bluetooth board and writes the values of the remote to variables.

https://dl.dropboxusercontent.com/u/7740278/CTRCexample.zip

This is not a fully features library, so you will have to make some manual adjustments depending on which UI elements are used in the Android app. But this is nothing more than Copy and Paste.

Potential Improvements

I put a lot of time into it, but the app still isn’t flawless. There is a settings-button, which currently has no function because there are no changeable settings. The Bluetooth protocol could use some reworking, too. But overall I am very pleased with the outcome of this project. 

HeadUp - A Webcam Posture Monitor

Since receiving a lot of feedback about my posture sensor solution I was thinking about how to improve it. Despite its functionality there are a few problems like the continuous clicking of the ultrasonic sensor and the way it is installed on the chair’s rest.

Ideas

One way of fixing the clicking noise would be to use an infrared-based proximity sensor. These are available in different operating ranges and prices but sadly I could not find one that was cheap enough and had the right range for the project.

Improving the chair holder is really difficult, too. It would require a lot of 3D-printed parts and still not work on chairs with low rests.

This meant the old idea had to be discarded completely and the project was suspended until I came across this great blog post about a pc-monitor-mounted sensor:

http://www.paleosun.com/blog/entry/prototyping-fun-with-a-posture-alarm

That’s how I got the idea of simply using the webcam as a sensor. Most people have cameras on their laptops and even desktop PCs so there is no additional hardware needed.

Implementation

Because I already used it in other projects I went with Processing as a programming language again:

https://www.processing.org/

To detect the position of the head the software has to scan the webcam image. This is a nearly impossible task without the help of a powerful library. OpenCV is the solution:

http://opencv.org/

OpenCV is an Open Source library that contains computer vision functions and was originally developed by Intel. Face detection is just a small part of it.

The “OpenCV for processing” library by Greg Borenstein can easily be downloaded with the Library Manager in Processing and already contains some example sketches to start with.

The software I wrote works in a similar way as the Arduino code in the posture sensor. It scans the webcam image and compares the height and the size (~distance) of the face to limit values. If the limits are exceeded for more than two seconds the alarm will sound and then silence after a few seconds. If the user gets up from his chair or the face is not visible the alarm will not be triggered. The alarm itself is just the Windows error sound. The user interface consists of the live image in the background. Three buttons are for setting up the limit values and for pausing the alarm.

Finally Processing can export everything into a standalone application.

Download

If you have a computer with a webcam you can give “HeadUp” a try:

https://www.dropbox.com/sh/j9ezha6dx37jlla/X0ZRXMsmrJ/HeadUp.zip

Instructions: 1. Download HeadUp.zip

                       2. Unzip everything into one folder

                       3. Run HeadUp.exe

Compatibility: Currently only windows is supported. And there are a few webcams it will not work with depending on the drivers. If there are problems you should try running the application in administrator mode. The newest version of java needs to be installed as well.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

I am also releasing the source code so you can customize and improve it:

Copyright (c) 2014 Coretech Robotics

import gab.opencv.*;
import processing.video.*;
import java.awt.*;
import java.awt.Toolkit;//this is needed for the windows sound

int ypos; // height
int rSig; //distance
int almTimer; 
int trigHeight = 0; //height limit
int trigDist = 0; //distance limit
boolean alm, pause;

Capture video;
OpenCV opencv;

void setup() {
  size(320, 240);
  //The next lines are for intitializing the camera and OpenCV
  video = new Capture(this, 640/4, 480/4);
  opencv = new OpenCV(this, 640/4, 480/4);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
  video.start();
}

void draw() {
  getDistance(0);  // run the OpenCV routine
  if (trigHeight!=0 && trigDist!=0 && !pause) { //check if limits have been initialized
                                                //and if pause is off
    if (rSig > trigDist || ypos > trigHeight) { //compare values to limits
      alm = true; 
    }
    else {
      alm = false;
    }
  }

  if (alm==false) almTimer = millis()+2000;  //reset alarm timer if alarm is off
  else if (millis() > almTimer) { //check if alarm timer has expired
    if (millis()-2000 < almTimer) { //do this for additional 2 seconds
      Toolkit.getDefaultToolkit().beep(); //call the windows alarm sound
      delay(150); 
    } 
  }
  
//The following part draws the 2 buttons and checks if they were pressed
  
  textSize(14);
  fill(0, 255, 0);
  text("set distance", 18, 220);
  text("set height", 136, 220);
  text("pause", 248, 220);
  
  stroke(0, 255, 0);
  
  noFill();
    if(mousePressed && mouseOver(10, 200, 100, 30)) {
      trigDist = rSig+3;
      fill(0, 255, 0);
    }
  rect(10, 200, 100, 30);  
  
  noFill();  
    if(mousePressed && mouseOver(120, 200, 100, 30)) {
      trigHeight = ypos+3;
      fill(0, 255, 0);
    }
  rect(120, 200, 100, 30);
  
  noFill();
  if(pause) fill(0, 255, 0); // this part draws the pause switch
  rect(230, 200, 80, 30);
 
}

void getDistance(int interval) { //OpenCV functions
  //pushmatrix and popmatrix prevents the buttons from beeing scaled with the video
  pushMatrix(); 
  scale(2); // scales the video to the window size
  opencv.loadImage(video);
  
  image(video, 0, 0 ); // this draws the webcam image

  noFill();
  if (alm) stroke(255, 0, 0); //draw all lines red if alarm is active
  else stroke(0, 255, 0);
  strokeWeight(2);
  Rectangle[] faces = opencv.detect();
  int dist = 0;
  for (int i = 0; i < faces.length; i++) {
    println(faces[i].x + "," + faces[i].y);
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
    rSig = faces[i].height;
    ypos = faces[i].y;
    int delta = trigDist-faces[i].height;
    //the following line draws a second box with the limit distance
    if (trigDist!=0) 
    rect(faces[i].x-delta/2, faces[i].y-delta/2, faces[i].width+delta, trigDist);
  }
  //This draws a line at the limit height:
  if (trigHeight!=0) line(0, trigHeight, width, trigHeight);
  popMatrix();
}

void captureEvent(Capture c) { //important OpenCV stuff
  c.read();
}

void mouseReleased(){ //check if mouse was released so the switch gets triggered only once
  if(mouseOver(230, 200, 80, 30)) pause = !pause;
}

boolean mouseOver(int xpos, int ypos, int rwidth, int rheight){ 
  //return true if mouse is over a given rectangle
  if(mouseX > xpos && mouseX < xpos+rwidth && 
      mouseY > ypos && mouseY < ypos+rheight) return true;
  else return false;
}

code formatter

Conclusion

The new software based posture sensor is reliable and has none of its predecessor’s flaws. You can easily run the program in the background while working on the computer and it will remind you if you are slouching into the chair or getting too close to the screen. 

About slouching: You should be aware that sinking into your chair and leaning back is not only more comfortable than sitting upright but does also relieve your spine. The software rather prevents you from hunching over which even unhealthier.

A very simple Posture Sensor

If you are reading this you are most likely sitting in front of a computer of some sort, maybe even slouching into the chair. Often we don’t care about our posture which can result in back pain and worse. If someone tells you to sit upright this is easy to do, but it won’t hold on for more than a few minutes. Now wouldn’t it be great to have a device that can remind you to improve your posture?

Ideas

There already are some very intelligent approaches on this topic. One of the best known should be lumoback or lumolift as the new version is called. As far as I understand, they use an accelerometer for a sensor to measure the angle at a certain point of the spine. While these sensors should be relatively accurate, they also need tiny electronics and clever power management to be wearable. Something that is really difficult to achieve with simple electronics and tools.

What is left is making a chair-mounted device. One solution would be using a few pressure sensors on the chair that measure the weight distribution. The downside is that a lot of those sensors are needed for a reliable result. And pressure sensors are not too cheap.

Something that changes significantly between good and poor posture is the position of the head. So what I finally came up with is a device that simply measures the distance to the head.

Hardware

The simplest distance sensors are ultrasonic or infrared sensors. I went with a SR-HC04 because it is cheap and sufficiently precise. There are no special requirements to the controller so I am using an Attiny85. A small piezo speaker provides acoustic feedback to the user. The only thing left is the power supply for which 5V are needed because of the ultrasonic sensor. You could easily use an USB port but I did not want to rely on a computer, 3 button cells deliver around 4,5V and should work for a few days.

Soldering everything together is very easy, the circuit does not need a single resistor although the piezo speaker could use one when longer used. The components are soldered onto a piece of prototyping PCB and some pin headers to connect to the sensor and power supply.

The whole device is not mounted directly onto the chair, but is connected to a piece of fabric that hides the battery and can be laid over the chair’s rest.

Software

The attiny85 is programmed using Arduino and luckily there is no difference in using the hc-sr04.  There are basically 3 modes: The configuration-mode waits until the user holds his head still and saves the distance of a comfortable position. After this the watch-mode starts, what compares the current distance to the saved distance. If your head is too far away it will sound an alarm. If you get your head back the alarm will stop immediately. If not the device beeps a few times and then mutes. After some time it enters standby-mode. This is meant for leaving the device alone, the sensor reads the distance only every few seconds during this to save energy. If you get back to your chair the configuration-mode starts again.

Programming this was not too difficult but there still are some strange bugs I cannot get rid of.

Conclusion

Apart from the minor software issues, the project was successful. This sensor can easily be built for less than 10$. 

But keep in mind: This is simply meant as a reminder to sit upright, not as a medical application.

Controlling Arduino with an Android device via Bluetooth

A Bluetooth module can add a lot of possibilities to a mobile robot while being relatively cheap and widely used. To control it you need a second microcontroller or a device like a smartphone with integrated Bluetooth which are complicated to set up to work with an Arduino

There already are some solutions in the Google play store but I could not find a decent remote that fit to my recent project. Most of them only have 4 buttons and a control delay of a few seconds. So I decided to create my own app.

The only real programming experience I had so far was in C and Arduino. Normally Java is required for android apps but learning it was not in my time frame. I remembered that last year I used Processing to write a small applet for an assignment. Processing is a programming language initiated in 2001 at the MIT. It has its own IDE, just like Arduino which is a descendant of Processing. Their user interface still look nearly identical. But while Arduino is based on C++ Processing sketches are translated into pure Java. You can easily create software for Windows, Linux, Mac and even Android using a syntax very similar to Arduino.

On my list of expectations for an app were: Multitouch, Two-way communication, decent speed and a nice UI. Multitouch turned out to be the biggest problem, after hours of research I found a working example program on the internet:

http://www.akeric.com/blog/?p=1411

There is also a nice tutorial on implementing Bluetooth and making it work with Arduino:

http://arduinobasics.blogspot.de/2013/03/arduino-basics-bluetooth-android.html

I put those two together and added the UI which uses the ControlP5 library for some parts. Making it really work took a lot of time, especially the communication part.

The app was meant for my 7” tablet, so the UI is only displayed correctly on a device with the exact same screen resolution. Last weekend I made some changes to the code to make it universally usable. You should be able to install and run the app without problems if you have a Bluetooth phone or tablet. A screen resolution of around 1024*700 is recommended, I could not get rid of some resizing problems yet.

Disclaimer: Use the provided software at your own risk.

Detailed Installation Instructions:

1.     Download the CTRControl.apk-file and put it onto the devices SD or internal storage:
        CTRControl.apk

2.    As this is not an official Google-approved app you need to turn on “unknown sources” in the Android developer settings tab

3.    Open the .apk-file with any file explorer and click on “install”

4.    Now the Icon should appear in the app menu.

5.    Before you start the app make sure the Arduino’s Bluetooth module is paired with the Android device. (It  should be listed in the Bluetooth settings)

6.    If you start the app it will prompt you to choose a connection. If there is a LED on the module it should light up after the connection is established.

On the side of the Arduino you need to read the data and write it to variables. I already created an example sketch which lets you control servos with the remote:
BT_example.zip

The “protocol” works like this:

Every slider has a range of 0 to 999. To distinguish them there is a letter in front of every transmitted value (e.g. a034 or c789). The data for the two Multitouch joysticks is transmitted all the time. Only if a slider value changes it will be transmitted too.  The buttons work in the same way, while 999 means true and 000 means false. The emergency off only transmits 999 to make sure that a double press doesn’t switch the system back on.

The whole code is not optimized and this is for sure not the most elegant way. I stopped at the point where it was functional, and the control delay is acceptable. It should even be possible to control a Quadrocopter. The entire app is surprisingly fast, it loads in an instant and takes not much more than half a MB of memory.

Plans for the future:

·          Multitouch for all sliders and buttons

·          Send accelerometer data for tilt control

·          Real two way communication

  Integration in the Google Play Store

An Attiny85 IR Biped Robot

Although wheeled robots would be better for beginners I wanted to build a legged robot. Mostly because there were no continuous motors in reach and my attempt to modify a servo failed miserably.

One of the simplest solutions is a biped robot that moves as it shifts its weight. Two servos are needed for the feet and another two to move the legs to go forward or backwards. It is boring to just make the robot walk until the batteries are dead. So I decided to use infrared to receive commands.

The whole thing would be way too easy if you could just plug the servos into an Arduino. A small 90ct Attiny85 chip has 5 (actually up to 6) programmable pins which is enough for the project. There is a nice tutorial about how to program the Attiny over an Arduino:

http://hlt.media.mit.edu/?p=1695

The real problem was finding libraries for servos and infrared:

http://forum.arduino.cc/index.php/topic,17965.0.html

http://www.cunningturtle.com/attiny4585-servo-library/

The infrared sketch works fine with my TV remote so I only had to change the codes.

On the hardware side I used aluminum and a lot of glue. The feet have rubber soles to prevent the robot from ruining the furniture. A prototyping PCB was cut in half to connect the few components. Four turnigy 1800A servos are running well with the 3.7V battery, there are no clocks or voltage regulators. The Attiny itself is not soldered in, it can be taken away easily for programming.

I would say the project was successful although I did not have the time to program it well (making turns is still difficult). The solution is also very cheap because of the small microcontroller. The only relatively expensive thing is an Arduino for programming.