HeadUp - A Webcam Posture Monitor

Since receiving a lot of feedback about my posture sensor solution I was thinking about how to improve it. Despite its functionality there are a few problems like the continuous clicking of the ultrasonic sensor and the way it is installed on the chair’s rest.

Ideas

One way of fixing the clicking noise would be to use an infrared-based proximity sensor. These are available in different operating ranges and prices but sadly I could not find one that was cheap enough and had the right range for the project.

Improving the chair holder is really difficult, too. It would require a lot of 3D-printed parts and still not work on chairs with low rests.

This meant the old idea had to be discarded completely and the project was suspended until I came across this great blog post about a pc-monitor-mounted sensor:

http://www.paleosun.com/blog/entry/prototyping-fun-with-a-posture-alarm

That’s how I got the idea of simply using the webcam as a sensor. Most people have cameras on their laptops and even desktop PCs so there is no additional hardware needed.

Implementation

Because I already used it in other projects I went with Processing as a programming language again:

https://www.processing.org/

To detect the position of the head the software has to scan the webcam image. This is a nearly impossible task without the help of a powerful library. OpenCV is the solution:

http://opencv.org/

OpenCV is an Open Source library that contains computer vision functions and was originally developed by Intel. Face detection is just a small part of it.

The “OpenCV for processing” library by Greg Borenstein can easily be downloaded with the Library Manager in Processing and already contains some example sketches to start with.

The software I wrote works in a similar way as the Arduino code in the posture sensor. It scans the webcam image and compares the height and the size (~distance) of the face to limit values. If the limits are exceeded for more than two seconds the alarm will sound and then silence after a few seconds. If the user gets up from his chair or the face is not visible the alarm will not be triggered. The alarm itself is just the Windows error sound. The user interface consists of the live image in the background. Three buttons are for setting up the limit values and for pausing the alarm.

Finally Processing can export everything into a standalone application.

Download

If you have a computer with a webcam you can give “HeadUp” a try:

https://www.dropbox.com/sh/j9ezha6dx37jlla/X0ZRXMsmrJ/HeadUp.zip

Instructions: 1. Download HeadUp.zip

                       2. Unzip everything into one folder

                       3. Run HeadUp.exe

Compatibility: Currently only windows is supported. And there are a few webcams it will not work with depending on the drivers. If there are problems you should try running the application in administrator mode. The newest version of java needs to be installed as well.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

I am also releasing the source code so you can customize and improve it:

Copyright (c) 2014 Coretech Robotics

import gab.opencv.*;
import processing.video.*;
import java.awt.*;
import java.awt.Toolkit;//this is needed for the windows sound

int ypos; // height
int rSig; //distance
int almTimer; 
int trigHeight = 0; //height limit
int trigDist = 0; //distance limit
boolean alm, pause;

Capture video;
OpenCV opencv;

void setup() {
  size(320, 240);
  //The next lines are for intitializing the camera and OpenCV
  video = new Capture(this, 640/4, 480/4);
  opencv = new OpenCV(this, 640/4, 480/4);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
  video.start();
}

void draw() {
  getDistance(0);  // run the OpenCV routine
  if (trigHeight!=0 && trigDist!=0 && !pause) { //check if limits have been initialized
                                                //and if pause is off
    if (rSig > trigDist || ypos > trigHeight) { //compare values to limits
      alm = true; 
    }
    else {
      alm = false;
    }
  }

  if (alm==false) almTimer = millis()+2000;  //reset alarm timer if alarm is off
  else if (millis() > almTimer) { //check if alarm timer has expired
    if (millis()-2000 < almTimer) { //do this for additional 2 seconds
      Toolkit.getDefaultToolkit().beep(); //call the windows alarm sound
      delay(150); 
    } 
  }
  
//The following part draws the 2 buttons and checks if they were pressed
  
  textSize(14);
  fill(0, 255, 0);
  text("set distance", 18, 220);
  text("set height", 136, 220);
  text("pause", 248, 220);
  
  stroke(0, 255, 0);
  
  noFill();
    if(mousePressed && mouseOver(10, 200, 100, 30)) {
      trigDist = rSig+3;
      fill(0, 255, 0);
    }
  rect(10, 200, 100, 30);  
  
  noFill();  
    if(mousePressed && mouseOver(120, 200, 100, 30)) {
      trigHeight = ypos+3;
      fill(0, 255, 0);
    }
  rect(120, 200, 100, 30);
  
  noFill();
  if(pause) fill(0, 255, 0); // this part draws the pause switch
  rect(230, 200, 80, 30);
 
}

void getDistance(int interval) { //OpenCV functions
  //pushmatrix and popmatrix prevents the buttons from beeing scaled with the video
  pushMatrix(); 
  scale(2); // scales the video to the window size
  opencv.loadImage(video);
  
  image(video, 0, 0 ); // this draws the webcam image

  noFill();
  if (alm) stroke(255, 0, 0); //draw all lines red if alarm is active
  else stroke(0, 255, 0);
  strokeWeight(2);
  Rectangle[] faces = opencv.detect();
  int dist = 0;
  for (int i = 0; i < faces.length; i++) {
    println(faces[i].x + "," + faces[i].y);
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
    rSig = faces[i].height;
    ypos = faces[i].y;
    int delta = trigDist-faces[i].height;
    //the following line draws a second box with the limit distance
    if (trigDist!=0) 
    rect(faces[i].x-delta/2, faces[i].y-delta/2, faces[i].width+delta, trigDist);
  }
  //This draws a line at the limit height:
  if (trigHeight!=0) line(0, trigHeight, width, trigHeight);
  popMatrix();
}

void captureEvent(Capture c) { //important OpenCV stuff
  c.read();
}

void mouseReleased(){ //check if mouse was released so the switch gets triggered only once
  if(mouseOver(230, 200, 80, 30)) pause = !pause;
}

boolean mouseOver(int xpos, int ypos, int rwidth, int rheight){ 
  //return true if mouse is over a given rectangle
  if(mouseX > xpos && mouseX < xpos+rwidth && 
      mouseY > ypos && mouseY < ypos+rheight) return true;
  else return false;
}

code formatter

Conclusion

The new software based posture sensor is reliable and has none of its predecessor’s flaws. You can easily run the program in the background while working on the computer and it will remind you if you are slouching into the chair or getting too close to the screen. 

About slouching: You should be aware that sinking into your chair and leaning back is not only more comfortable than sitting upright but does also relieve your spine. The software rather prevents you from hunching over which even unhealthier.