Prototype 02 : Fania

My second prototype built on the first one but improved a lot of the ideas and techniques used in it. It also included 6 video frames in one large window, simply because I wanted to improve a lot of the previous ideas before specialising in one specific technique. Generally, this prototype mainly focused on optimising the background subtraction and threshold to get the best blob contours as possible. To smoothen the contours, I added an adjustable blur effect. Also, the threshold could be changed manually in the running application. I played with colour effects, like having a random colour for each blob. I also added a transparency attribute for the blob filling. I experimented with the history of blob contours and with the blob centroids (the central point of each blob). In terms of programming I think for this prototype I played a lot with loops.

The first video (top left) was the foreground of that original video with added blur or adjustable threshold.
The second video (top middle) showed all of the blobs as a pink rectangle and the contour in light blue.
The third video (top right) was the original video with the contour history trailing behind the current contour.
The fourth video (bottom left) was a test of going through all pixels of the video and applied some effect to it.
The fifth video (bottom middle) showed the filled contours with a random colour for each blob and transparency set to around 50%.
The sixth video (bottom right) was a test of drawing the centroids and their history to form a sort of line. But whenever two blobs merge when they get close to each other, the centroids would jump to another place and the line got messed up. It still worked quite well for some blobs though.
At the bottom you could see the current amount of simplification for the contours, the blur amount and the threshold and there was some space to draw by clicking and dragging the mouse.
There were also the customBalls to add by clicking space or b on the keyboard. The drawing line was added to a linestrip to be part of the physics world so that it could interact with the balls.

Below are the source files of prototype 02 written in pseudo code for easier understanding.

main.cpp

include stuff

// Main
//--------------------------------------------------------------
open window of size 1050,800
run main c++ file (testApp.cpp)

testApp.h

Include addons like ofxOpenCv, ofxVectorMath, ofxBox2d, ofxContourUtil and ofxBlobTracker

// COLLISION

customBall class which sets size and colour of balls

name physics worlds

name the linestrip for the contours

name the linestrip for drawing line

name the contact handler for the physics worlds

name the vector for the customBalls

name the vector for the balls

name variables for simplification, threshold, font, image, mouse location, and various others

name colour images to store the video pixels in

name grayscale images to store the foreground and background pixels in

name contour finders to store the contours in

name video grabber for the webcam or video player for movie

name the vector to store the points of the drawing line in

name the blobTracker for the history of contours and blobs

testApp.cpp

// Setup //-------------------------------------------------------------- set background colour

initialise video grabber
(or load movie)

allocate memory for the original video, the foreground video and background video

set gauss parameters
create gauss background

load font

// DRAWING
set number of points of drawing to 0 to begin with

// COLLISION
allocate memory for original video and contour frame
initialise physics world
enable gravity
set bounds to contour frame only

// BLOB TRAILS
setup blob listener

// Update
//--------------------------------------------------------------
set window title to current framerate for reference

store pixels of input video (camera or movie) into respective frames
update background and foreground models including blur and threshold values
find contours using the foreground model

// BLOB TRAILS
track blobs

// Draw
//--------------------------------------------------------------
draw foreground video at 20,40 (top left)
draw background for contours
draw contours at 360,40 (top middle)

// TEXT
write text

// HISTORY
draw original video at 700,40 (top right)
draw history of contours at 700,40 (top right)

// VIDEO
draw background for pixel video
draw pixel effect of video or live feed at 20,320 (bottom left)

// RANDOM COLOR CONTOURS
draw background for contours
draw and fill contours with random colours at 360,320 (bottom middle)

// CENTROIDS
draw background for centroids
draw centroids and their history at 700,320 (bottom right)

// BALLS
draw customBalls

// DRAWING
draw line on mouseclick at mouse location

// KeyPressed
//--------------------------------------------------------------
increase or decrease simplification of contour by clicking left or right
increase or decrease blur by clicking up or down
increase or decrease threshold by clicking pageup or pagedown
draw big balls by clicking space
draw small balls by clicking b

// MouseDragged
//--------------------------------------------------------------
add current point of mouse to drawing array and linestrip

// MousePressed
//--------------------------------------------------------------
clear drawing array and linestrip on right click

// MouseReleased
//--------------------------------------------------------------
stop adding points to the drawing array and linestrip