• Description

• Inspirations
• Favorites
• OpenFrameworks
• Past BBC Events
• Others

• Project Brain

• Research
• Books
• BBC
• Technical

• Requirements

• Development
• Prototype 01
• Prototype 02

• Outcome
• Summary
• Prototype 03
• My Brain

• Further Development
• Technical
• BBC Submission

• Critical Commentary
• Initial Thoughts
• Tron Inspiration
• Problems Encountered
• Afterthoughts

• References

• About

Language and IDE
Finding the right programming language was harder than I thought. Java, C, C++, Flash, Max/Jitter or something else all together… There were a lot of choices and I had to consider pros and cons of each of them. Max/Jitter immediately failed my test, as it wasn’t a free language to use. Flash was something I used during my first semester and didn’t feel like doing the same thing again. Java I had done extensively during my first degree, so I immediately felt more drawn to C++ or C which I only knew a little. The next problem to think of was the IDE (Integrated Development Environment). It had to work with the language of my choice and be free to use because I like Open Source :). Before I knew it though, I had found openFrameworks and that recommended Code::Blocks as an IDE. I liked both from the start and decided to use them for my project.

Addons
Fairly soon after I started coding I realised that I needed a lot of addons and there was a big choice of them out there. The forums of the openFrameworks community are very vivid and it was hard to get hold of the newest version of an addon. Eventually I found some of the developer’s SVN addresses and had access to their up-to-date code. I played with a lot of different addons but ended up with only a few essential ones. I obviously needed one to integrate OpenCv into my project, so ofxOpenCv was an obvious choice. I also added ofxContourUtil which added some more functionality to my contours and ofxBobTracker to add my blob histories. I did have to rewrite some of that code though, just to add some custom functionality like being able to have random colours for each blob. I also picked ofxVectorMath and ofxBox2d for the collision and made a few custom changes.

Blob Detection
During my actual coding I came across several major problems, the first one being my motion tracking setup. At first I didn’t have any automatic background subtraction but had to manually set the background and then look at the difference of that frame to the new one, to recognise any motion in the video. Finding a function that allowed automatic background subtraction was crucial for my project to run nicely, as the application for the BBC Big Screen obviously had to run unsupervised too. I did find the Gaussian mixture model to help me do that. Given a few parameters it would work well enough on subtracting the background with a good and adjustable learning rate. Finding the right threshold was quite hard at first, I couldn’t quite get it to work smoothly but by adding a blur to the foreground image it generated, I could get reasonable results with my blob detection.

Contours in physics world
I then concentrated for quite some time on getting the contours to work as a linestrip within my physics world, so that they could collide with the little floating balls. I managed to get it work reasonable well, even though it was very slow.

Centroids
The issue with the linestrips of centroids was that the centroid of a blob would change location when the blob merges with another one. In order to get that working properly, I would need to change the behaviour of blobs merging. Or rather stop them from merging altogether. That is easier said than done, so it's on my list of to-do things.

Performance
My biggest problem overall though must have been the performance issue. I used to write and test my code on fairly small videos with just 320x240 size. As soon as I increased that size to 720x576 (the resolution required for the BBC Big Screen), then my performance would drop to being unacceptable. My frame rate went down to about 2 frames per second! I had to play with my code again and again and make little changes here and there to reduce the calculation time for the computer and make the code run faster. I changed the parameters for my contour finder for example. I narrowed down the range of blob sizes to detect and also changed the amount of blur. I also decreased the size of the actual video frame that my algorithm was looking at and then increased it back afterwards. This process would naturally decrease the amount of points I could get in my contours but got rid of the need of simplifying them to begin with. I still think the performance is not really acceptable and needs to be improved a lot but that’s for my further development now.