Research

PhD

I earned Masters and PhD degrees in computer science at Freie Universitaet Berlin in 2002 and 2006 respectively. Back then, bachelor of science was on its way of being implemented in the German system. My PhD advisor was Raul Rojas, who was considered the authority on Neural Networks at the time. I was always interested in the more general properties of data and build a multimedia system that would enhance the audio and video quality of a life-recorded lecture for webcast. A side effect of that research is the image segmentation system SIOX (Simple Interactive Object Extaction) which has been part of several open source image manipulation programs (including GIMP and Inkscape) since 2005.

Multimedia

I was one of the initiators for the release of the YFCC100M corpus. The largest research collection of images and videos for research. It contains 100M images and 1M videos. I am still the maintainer of that corpus working with Amazon to make it easy to do research on content-based video search. Check out the link below to go to the Multimedia Commons portal.

I also have a track record in working on various video retrieval tasks myself. I participated in NIST's TrecVID MED evaluation, in the European MediaEval and I won the ACM Multimedia Grand Challenge twice with retrieval algorithms. I also wrote a book on geolocation estimation co-authored with my student Jaeyoung Choi. My main focus, however, has always been on audio algorithms. So many people work on visual data, so few people work on acoustic data. One topic I was always interested in was separating speakers and other acoustic segments other than speech or music. A cool demo can be found on the link below. I also built a speech recognizer board for the IoT and Maker community and worked on health projects.

Data Science

As the math-driven disciplne of computer science starts to become less important and makes way for the science-driven discipline of data science, we become more conscious about observations and how to predict them. As part of that, I am working with physicists and signal processors to craft engineering laws for machine learning. Check out our online demo of how to size a neural network here. Much more to come on this one.

Privacy

I also work on researching the privacy implications of uploading videos and images onto social media. Users often do not realize the implications of a photo and video and that this data can be correlated with other information. More background on this work can be found in the video on the bottom.