By M.A. Rahman
Published: October 17th, 2018
Michael Mandel is an Associate Professor at Brooklyn College in the Computer and Information Science Department, where he has worked for the past four years. He is an expert in machine listening and has engaged in a myriad of research projects in the field, including a recent study on the migratory behavior of Alaskan wildlife using acoustic monitoring that will examine how these behaviors are affected by climate change and man-made sounds. This project has garnered Mandel a $500,000 grant from the National Science Foundation as a part of $3.1 million dollar effort in collaboration with researchers from Columbia University, the University of Alaska Fairbanks, and Colorado State University. In addition, Mandel is currently working on another NSF-funded project to study human listeners’ abilities to recognize speech in noise and use insights from these studies to improve the performance of current software on these tasks.
Q: As an Associate Professor at the BC CIS department, your job concerns the analysis of information through different mediums such as audio correct. So, in layman terms, what do you exactly do?
A: Yeah, my focus is on ‘machine listening,’ which is like machine vision but with sound to try and get computers to understand sounds and do useful things with sound.
Q: I hear that you working on a machine that’s able to distinguish what I think is human sounds from noise?
A: Yeah, I have several projects that try to separate speech from noise. So speech is a big problem for all sorts of voice interfaces: for hearing aids, for automatic speech recognition for Siri and Alexa, and for mobile phones. So if we could separate the speech from the noise we could make all those things work better.
Q: Growing up, how did you come to take an interest in this field? How did you take an interest in Computer Information Science?
A: I had an Apple IIC growing up and my grandfather taught Computer Science, he [also] taught science at a middle school, so he got me started on programming stuff. Even as a kid, just playing with stuff, writing games in basic and LOGO. At the same time I was also playing music; I was playing the saxophone when I was in high school and still in college. One day in college I was looking around a lab and I saw a poster for a research project that was trying to evolve synthesizer patches using genetic algorithms. Basically it was combining music and computer science, which were the two things that I was passionate about so I walked into the lab and said, “Hey, do you guys have any jobs or anything?” [chuckles] And they said, “Sure, why don’t you come work with us for the summer?” So I worked with them over the summer and I continued to work with them for years, and one thing led to another, and here I am. [Chuckles]
Q: And are you due to receive a grant from the National Science Foundation for some of your research or have you already received it?
A: So it already started as of September 15th. The overall grant is in collaboration with a few other schools.
Q: So it’s not just you working on this project?
A: No, it’s not just me. The five hundred thousand dollars is for us to spend here at Brooklyn College, but the whole project is over three million dollars and that includes Columbia [University], Colorado State [University] and University of Alaska Fairbanks. So there are different teams on different sites, each one with complementary expertise.
Q: Okay and you are studying, correct me if I am wrong, “the movement of animals in Alaska.” So how exactly do you go about doing this? I hope you aren’t just sticking a walkie talkie next to a fracking plant in Alaska and hoping for the best.
A: No, so there are these sort of ruggedized recording units that we’re going to get – they’re not too expensive at like 200 bucks each – but it’s basically got one or two microphones, a big battery, a big memory card, and a little processor, and you set them to record a few minutes every hour or so. Basically you put it out there, and there’s some armor on it to protect it from the elements and animals. So we’re going to put them out there, leave them for the season and then pick them up again in the fall, I guess. We’re going set up a hundred of these across the Alaskan wilderness; we still have to figure out where to exactly put them but we have plans. There’s a few ways to set them up. We’re going to set them up to monitor songbirds, waterfowl and caribou; although, caribou I don’t think make much noise, so we’ll have to set up some cameras for the caribou.
Q: So why Alaska, and why these animals for that matter?
A: That’s a good question, these areas are being affected a great deal by climate change, much than any other state and as far as I know it has the most pristine wilderness in the US. So we want to see how it’s changing and how we’re affecting the animals up there. Basically the grant is based on some preliminary work that my collaborator in Columbia did; they set up these recording devices over five years at like half a dozen site and observed when the songbird migration arrived and they found that it was getting earlier by a few days over those five years.
Q: And you’re looking to do similar work but on a bigger scale?
A: We are looking to scale this up and find better techniques for analysis, my part of this project is more so the data analysis. We’re planning to get a hundred thousand hours of audio, so I’m getting several tens of terabytes of data.
Q: Won’t that be quite time consuming?
A: Well that’s why we’ve got the computers to do it.
Q: So the computers do the data analysis?
A: Yeah, yeah that’s CIS, we’ve got to tell them what to do. So the idea is that there’s this game that I’ve developed previously applied to music. Basically, people come in and listen to like ten seconds of a song, they describe it with whatever words they want and when their descriptions agree with other people’s descriptions they score points. They have to be relevant to the sound and if too many people have used that same words then they don’t gain points, so they have to be novel as well. For music I was able to use able to collect about a hundred thousand associations between words and clips and use them to train automatic taggers use that you could use it for certain vocabulary words it could propagate those words to new music that no person has heard. So the plan for this is to do something similar with the environmental recordings, to set up a game, people could come in and listen and then we use their descriptions to train automatic algorithms to label them.
Q: For a lot of people that already know climate change exist, they might ask themselves, “What’s the point of this research, why study the migration of songbirds, what’s the important data we could get out of that?”
A: Well in addition to climate change, there’s this oil and gas exploration that starting to happen in this area just recently, so we’re going to go in to get a baseline before that starts happening. While its going on we can go and measure the effects of how that is affecting the wildlife that are there. My understanding is that there was a ban on oil exploration in the Arctic National Wildlife Refuge that was lifted by President Trump and that going into effect soon.
Q: What advice would you give to someone interested in this manner of work or majoring Computer Information Science?
A: The first step would be to get in touch. As a part of the project, I’m going to be hiring someone to be our graduate researcher, as I’ve had a bunch of graduate researchers in past projects. I find it to be a good way for students to learn; I mean, class is different than being an actual software engineer, so I think this is closer to the real world experience. It’s good for the resume; it’s good for all sorts of things.