Hey all,

So I told some of you at dinner yesterday evening about what I do -- Natural Language for Car Navi. systems and media players (well, those will be the first two products, there will be others later on).

Video demo (Quicktime)

Video demo (Windows Media Player)

-----------------
The company (Speak With Me, Inc. -- www.speakwithme.com) is based off of research I did with a Grad student @ CMU -- I've worked on this stuff since 2001. It's a semantic parser, not a speech recognizer. We do utilize a speech recognizer, but fundamentally, we are the parser with extracts the meaning of your phrase. Overall results are more than 22% more accurate for music selection than trying to use a speech recognizer alone. We can adapt to many errors in the speech recognition, which also helps our system respond intelligently when it doesn't understand something.


----------------------
Features you'll see in the video if you look closely:

1. resolving confusion. There are a couple times I ask for a song name by the wrong artist, and so the system prompts me for that song I asked PLUS all the songs by the artist I asked. There is another example of prompting me when I have two songs with the same title but by different artists (Yes, I know Roger Waters is ex-Pink Floyd, but that is a live version by him).

2. Dealing with lots of noise... there are some parts that are really noisy, like when I ask for the beatles song, I do have to repeat myself once, but the system doesn't get a single utterance wrong! This is on a database of over 1000 songs. I too can't stand that text to speech voice for too long, thankfully we can tell it to shut up. There WILL be better Text to speech voices in the commercial product.

3. The system can tutor you on how to use it when it launches. A "dialogue" can also be used on launch to set up user preferences.

------------------
Other features we have now, but not shown in this video:

1. Nesting of queries. If I said "Play foxtrot" and then after the responses come with a lot I can say "Frank Sinatra" and it will narrow the query to "foxtrots by frank sinatra."

2. Backtracking. You could say "scratch that" or "I didn't mean that..." or orther phrases of that type to undo an action. Backtracking isn't included in music selection due to the simple nature of the task (as compared to car navigation).
-----------------

How's it work? Lots of really complex semantic parsing to determine your sentence structure and it keeps track of what you said before, too.
------------------

Cliff Notes
Go download the video and see what is coming to your car stereo in 2007

P.S. If there is a car stereo shop in Cali that would like to sponsor my car so I can afford to put this into my own car faster... let me know -- I will be attending the shows in NorCal and some in SoCal too.

 

Looks good, but does it still work well with road noise?

I programmed my own voice recognition system for music databases that I used in my own thesis and found it would easily get well over 90% recoginition in a controlled environment. However, as soon as you try to use it in the car, recognition drops off dramatically using the best microphones I could find. A headset microphone worked ok, but I think thats unreasonable to use in a vehicle.

The major problem in vehicles is overcoming the issue of road and engine noise and forget about using voice recognition with the windows open. If you make a demo video of your system actually working well in a real world enviornment, i.e. in the car driving at 60 mph w/ windows down, I'll see some real potential.

 

The road noise is, as you pointed out, a problem for the microphones. With the right array of mics, and some DSP trickery, you can cancel out both the road and the cabin noise VERY well. There was a phD thesis in Germany funded by Bosch that was a sister project to ours on noise cancelation in the car. We've got that end covered, but that is not our primary business, we are a software vendor, not a do-everything shop.

 

I don't understand a word of what you said but I'm still facinated!

Truly incredible!

 

I assume in your system you have an alternate voice to Microsoft Sam
Reminds me of the old Macs Id **** around with and have them say things

If road noise can be overcome successfully this would be amazing

 

We can choose from an assortment of TTS voices. We may even use celebrity voices.... but remember, my budget is low till we get external funding. My pals at Cepstral (www.cepstral.com) make TTS voices.

(speaking of which, I have a presentation to panasonic in 2 weeks!)