REALbasic University Resources: RBU: Glossary Defines common REALbasic programming terms   Archives Previously published columns Translations: Dutch Courtesy of Floris van Sandwijk   Japanese Courtesy of Kazuo Ishizuka   Chinese Courtesy of Dong Li   RBU Translation Guide Information on Translating RBU into other languages Books: Matt's Book (2nd Edition!) Ideal for experienced programmers   Erick's Book Best for beginning programmers Websites: Mother Ship The publisher of REALbasic   RB Webring Links to hundreds of REALbasic websites   RESExcellence Another REALbasic programming column   REALbasic Developer Magazine The premiere source for REALbasic instruction. REALbasic University is Sponsored by Make your Mac do what YOU want it to. Create games, utilities, cool Mac OS X tricks. Download REALbasic now and create your own software.

REALbasic University: Column 100

OOP Intermission: iConversation Part One

In our last lesson we were in the middle of exploring the design problems associated with progress bars. Put that on hold for a minute -- we'll get back to it shortly.

Meantime, as an apology for missing so many lessons recently, I'm bringing you a fun little intermission. This will be a quick two-parter. I think you'll like it -- we're going to converse with our Macs!

Last week while I was at Apple's Worldwide Developers Conference in San Francisco, someone from Apple's speech recognition department was wanting to know if REALbasic could support speech recognition. Of course REALbasic doesn't support speech recognition directly, though you could access it via OS declares.

Declares can bit complicated to figure out, however, so I wasn't enthusiastic about that. But when the guy showed me how to do speech recognition via AppleScript, I was able to quickly adapt it for REALbasic. It's really cool!

Recognizing Speech

To test out this remarkable feature, we're going to create an application called iConversation. It's a simple Eliza-style app that will let you have a voice conversation with your Mac.

To do this, we're going to create an AppleScript that talks to Apple's speech recognition server. We'll communicate with that AppleScript from within REALbasic. The result will be an app that talks and listens to you, just like a real person.

A Few Caveats

While not especially complicated, this project does have some requirements. It currently will only work under Mac OS X (a Mac OS 9 version appears possible, but I didn't take the time to develop it). You'll need Script Editor (included with Mac OS X; usually within the AppleScript folder inside your Applications folder).

It should be obvious that iConversation requires a Mac with a microphone and sound input (duh). That should include most recent Macs. I tested this on my PowerBook G4 with the built-in microphone and it worked great, but I can't vouch for every system out there. Third-party sound input devices (like Griffin's iMic) should work fine as well.

I also tested this with REALbasic 5.1, though I don't see why it wouldn't work with earlier versions of REALbasic. Let me know if you run into problems.

One key disadvantage of using the AppleScript approach is that your application stops working until the AppleScript is finished. This means while your app is in a "listen" mode it can't do anything else. For some apps that might mean this method of speech recognition is useless (for instance, if you wanted your app to be controlled via voice). A different approach (accessing speech recognition via AppleEvents or declares) might work better in those situations.

Setting Up Speech Recognition

One key thing I learned from the Apple rep is that to get speech recognition to work properly, it is critical to adjust your settings for your current environment. I never knew that and when I first got my PowerBook I remember being disappointed by the poor speech recognition.

To adjust your settings, go to System Preferences and click on the "Speech" icon. Click on the "Listening" tab. Your window should look something like this:

At the lower right corner is a strangely labeled button called "Volume..." Click on it to bring up this window:

This lets you set the sensitivity of the microphone, which is critical as if it's too sensitive the Mac picks up too much background noise and that confuses the speech recognition system. The Apple guy did this at WWDC and even in the crowded exposition atmosphere it worked surprisingly well.

Adjust the slider and test it by saying the phrases on the left. Each phrase will blink when it's recognized. When recognition is optimal, you should be able to speed down the list saying each item quickly and having it instantly recognized.

You should redo this setup whenever your sound environment changes. For instance, if you first do it with no background sound, but later have the sound system playing music in the background, you'll need to do it again. If your sound environment is similar, it should work. If you experience recognition problems, go back and adjust the microphone until it's working well.

AppleScripting Speech Recognition

The key to using AppleScript with speech recognition is to use a program called SpeechRecognitionServer (note the lack of spaces in the name). Your system should know where it is, but if not, do a search for it. (A search is highly recommended: it's buried about a dozen folders deep.)

If the following AppleScript won't compile because it doesn't know where SpeechRecognitionServer is, that tells you you'll need to find it for the system.

Here's the AppleScript that's at the core of iConversation:

Here it is in text form in case you want to copy it:

on run {theString}
  set thePhrases to every paragraph of theString
  tell application "SpeechRecognitionServer"
    listen for thePhrases
    return the result
  end tell
end run

After you've got this in a script, make sure it compiles by clicking the checkmark icon in the Script Editor (labeled "Check Syntax"). If that works, save this script as listen.script. Be sure to save it as a compiled script (not text or application).

While the above script is fairly simple, let me explain what's going on. The first line lets us pass a parameter to the AppleScript when it runs. That's critical because without that we couldn't dynamically tell the script what speech phrases to listen for.

The next line took me some time to figure out. If you're familiar with AppleScript, you know it has a List data type. The SpeechRecognitionServer application is expecting a List of strings as the commands to listen for. Unfortunately, REALbasic doesn't support the List data type: the only types we can pass to an AppleScript are integers and strings. Therefore we must coerce the string passed in theString into a List. By separating each speech command with a carriage return (chr(13)), we can then tell AppleScript that each "paragraph" of our string is a List item. The resulting List is put into thePhrases, which gets passed to SpeechRecognitionServer a couple lines later.

Finally, we return the result, which contains the text of whatever phrase the user spoke. By examining what's returned, iConversation can then tell what the user said!

Note that under Mac OS 9, you can use the same script but talk to "Speech Listener" instead of "SpeechRecognitionServer" and it should work. I didn't take the time to get it working under OS 9 -- under Classic it wouldn't work because it said Speech Recognition wasn't installed, even though it was. Perhaps speech recognition doesn't work in Classic.

iConversation

Now let's create a new REALbasic project. The first thing we'll do is drag "listen.script" into the project window. That will add a "listen" command to our project. Save the project as iConversation.rb and we're ready to code.

To begin, let's open Window1 and add some stuff to it. Try to make it look like this:

The ListBox should be named phraseList and the button startButton.

Next, we're going to add our own class to the project. Go to the File menu and choose "New Class" and name it speechCommandClass. Double-click on it and add the following properties (Edit menu, "New Property..."):

Each instance of our speechCommandClass will contain a speech command (what iConversation listens for) and the reply (what iConversation says in response). There's also a stop property, which is used to halt the conversation (by default the program waits for new speech input after each phrase).

The speech commands will be stored in an array within Window1. Let's add that now. Open Window1's Code Editor (press Option-Tab while Window1 is selected) and add a new property (Edit menu, "New Property..."). The new property should look like this:

  
  theCommands(0) as speechCommandClass 
   

Perfect! Now let's add the code for our "Start Listening" button. Double-click on the button to open its Action event. Put in this code:

  
    dim thePhrases as string 
    dim i, n as integer 
     
    n = phraseList.listCount - 1 
    for i = 0 to n 
      thePhrases = thePhrases + phraseList.list(i) 
      if i < n then 
        thePhrases = thePhrases + chr(13) 
      end if 
    next // i 
     
    respond(listen(thePhrases)) 
   

All we are doing here is taking each line from phraseList and putting it inside a string variable, thePhrases. Each line is separated with a carriage return. We only add a return if it's not the final item, since the last item doesn't need a return after it.

Finally we call a method called respond and we send it the result of our listen AppleScript (which is passed our thePhrases string).

We might as well go ahead and add that method. From the Edit menu, choose "New Method..." and name it like this:

We're almost out of space for today, but let's quickly add one more method (File menu, "New Method..."). Name it speak with theSpeech as string as its parameter. Here's the complete code for it:

  
  sub speak(theSpeech as string) 
     dim i as integer 
      
     #if TargetCarbon 
     Declare function SpeakString lib  "CarbonLib" (SpeakString as pstring) as  integer 
     #endif 
      
     #if TargetCarbon 
     i = SpeakString(theSpeech) 
     #else 
     msgBox theSpeech 
     #endif 
  end sub 
   

This simple method uses a declare to access the Mac's built-in speech routines. Assuming you've got a Mac that's capable of speech, whatever text you send this method will be spoken (otherwise it will display the text in a dialog box).

Try this: add a button to Window1 and its Action event put this:

  
  speak "Hello, I am Macintosh" 
   

It should speak the text when you click on the button.

That's enough for today. Unfortunately our app doesn't do much yet, but we'll get it fully working next time.

If you would like the complete REALbasic project file for this week's tutorial (including resources), you may download it here.

Next Week

Have a conversation with your Mac!

Letters

This week we've got an interesting letter from a Windows REALbasic user! Matt Eastburn writes:

Hi Marc,

Thankyou for your wonderful tutorials - I am learning a lot from them - but I am actually usin RealBasic 5.1 on windows XP and am having at the moment one little problem, and here it is:

It's with Tutorial 57 - and to do with saving the kiddies art work file:

  
  dim i as integer  
  dim f as folderItem  
   
  i = 0  
  do  
    i = i + 1  
    f = getFolderItem("").child("Picture " + str(i))  
  loop until not f.exists  
   
  return i 
   

This particular part of code does not seem to work for me - the part where you check "f = getFolderItem("").child("Picture " + str(i))" - because whenever I run the program it always saves "Picture 1.bmp" - do you have any idea why it would not work???

There was something in the REALbasic Laguage reference talking about the "child" as in - Mac will automatically fix aliases - and windows doesn't - What does that mean???

Matt Eastburn

Thanks for the question, Matt! As REALbasic for Windows is a recent product I haven't had a chance to test my old code with it and there are probably a number of small but annoying bugs. If you use REALbasic for Windows, please let me know of any problems.

There are also problems with older projects not working with REALbasic 5, even on the Mac. I have plans to revise my old columns to fix these problems (I'm even contemplating buying a PC to help with Windows debugging), but that probably won't happen until the end of summer at the earliest. Meanwhile, let me know if you see problems.

As to Matt's problem, I think I see the bug.

I haven't tested it, but I suspect it has something to do with the file extension. Mac's don't require them (though under Mac OS X they are encouraged), but Windows wants them. It looks like REALbasic is adding a ".bmp" automatically. Since our loop is looking for a file named "Picture 1" not "Picture 1.bmp" it doesn't find it and thus i never gets greater than 1.

Try changing the middle line to this:

  
  f = getFolderItem("").child("Picture " + str(i) + ".bmp") 
   

That will add the extension and force the loop to look for files with the extension. Hopefully that will work -- if you can let me know if it does or doesn't, I'd appreciate it.

Of course the above is a Windows-only fix. To do the fix universally, we should do the code like this:

  
    #if targetWin32 then 
    f = getFolderItem("").child("Picture " + str(i) + ".bmp") 
    #endif 
     
    #if targetMacOS then 
    f = getFolderItem("").child("Picture " + str(i)) 
    #endif 
   

This conditionally checks to see which platform we're compiling for and changes the code appropriately.

Matt's other question was about aliases. Aliases are called "shortcuts" under Windows. They are pointers to a file in a different location. On the Mac a folderitem's child property to an alias will resolve to the original file (the file the alias points to), while (apparently) it won't under Windows (it will return the shortcut file itself). That's a significant platform difference and might be good to keep in mind.

Each week we answer select reader questions, and we're always open to ideas for future columns. Send your questions to . (Keep your questions simple and specific. General queries like "How do I write my own web browser?" will be neglected.) Your question won't be answered immediately, but will be answered in a future column. (If you don't want your correspondence published, just be sure to indicate that when you write. Otherwise it's fair game.)

About the Author
is an author, philosopher, graphic designer, photographer, film director, soccer fanatic, and programmer (among other things). He writes for MacOpinion, runs his own software company, Stone Table Software, which sells the revolutionary Z-Write word processor, and is Publisher and Editor of REALbasic Developer. He lives in Northern California with his cats, Mischief and Mayhem, and is rapidly running out of free time.

See the REALbasic University Archives

.