This is a question that Paul Haas, music director of the Symphony of Northwest Arkansas (SoNA), has encountered many times over the course of his career.  He gives us an answer, in layman's terms, over on his blog - Paul Haas.  He's a very witty, entertaining writer, and you will enjoy reading his explanation to this oft-asked question. 

"So invariably, this happens. I’m at a party or gathering of some sort, and just as I’ve stuffed my mouth full of the ubiquitous cocktail peanuts, someone who’s found out that I’m a conductor will approach me and ask, 'So, no offense, but what does a conductor do? I mean, it doesn’t even look like the musicians are watching the conductor!' At which point, I point to my mouth to indicate that I am somewhat indisposed." 

Click here to hop over to Paul's blog for the rest of the story.