Yeah, I'm not sure if you actually wanted me to chime in or not; I can give you more advice on living here than visiting here, but chuck's right about the Airtrains.
They kick ass if you're not the kind of redneck who is afraid of the train/subway system here. If you fly into EWR, you can AirTrain it for $5 to Penn Station, which puts you on the 1/2/3/A/C/E subways which will get you where ever you want to get to in Manhattan (direct or via transfers). It's only about 45 minutes, you don't have to worry about traffic, and with sub fare, it's less than $15 a person.
Also, there's Carmel Limo who has decent flat rates for towncars from all the airports. We usually hook up family with a car thru Carmel when they come in to town.
If you'll be here for more than 4 days, get a 7 day Unlimited Metrocard. Having unlimited bus and subway will change your trip immeasurably.
What kind of stuff are you looking to do? As the others said, if you're looking to do the standard weekend-in-NY thing, stay in midtown because you can walk to all your favorite touristy crap and a ton of restaurants. Plus the sub system is really built to service midtown (every subway but the G runs through midtown at some point).
What kind of restaurants do you like? Who all is coming with you? Are you more interested in safe and expensive (the Upper East Side was a good call if you want to be near the Big Museums and Central Park) or eclectic and expensive (like the Village, which is where I would stay if I had the choice).
Stay away from the hotels downtown. It'll just piss you off when you realize there ain't shit to do down there (I'm talking the Financial District and WTC, not "Downtown" as in Tribeca, Soho, Greenwich, etc).
Also, people have told me there are people and buildings and things to do on the other sides of the East and Hudson Rivers, but I don't know anything about that. Frankly, I don't want to know. If people choose to live out in Indian country (Brooklyn and Queens), it sounds like crazy talk to me. What with all the ranch shootings and freezing tundras.