OK, my first married friends have had a baby, and now they're coming to New York, and we're having brunch on Sunday. The baby is only four months old, so I don't think we can count on calm baby behavior.
I've seen the Odeon recommended, but that might be a tad far. I live on the Lower East Side and it would be to stay around there. (The East Village would work too.)
On another note, I hate traditional brunch food (pancakes/omelets/waffles/granola) so I would at least need a decent burger and salad for myself.