Some of our absolute favorites when we visit Cape May in the summer (sorry I can't vouch for off season)
McGlades Backstreet Cafe (BYOB)
600 Park Center
West Cape May, NJ 08204
Freda's Restaurant (BYOB)
210 Ocean St Ste 1,
Tel: (609) 884-7887
We also love Doc Mike's Pancake house for breakfast (just over the bridge on the way in).
Tel: (609) 884-2855
Lousia's is also very good, but has limited hours in the summer, I'm not sure if they are open off season.