Another trip to Beaufort, another restaurant to recommend!
The Whitehall Plantation Restaurant is on Lady's Island. There's a small driveway on the right immediately after you go over the bridge from Beaufort - keep your eyes peeled. Its an old plantation house on the water's edge. Get there in enough time to watch the sun go down.
I had She Crab Soup, and Lobster Savannah which were both excellent. I was astonished at the quality of the food - I wasn't expecting that in Beaufort.
I think it would definitely be worth the drive from Hilton Head if you're staying there and not Beaufort.