We just returned from the 2nd trip down this season and while I love the keys, Key West is defintely more tawdry, crowded and less and less attractive. 7 Fish was recommended here and seconded in a local art shop. A long walk along Olivia turned up a place whose ONLY plus would be the food - NO CHARM, NO AMBIANCE. I need all three for an evening out. Mangoes on Duval was not a great choice but we were tired. The bar service was inferior (cant''t make a decent martini or old fashioned), the food was interesting especially the appetizer (peri peri shrimp). The ambiance is very nice and over all the evening was pleasant..
A little north of Key West at mile marker 19 is Mangrove Mamma - a very cute place with very flavorful fish selections (small servings though if that is important to you).
Something new in Key West seems to be a warning in the menus that a gratuity may be added to the check (an inquiry revealed it is for European customers who expect it to be included and do not tip otherwise). None the less it leaves one open to a tacked on fixed charge regardless of the quality of service. It leaves me very suspicious and willing to find another place.