We haven't noticed Saigon Grill going downhill, but then we never had the whole fish. While fish freshness is unfortunately variable in places like this, the sauce quality is certainly a concern.
We eat at or order from here at least once a month, and generally have salads, soups, appetizers, and the grilled dishes. Our faves on the grills are the prawns, eggplant, and the chicken. The best salad is beef and papaya (that's not mangoin there!!!!!!). They used to serve it with a side of coconut cream, but no longer do. Too bad, 'cause it was great. The salad still helps to make living on the UWS a more pleasant experience. However, I have always been disappointed in their grilled pork chop and never order it anymore. New Pasteur on Mulberry has stolen my heart on this one and I can't bear to eat it anywhere else.
Another very fine dish is their sauteed beef cubes on salad. Simple, elegant (as elegant as anything can be when you are crammed in a fishbowl), and delicious. It only pays to eat the beef cubes there, though, since they don't travel well at all.
We only eat other dishes for variety , and gnenally find them ok, but nothing special. From your report, it doesn't sound as much downhill, as consistently inconsistent.