I live in Northern NJ which is not really known for food...but there are 3 local stores that I really love and keep me saint.
One is a store called Jerry's in Englewood. I read the owner, Jerry bought the store to use it as warehouse for his imported perfume/cosmetic business he was in years ago. He took a trip to Europe with his wife and saw all the gourmet food they were selling and thought to himself, why don't I do this in US? and the rest is history.
The store has tons and tons of European goodies from English clotted cream to duck fat to French butters to homemade butter to cheeses, pate, wine, prepared food to tons of imported olive oils, vinegars and pasta. It really is a heaven for gourmet food lover. Not only do they have great varieties, they have excellent prices. Their homemade desserts are excellent as well.
My 2nd store is French bakery Balthazar. This is actually their 'factory' but they have a small retail store in the front. Yes it's the same shop as the popular restaurant in NY. While I am not a big fan of their baguettes, I do love their potato onion bread and some of the sweets they create like fromage blanc tart, chaussons aux pommes and famous sticky buns.
My 3rd shop is bagel shop called Celebrity bagel shop in Tenafly. I honestly like their bagels more than some of the popular bagels in NYC like Ass a bagels. On the weekend morning, we buy them while they are hot....so hot you can't even touch them hot, bring them home, put some of their cream cheese and lox on and have it with coffee. They are large bagels but my husband and I can never have just one.