New York gourmet shopping
Keep walking around 15-20minutes until you hit the Chelsea Markets. A former factory has been converted into a strip of gourmet wholesale and retail food stores and cafes. You’ll find some interesting produce including a plethora of exotic mushrooms including lobster mushrooms, fresh porcini (a bargain at US$26.50 per pound! gulp..); as well as donut peaches; baby sweet pineapples; fresh white asparagus and some gorgeous bakeries and homeware stores.
At the Union Square Markets, you will find fresh produce including apples straight from the farm, the largest melons (or cantoloupes as they call them!) and some white eggplant I’ve never seen before. If you’re into the local sketches and paintings, a few artists have set up shop there too. 
And as a bonus, the largest Whole Foods store is on the other side too!
Balducci’s is an amazing upmarket providore, selling a wide variety of pre-prepared meals, fresh produce as well as a huge variety of groceries – exotic coffees and teas, oils, honeys, and a selection of bakery items. I would compare it to David Jones food hall perhaps? I believe it’s also mentioned in the movie “Just My Luck”. (bad movie, good taste).
Other places to go: Ferry Terminal, San Francisco