Agree, good places are sparse in that part of town.
If one has to eat lunch near the Frick, I have been happy at Sant Ambroeus and E.A.T. on Madison. I’ve enjoyed coffee and cake at Lady M (Japanese crêpe cakes and other pastries) (41 E 78th) . I like the soups and desserts at Café Sabarsky inside the Neue Galerie. If you haven’t visited the Collection at the Neue Galerie, it’s worthwhile.
Lexington Candy Shop, if walking to or from Lexington, or the 6, has Egg Creams which are a draw for me. Food is pricey for okay diner /coffee shop food.
