Yes, remove the fruit. Snip them off. The tree looks fine. I planted a Day tree in January. The first thing I did was remove all the blossoms, and there were lots of them. I planted in full, very hot sun. The tree has had no growth flushes since the first one in spring. I don't think it likes such hot sun. My Monroe tree, on the other hand, loves the sun and is growing like a weed. Day seems to be a little fussy about growing conditions. At my last house I planted a Day tree in a windy spot, and it didn't like that either, and eventually went into decline. If it doesn't adapt, I may have to replace it. On the plus side, once Day is established, I've seen that it's a heavy producer. I know that Wurtz isn't fussy and also produces well.