Too long on tree then the fruit flesh turned light brown or rust and loosing its taste.
Less fruit on such a big tree could be due to Bill's cooler coastal climate is my guess. On the opposite side I had a hard time growing a Lychee tree -- 7 years old and it was only 3 feet tall and less than 1" trunk dia, gave me about 8 fruits total in those years and finally died. I think the location was too hot and no shade for the young tree.