Some fruit trees don't show their colors until 3 years after first fruit. For some varieties it's much longer.
Mandarins in particular are said to take 7 years to show their true colors. This does vary with rootstock, dwarfing rootstocks producing satisfactory fruit in much shorter times.
I've eaten so many gold nugget mandarins, and they are almost all as you described. Hard, pebbly, dry flesh, often with off flavors, I've tasted cilantro in GN. I'd never plant one if that was my only example, though it was repeated so many times.
GN is good when from a mature tree, sweet and juicy. Fruit trees are for the patient.