Generally speaking, a tree which produces light crops will produce larger fruits and vice versa.
Seedling trees can be unproductive like that. A neighbor of ours has a giant avocado tree (from seed) which produces 3 or 4 fruits per year despite wide scale flowering. Imperfect flowers maybe?