The disease issues aside, 15 fruits and then getting less? 15 fruits per tree makes the tree a waste of space, I couldnt imagine keeping it with so little number of fruits. Getting less wohld make me get the chainsaw out.. Is it a seedling?
Do you see any signs of borers on the main trunk?