I agree with bsbullie unless you want to stunt your tree. I have many Florida grafted mango trees and they grow so slowly that I use them for scions for grafting onto seedlings and Lavern Manilla trees. If you are doing similar, go ahead and let it hold fruit but if this tree is a keeper, I would remove the fruit.
I do see that the fruit is quite large already but can't exactly tell just how big the remaining fruit are. If they are close to full size, I guess most the damage is done already.
I removed all the mangos from my Florida grafted Glenn mango tree for the first 4 years and finally let it hold fruit last year and my tree is only about 4 feet tall. Because I let it establish for so long before letting it hold fruit, it is finally starting to show some growth even though it is on Florida rootstock.
By the way, your tree looks beautiful. What other mango varieties are you growing?
Simon