My nursery guy who is a fruit tree collector himself would answer that with an incredulous look and "Now, why would you go and do that?" Keep the fruit, you wont get much anyway so whatever you end up with is a bonus and tease for later on. I have two citrus trees of the same type and age I planted early last year. One I have pulled the buds that spring, the other I didn't. I ended up with only a couple fruit and both trees are almost the same size, the one that kept the fruit is actually taller and bushier.