The graft union and branch will normally grow, along with the fruit. If you only have one surviving graft, you will probably want to remove the fruit to be surer of getting plenty of vegetative growth. But if you have several grafts, even if they are only a couple of months on the big-enough tree, then you do have the option of allowing fruit to remain.