My Nam Doc Mai is flowering in September. Should I allow it to continue to try to fruit or should I tip off the flowers? I just took off some vertical branches, see pictures. This year's crop came at 2 different times June and August. I had very little fruit compared to last year 4 vs. 9. I planted this tree 4 years ago.