According to the bamboo section in my ECHO handbook, bamboo does best when you leave the natural leaf mulch in place because it keeps the soil moist and allows the plant to recycle nutrients. However, those same benefits would also apply to your fruit trees. I have seen a local person who rakes them up all the time and his bamboo is doing fine. He actually adds them to his tilapia pond, claiming that it improves the flavor of the meat. Presumably, this is because the tilapia are eating the algae that grows on the leaves instead of "bottom muck."