If you're using parafilm or buddy tape, don't remove it, period!
The buds will easily push through. This is how it's supposed to work.
Parafilm will degrade over time and doesn't necessarily ever have to be manually removed.
However, by the next year I normally remove any that is left.