That's three years from being a medium tree ready to bear fruit.
I would decide how low you want the initial branch and prune all the growth above it to the nearest branch angling 60-90 degrees. You have to make that call, but as close as it is to the fence, you will have rats invading your tree.
I have made sure to prune off all lower growth on my trees to a height where I can put an apron on to prevent rat ingress up the trunk, but with a fence that close, that wouldn't help.