What type of fence is it?
Passion fruit (dont bother with the sweet one)
Bananas
Jeff mentioned Eugenia. ..you tasted a Zill Dark Surinam and didnt like it. There are others but from whst i have seen of your likes and dislikes, i would pass on Eugenia if i were you
Ascerola/Barbados Cherry (with creative pruning)...though might have same issues as Eugenia. ..you should taste first
Black pepper
The list is endless if you consider things like lemon grass, hot peppers, herbs, etc.
You could try and espalier many things such as star fruit, persimmon, annonas...