From what I understand, jabos like a lot of water, especially the red and sabara varieties. How much are you watering the tree? I have seen some people keep trays under the pots to help keep the plants moist between waterings. It sounds like your tree isn't getting enough water. I know some people have also repotted a languishing jabo by up-potting and replenishing the potting soil, seeing a new flush shortly thereafter. I have also read jabos like to be pot bound.
I have a red jabo planted from a 3 gal a couple of years ago in muck in an area that never dries out and is frequently under water in the summer and it is thriving. I also have a grimal that I planted, in winter that is about 4 feet tall, in a drier area. The grimal doesn't really get watered often, but the area it is in is also a bit mucky and stays wet for awhile. The Grimal seems to be doing ok, but is not doing much because it has been very dry with very little rain these past few months. I expect it will flush out when the rains start.
These Jabos grow slow, so I'm not sure I'd worry about size since you can always trim them back. I would recommend using organic matter, mulch, and compost, if you decide to plant it out to keep the moisture in the ground and keep the plant from drying out quickly. I have heard planting the trees out can delay fruiting, since the tree will devote energy to building roots. Not sure if that matters to you, but it also sound like your tree is years out from fruiting.