Seashore is actually one of the faster growing garcinias. Ours has really taken off, and dwarfs all the other family members we grow. Seems pretty tough too.
Your main issue might be humidity. A greenhouse would be best.
If you start with a reasonably sized tree, who knows how long to fruiting. From seed, will take a long time, but could be a fun experiment.