hak ip belongs to a subgroup of lychees known as 'Mountain-type', which also includes the varieties Mauritius and Emperor. These lychees are generally a little bit more drought tolerant and cold tolerant, with a more dwarfed tree size and rougher texture on the skin of the fruit. (Mountain-type lychees are so called because they traditionally grew at higher elevations, as opposed to the more common 'Water-type' lychees that grew in lowland flood plains)
Is it possible the soil could be getting too waterlogged?
If that doesn't appear to be the problem, it could indeed be possible hak ip has a slightly higher chill requirement and that might explain the lack of vigor and why it is succumbing. I've read reports lychee seems to produce a little bit better in climate zone 9b than it does in zone 10 (assuming it still has enough heat the rest of the year, and not talking about desert climates either).
Maybe you could try putting it in a cooler part of your greenhouse during the winter to make sure it gets just a little bit of cooler temperatures ? (but if they are young seedlings you should make sure they don't get too cold)
I don't know, you could try treating the soil with willow tea, and maybe applying a little bit of dilute gibberellic acid to the plant in Spring.