Not in my experience. I have had both and have a half dead longan brought down by cold and lychees with fruitlets on them currently. Potting a longan would be ok if brought in during the coldest weather. There's a commercial grove of lychee down the road from me as well. You could grow longan as long as you protected them, but I find them less hardy than mango.
Stuff that does work aside from them: Mulberry, several different apple varieties, bananas, supposedly pomegranates(I have four who do great, except they don't fruit), all-spice, blueberry, loquats, mango(with protection), raisin tree, melon berry, Surinam cherry, Cherry of Rio Grande(with protection), Barbados cherry in a pot, ice cream bean, jujube, pecan, and persimmon.
Not all of these do extraordinarily well but they survive.