While the trees will be a bit more cold resistant once they're bigger, they don't adapt to colder and colder temperatures. It's just that they are physically bigger and more established and can take a bit lower temp because of that. A bit. Owari will be killed by persistent low teens unprotected no matter what.
In your zone, they absolutely will need winter protection regardless of how long you grow them. Maybe Swingle might not, I'm not sure about its ultimate hardiness.