Generally at temperatures bellow 50F Key Limes don't grow well.
I mean I've heard that, too, but my greenhouse was in the low 40s most nights when my key limes started their spring flush. The mandarins didn't start theirs until the lows got above 50°F almost a month later.
I also have had one potted key lime outside since early February, and it started its spring flush almost a month ago, with overnight lows regularly in the upper 30s to low 40s at that time. The Persian limes and Kishu mandarin that I also put outside at that same time have only just barely started their spring flush in the last week or so, about a month later than the key lime.