Key lime is one of the quickest citrus to fruit from seed. Three years is the often cited, but that varies wildly based on climate and other conditions. I did not fruit Key lime in three years from seed, though I did get a couple of flowers after four years, which I considered a triumph considering it was grown in a container. The tree was too small to hold fruit, and I did not want it expending the energy flowering.
Key lime is true to seed and can be kept small when grown from seed, both of which are qualities not shared by all citrus. So, if you are willing to wait for it to mature or cannot source a quality grafted or cutting-grown tree, Key lime is a citrus you can grow from seed successfully.