I like the idea except it gets too cold in 7b for olives. You may get lucky a few years, but eventually it will die without protection.
You could keep it as a potted plant and bring it into the garage when it goes below 20f.
You could also keep it outside in ground and protect it when there are cold snaps. I left mine outside and it made it on year, another year it died back to the base, and then last year it died to the cold.
They will fruit in a pot.