Oscar, yes, it's growing in Southern California. One other thing to note, is that the leaves stay mostly evergreen. Even in winter, with occasional frost, the tree is never bare.
I also considered and looked in the walnut family, but after seeing the ripened fruit for the first time, I'm thinking it's not a nut? Last year it fruited for the first time, but I didn't realize that until I was pruning the branches and saw green fruit. Walnut leaves also seem much smaller and thinner, but similar shape.