Quail is great.
Since you asked about the colleges. I can share my experiences.
SDSU has a number of interesting rare fruit all over campus, though parking is quite a bad situation.
You can find a number of loquats fruiting there at the moment. Look by the library, and in the quad between physics and engineering.
There are other interesting fruit trees there, but they aren't fruiting at the moment. Across college, there is a allotment area/community garden, it's by parking structure 6 I believe, there are a number of fruit trees planted in the stadium conglomerate soil in that area.
You can check out Cuyamaca College in Rancho San Diego, there a local CRFG member planted fruit trees which make interesting landscape trees. They also have a great horticultural department on that campus.
Speaking of CRFG, the San Diego Chapter has its monthly meeting in Casa Del Prado Room 101 in Balboa Park next Wednesday at 7.
Balboa has plenty of interesting plants, but you have to be willing to explore to find some of the more interesting selections.