Indeed - needs both chill and summer heat. It's the latter that prevents me from growing them here. They're quite temperature-hardy, but even the least heat-demanding cultivars can be hard to ripen in for the example the Pacific Northwest due to heat requirements.
Pawpaws are adapted to continental climates, where it gets both cold and hot. That said, there are California pawpaw growers (for example, Lagier Ranch in the San Joaquin Valley), although I've never heard of any growing as far south as San Diego. You can always contact KSU (the foremost experts in pawpaws) and ask. But I imagine chill will be a big problem (in the SF area, both chill and heat can be problems, depending on the location and cultivar)