HOAs wield a surprising amount of power. There's nothing that precludes them from prohibiting mango trees in the backyard. It all depends on the whims of your HOA board... They could easily prohibit you from planting edibles on any portion of "your" property.
Generally, the newer middle and upper-middle class neighborhoods seem to be the most finicky when it comes to planting edibles. For whatever reason, there is an inverse relationship between tolerance to fruit trees and income level. I think there is also an inverse relationship with age. So, the higher the income of the surrounding neighbors (or the older they are), the more likely they are to object to anything outside of status quo.
My HOA is little more liberal, and it helps that the median income in my area is in the lower-middle class range -- they are more busy worrying about how to pay the mortgage than counting how many square feet of st augustine can be viewed from the street. So folks here are generally receptive to residents utilizing their properties for more than simply cultivating seas of green grass. But if they ever started telling me to remove trees, I'd be listing the property for sale :-).