What you've said is very true. And there are a lot of domesticated plants (not just citrus) that reproduce in non-standard ways, especially with formerly non-self-pollinating crop plants adopting self-pollination as a strategy.
But in the long term, plants that don't prioritize crossing tend to be evolutionary dead-ends.