I know nothing about homeowner's associations but I reckon this entire thing is bs and your trees are basically being scapegoated. Make a case.
Morally speaking your neighbor/neighbor's landlord is strongly in the wrong, hopefully the homeowner's association sees this the same way.
Plus, isn't your neighbors spraying your yard without your knowledge a safety hazard??? And if your trees are not attracting rats then surely you could argue that your neighbors had no evidence that your fruit trees were causing damage and what they did is totally irrational???
You could potentially say that your neighbors risked your safety and damaged your property on an assumption that you potted fruit trees were attracting rats.
Perhaps I got carried away, but good luck Alex, with your poor poisoned fruit trees and whatever action you wish to take.
Rob.