FYI, I give mine irrigation sparingly in winter mostly letting whatever rain we get supply water unless it's been really dry. When the fruit is getting large I may supplement rain with irrigation between rains. In summer I generally do not need to irrigate much but if I do, I feel there's no need to soak them.
They have been in ground 7-8 years now with several generations of pups giving me fruit.
Early on I learned too little water will cause die off too, so I gradually increased water to a happy medium. I'll bet your die off is from excessive water, not close proximity of neighboring plants.
Anyway, I could be off on this as I haven't been growing them that long, but it seems as if too much water will bloat the fruit too fast making the stem collapse under the weight of the fruit before the stem can mature enough to hold the fruit, too little water and the stem may also wilt letting the fruit sag it.
My Carambola are my garden water sentinels, they seem to be the most sensitive plants I have to lack of water. If any show the least sign of wilting, I water EVERYTHING. Luckily I'm on sandy loamy soil that holds water, and my yard is now more shaded from wind and sun by trees, were it not I might have to water more often.
