2 hours(!), that's your problem. You should amend the clay layer (heavily) with sand, perlite, or whatever you have that'll loosen up the soil and increase drainage. If your neighbors have healthy trees, the underlying soil layer(s) should support your tree. You just need to give it a little help in developing a strong root system. That clay is holding water too long and is tough to push roots through.