I don't know what instructions you followed, but you do need to start with those that open and are becoming brown, because those are the ones with the pollen. They start off female, and become male, and then fall off. So, you need to wait for one to become old enough to open all the way and have pollen inside, and then take that pollen and put it up inside a female one that is still green but just opened enough to get the pollen in it. If you only have one tree, this is challenging, especially if the tree is young. It may not hold them all either, if the tree is too young to hold many fruit yet. What I did last year is to put a few old male flowers in a tiny ziploc seed bag so I would have pollen available, and then use the pollen on the new female flowers, because the male and female were never open at the same time. It seemed to work, because I got a few fruit, and got no fruit at all except those that I hand-pollinated. I expect to have more than one tree flowering this year, so that will help. But mine are still young and relatively small. I don't have any flowers yet on my trees, so another problem could simply be that's it's a bit earl in the season, and it's also very dry ot and they will probably hold better when there is a lot more rain.
So maybe something I mentioned in here will help you. I know last year I had a lot of trouble figuring this out, but after a number of tries I finally got it.