Is that dirt mulch? They'll grow in it but citrus doesn't like mulch. Looks like some kind of parasite or virus. I had tiny almost invisible slugs eating the sap once. Sprayed it with neem oil and got rid of them. It looks kind of like that to me like I'd try spraying it with neem oil.
Could be a nutrient issue. I sometimes crush up a centrum vitamin and mix with water for micro nutrients for all plants. Bone meal is good on citrus because they like calcium they respond well to bone meal but that's a very slow small release. The fertilizer you mix with water blue stuff is best or just citrus fertilizer. Which I would do bone meal and vitamin on top of it for micro nutrients.
If it's getting too wet just throw sand around the base so there's not a hole around it but level or a slight mound. Though if too dry a slight hole helps.