The fruit here is generally considered inedible but you have to take it with a pinch of salt; in the same place where you can grow bergamots, you can also grow premium oranges and lemons, so it isn't strange that such fruits hasn't reached its fame for its edibility.
I think that the toxicity problems are overstated, but apparently bergamots have chemicals increasing photosensitivity; so i wouldn't exaggerate with them, especially in summer or if i have to expose myself to much sun.
I think that zest can be used without much trouble. Here in Italy they are used to produce a liquor similar to limoncello, called bergamino but it's a liquor pretty rare to find. Best fruits for that use are the yellow-green fruits. There aren't many other uses for the fruits, if you don't take in account, of course, the oil extraction.
Last thing i want to add, is that bergamot is a generic name as "orange"; there are three differents cultivars of bergamot, that are seldomly nominated: they are Femminello, Castagnaro and Fantastico. First one has smaller fruit, but produces a lot of them; second one has bigger, bumpy fruits; last one has elongated fruits.
