Hello,
In my experience, I've been able to get a larger plant growing out of a fabric pot than an identical-sized plastic pot-- meaning you don't need as large a pot to get a plant to bear fruit, and you don't need to repot quite as often. I would say it is still important to start with a smaller pot and gradually step it up, to encourage as dense a root system as possible.
My only complaint with the fabric pots (and I've tried several types) is that roots will grow right through them if given a chance. Fabric pots placed directly on the dirt will quickly have the plant's roots growing through the fabric on the bottom of the pot and into the ground. This has some adavantages, such as helping to prevent the plant from blowing over. If you wait too long before moving the plant and severing all those roots, it may become quite a chore- much worse than a plastic pot with only a few holes in the bottom that you would need to prune the roots from. If fabric pots are placed so that they touch neighboring fabric pots, the roots will also grow right through the side and into the other pot- at one point I even had a Jamaica Cherry send a runner through the side of a pot and into the guava next to it. So fabric pots are certainly out for containing anything invasive.
This isn't a problem as long as I enforce a 1/2 inch gap between the pots, and move them occasionally if they are placed directly on the ground over the summer.
Kevin