Concerning the "they put chlorine in the water for a purpose": Yes, but that purpose is first to kill anything that's in the water when they introduce it to the system, and then to prevent things from growing as water sits in the system before it reaches your house. Unless water's sitting for very long periods of time in the pipes in your house, it's unlikely you have anything to worry about.
That said, the concept that "chlorine is as bad for people as it is for plants" is also wrong. Plants are much more chlorine sensitive than humans. Humans ingest large amounts of chlorine ion every day not simply from water, but most famously in the form of salt (sodium chloride, Na+ Cl- (aq)). About 100 grams of the average human body is chlorine. And if you think chlorine is bad in your bathwater then you better not even go *near* the ocean where it's many orders of magnitude more concentrated.
That said, filtering your water isn't in most regards a bad thing. One, the most common water health issue is inadequate disinfection, so if you have a filter that can filter out infectious organisms as well, you're helping yourself out there. Harmful contaminants in water are limited by regulations, but with a filter you're adding a layer of protection for yourself against accidental violations of water quality guidelines, and of course, even the limits can be improved upon; it's never bad to reduce your intake of, say, lead, even if the levels are already low. The only negative problem that comes to mind is if your filtration system filters out fluorine (which if it filters out chlorine it almost certainly will do). This will give you a statistically significantly higher risk of dental caries unless you consume enough from other fluoride sources, such as fluoridated salt or toothpaste. Regular exposure to fluorine ions (such as from fluoridated water, or water from streams and wells naturally rich in fluorine) converts the hydroxyapatite that makes up teeth into the harder and more acid resistant fluoroapatite.