Of course some have a foul odor that lingers and almost stains. Across its primary growing and consumption regions it is often banned from enclosed spaces. That wouldnt happen if it wasnt an appealing, faint odor to the majority.
Im not against durian or its smell, but have encountered a few with a rancid cabbage boquet, like you get when visiting NY chinatown mid summer. Not all smell like this, but some certainly do (and if you dare to eat one that smells bad olfactory senses will often correlate this to your tastebuds, giving you a terrible experience). Im unsure how folk state they dont smell that strongly, when even in places they are appreciated and widely available they are acknowledged as being rather smelly.
I was also told by a durian fan that if you didnt like the smell or taste of durian you were an unfit person, and that all fit and healthy people would appreciate the taste and smell of durian. That was undoubtedly the stupidest thing I have ever heard.