I think it's a position that often comes from ignorance. Usually these people have no concept that games can involve high quality character and plot writing like Mass Effect, or teamwork focused gaming situations like MMO raids. They either never had any exposure to games or decided early on (usually in adolescence) that games weren't for them.
Ironically there's often a lot of overlap between game haters and people who spend more hours in front of the TV than most of us do gaming.
I too am a happy customer. The ability to tweak individual site ranking especially makes searching the internet so useful again.