I've been the coworker who never missed a chance to drink with coworkers and I've been the one who hated my job so much that the last thing I wanted to do at the end of the day was spend more time with my coworkers. That turned out to be a mistake, though: at my first job out of college, the occasional beer with coworkers kept me sane, but it took me a year to even consider it. One coworker there said I seemed "stuffy" to her, which would have been hilarious to the staff and volunteers at my next job. I drank with them regularly and developed a rep as something of a party girl. During the day, I fended off sexist comments and had to fight to be respected; being known as cool, fun and likeable probably worked against me.
A massively unfair catch-22, that: skipping the office Christmas party and keeping your head down at work can get you the wrong reputation in the office, but so can being known as the lampshader. (While men aren't immune to this kind of gossip, I frankly don't hear nearly as much murmured concern that the guy who got too lit last weekend might be a little more committed to the good life than he is to his job.)