I firmly believe that not one single thing in the entire universe simply serves no purpose, not even small talk. It seems apparent to me that small talk as a social construct serves to perpetuate a state of societal homeostasis which could be succinctly described as a mindset of "I'm okay, and you're okay too."
The purpose of this state of homeostasis is to provide a sense of order as well as a lack of danger. It's no mistake that the typical neurotypical will elicit an emotional response coming from somewhere on the fear-spectrum of feelings when you answer their "HI! How are you doing?" by telling them you're not okay and why. Fear reactions come straight from the stimulation of the amygdala - meaning their automatic, involuntary response to this breach of social protocol (not answering in the positive) is a primal, hard-wired response to the threat made against the state of societal homeostasis from which the axioms of belief that constitute the totality of their perceptions of reality are derived.
To further explain, every individual has a slightly different perception of reality. Our own personal iteration of reality is the sum total of our axioms of belief, aka world views, which are basically beliefs that seem to be so apparently and unarguably true that we accept them as "laws of the universe", so to speak, and are the tools our brains use to make sense of the abstract. The reason small talk seems so alien, pointless, and difficult for many of us is the same reason that we, as individual members of the autism community, are so diverse from even one another. It's all about how those world views form.
NTs with their fancy mirror neurons grew up placing all their value in mimicking their peers and adults with the primary impetus being the attainment of a sense of belonging, which a child may value above all other needs, including physical safety (see: children who cling to abusive parents). Their emotional needs being met by successful participation in society caused them to adopt socially-centric world views and to subsequently practice all the rituals that reinforce and validate them, such as small talk.
On the other hand, I don't think I've yet heard of an Aspie with a childhood that was anything less than traumatic, almost always with themes of social rejection and usually inhuman torment as a result of our attempts to gain the very same sense of belonging. Lacking the coveted ability to join that wonderful social club, we often formed our world views based on what life experience gave our brains to make sense of. Whatever the product turned out to be, our world views tend to be empirical (based in reality, through observation) in nature, sharply contrasting that of the NT's.
Thus, we have our disconnect when it comes to small talk. NT's and us simply worship different gods, so to speak. The best thing to do is accept things as they are and work with them. Plenty of people on this thread have already said what needs no echoing, but I'll add that while you should be proud of who you are, never be scared, and never back down, the NTs really have us beat if in no other way but sheer numbers. I'm not singling anyone out in any part of this post, and this isn't even close to fair, not by a longshot, but the status quo dictates that we have the choice to either give up and go live on The Island of Broken Toys or learn how to play the game and accept it as a positive thing.
OH, but all that aside, I should mention there is actually a book out there that's pretty much a how-to manual on how to effectively communicate with NTs. It's called Games People Play by Dr. Eric Berne. It's not some cheesy self-help book targeted to an Aspie audience, rather it's a unique and different perspective on human interaction. It's the best $8 you'll ever spend, it's a fairly short read, and when I read it I had one of those "Ohhhhhh..." moments about every three pages. It cut years of painful trial-and-error out of my learning process.
Sorry for the brain dump. I'm just not one for short quips or explanations of a complex phenomenon that could fit in a twitter post.