Anger may be defined as an impulse, accompanied by pain, to a conspicuous revenge for a conspicuous slight directed without justification towards what concerns oneself or towards what concerns one's friends.
In which case, everyone who is angry feels (in both the sensitive and intellectual sense) he has been conspicuously and unjustifiably slighted.
Who or what might an angry atheist feel he has been slighted by?
Various theological and psychological suggestions have been made. Maybe atheists are angry at God for not making things be other than they are; maybe they're angry at their fathers and just transferring their anger to the greatest Father figure of them all.
Atheists themselves, though, tend to say they've been slighted, not by a God they only pretend not to believe in, nor by a universe that fails to live up to their expectations, but by theists. Hypocrisy, irrationality, and wickedness are three specific charges I've heard atheists make against theists.
Hypocrisy is an insult to the intelligence. It says, "You aren't smart enough to see through my pretence." It's also an affront to justice to benefit from telling lies.
All religious faith may seem irrational to some atheists, but I think what really drives them up the wall is an attitude like, "I don't care what science or logic can prove to be true, I'm going to continue to believe whatever it is I happen to believe." That attitude strikes at the highest good of rational atheists. Without God, human reason is all we have to go by to define the meaning of life; if we can't get as far as a definition, we can at least prescribe the area within which we can choose to declare our own meaning. And if the one thing you value above all else is rejected as completely worthless, that can sting.
By wickedness, I mean here that many atheists regard various religious actions as inherently evil. Google "crusades" or "inquisition" for examples. Some of the wickedness imputed to theists I, too, impute to them (e.g., forced conversions, human sacrifices); some is imputed through historical ignorance; some is imputed through a flawed moral view, as e.g. the view that raising your children in the Christian faith is child abuse.
Now, hypocrisy in particular and wickedness in general are both reasonable causes of anger. And if irrationality as such isn't a conspicuous slight directed toward another, it can certainly give rise to any number of wrongs. Broadly speaking, then, atheists are angry for many of the same reasons that theists, conservatives, environmentalists, and Orioles fans might be angry.
I do suspect that the more conspicuously angry atheists -- those who write books and maintain blogs in which they stoke their anger -- find in the hypocrisy and irrationality of theists extra barbs that prick their egos. If so, though, I'd say it's not because they're atheists, but because they're egoists.
St. Thomas follows Aristotle in teaching that "excellence makes men prone to anger... those who excel in any matter, are most of all angry, if they be slighted in that matter...." Those who regard themselves as excelling in clear thinking are most of all angry if they be slighted in their clear thoughts.