Given the nature of E-Cig batteries -- at the risk of sounding like i'm editorializing -- it's definitely necessary to be mindful of safety issues. There's a good ECF stickie on this topic:
http://www.e-cigarette-forum.com/forum/battery-mods/19698-abcs-batteries.html
I can't speak about other vapers, but I believe that if i follow some basic precautions and use common sense, vaping is not inherently more dangerous than any number of other things that I do on a daily basis.
To keep the safety issue in perspective, think about all of the fatalities and serious injuries that result from fires caused by cigarettes, not to mention the large numbers of smoking-related deaths (cancer, heart attacks, etc). Yet how often do we see dramatic media coverage calling for tighter regulation or outright bans of cigarettes.
So, short answer; e-cigs can be dangerous, especially if used improperly. But a lot of the reporting is alarmist, and more political than public-health-based.
-- Glenn