Articles
Official Statistics
A nice example of Simpson's paradox (although the trend is not actually reversed).
Could Facebook cause cancer?
A good example of how not to use statistics.
xkcd on statistics: correlation and causation, significance, null hypothesis.
Stats Sites
Bad Science
Enjoyable blog about misunderstanding science, often with a statistical element.
Gapminder.org
Great use of graphics concerning global issues.
More Or Less
BBC Radio programme about Statistics in everyday life.
Name Voyager
Shows popularity of US baby names. Clever graphics.
Discussions
Statistical Pragmatism (see also here for links to discussion and rejoinder)
A pragmatic 'philosphy' to bridge the supposed frequentist vs Bayesian divide (by Rob Kass).
Minard's Graphic
Discussion of Minard's classic graphic of Napoleon's Invasion of Russia.
Software
R Project for Statistical Computing
Excellent scripting software for doing real statistics
OpenBUGS
Site with information on using BUGS for Bayesian inference. (get WinBUGS here)
LaTeX
MiKTeX and an editor such as TeXnicCenter can be combined to use LaTeX in a Windows environment.
SINGULAR
Software for automatic algebraic computations.