Here’s a pivotal article from O’Reilly titled “What is data science?”
Quora’s thread on What is data science is a good starting point too.
Hans Rosling’s TED Talk is probably the best to start with. His “200 countries, 200 years, 4 minutes” is an extra-ordinary historical narrative via numbers. The BBC made a one-hour programme called The Joy of Stats which is well worth watching. The statistics behind these can be explored at GapMinder.org.
The New York Times has produced excellent visualisations. Their gallery has visualisations using their API. The sites of some of their team, like Matthew Ericson, Steven Heller, Shan Carter and Graham Roberts, are full of excellent visualisations as well.
Peter Norvig’s The Unreasonable Effectiveness of Data [PDF]
Quora’s thread on How do I become a data scientist has some useful tips.
Reading the best blogs about data can keep you in touch with the latest in the field.
Gapminder has various statistics about countries
The Guardian Data Store has a mishmash of data from various topics as Excel documents
The following links have not been vetted. Your mileage may vary
The Open Data Hackathon tools list is a good compilation.