The United Nations Statistics Divisionprovides an enormous amount of statistics on member countries. Some of their data is available only for ordering or for a fee, but a great amount of it is freely available.
They also offer links to individual countries' statistical systems, and a wealth of technical detail regarding their methods and the limitations of their data.
UNData is their primary data portal. But you might want to consult some of their other statistical compilations and/or databases: