Skip to main content

Census Information: Comparability

access to comprehensive guides covering the organization of Census data on the Web, the Library's Census holdings, and using American FactFinder

The Importance of Definitions & Methodology

The two main reasons for difficulty in comparing statistics from one Census to another are:

  • Differences in questions (ways of asking about race and Hispanic origin can cause particular problems and these are addressed in some detail under the Race & Hispanic Origin tab)
  • Differences in geography

The problems arising from differences in questions are the most complicated and are discussed in the box below. Some issues to be aware of in dealing with geographic changes are presented in the box to the left.

Comparability: Geographic changes

While not as challenging as comparability problems arising from differences in questions asked, geographical changes can be very important nonetheless. USA

Metropolitan areas. As you can imagine, in the ten years between Censuses, new cities will attain Metropolitan Area status (although the changing definitions applied to urban areas account for some of the differences, too. And the boundaries of MAs will shift considerably over time as well, making comparability of data for some applications difficult.

Census tracts. Tracts are also variable over time, since shifts of population concentrations may necessitate new boundaries. 

Even such supposedly stable geographic entities as counties can be changed over time.

Comparability: Definitions & Methodology

Questions

When you keep in mind that all Census data is derived from the questions asked of the population during the Census-taking process, it is obvious that the inclusion or exclusion of a particular question will have a great effect on the data available for that particular year.

General information on the process of formulating questions, including examples, can be found in this American Library Association paper: "Reflecting our Changing Culture and Society: How the US Census Bureau Modifies its Survey Questions." If you are interested in the reason behind the questions asked in the 2000 Census, consult this page: Federal Legislative and Program Uses: Why is Answering the Census Required by Law?

question markFor example, the 2000 Census is the first one which asked a question about grandparents as primary caretakers of grandchildren. Therefore, this 2000 data cannot be compared with data from earlier years, at least not with Census data. (Other organizations may have information on this subject from surveys or other means.) Conversely, the 2000 Census did not ask about the source of a household's water, a question which had appeared on previous Census questionnaires. Therefore data on this topic from a previous Census cannot be compared with 2000 data, since there is no 2000 data (from the Census, at least) on this topic.

Which questions are asked (or not) is largely a matter of Congressional mandate or executive agency requirements, making the seemingly arcane and pedestrian subject of Census methodology subject to all the whims of political maneuvering and varying public opinion.

noteTo complicate matters further, other factors than simply the presence or absence of a question can affect comparability. For example, questions may be switched from the 100% section of the Census questionnaire to the sample section, and vice versa. Some basic questions (age, race, sex) are asked of all residents, while approximately one-sixth of the households in the country, through the 2000 Census, were asked a set of much more detailed questions (income, education level, etc.). The Census Bureau will report data for all of these items, of course, but in comparing one set of data to another, users should be sensitive to whether the data came from a 100% or a sample question.

Note that the 2010 Census did not ask more detailed questions of a sample of the population, but rather refers to the American Community Survey, a continuous sample survey, to provide this kind of data. Therefore comparisons of pre-2010 sample data and the same sorts of information post-2010 will become even more complicated.

Answers

Differences in the questions themselves are not the only comparability problem stemming from the actual Census questionnaires. The answers may also be different from Census to Census. Some Census questions leave space for open-ended answers, but for most of them, respondents must choose one (or more) answers from several options.

Since the responses are constrained in this way, the data will be similarly constrained, and these constraints might have differed over time. For example, the answers to a question about disability were greatly expanded in the 2000 Census, in order to gather more detailed information about the nature of the disabled population. The total numbers of disabled people as reported in 2000 should be comparable to total numbers from previous Censuses. But any details on the nature or extent of disabilities will not be.

Daniel Cornwall, at the State Library of Alaska, developed a detailed Census 2000 tutorial that explains a lot of Census problems very clearly. The parts of the tutorial dealings with questions and answers, and with sample vs.100% questions, are linked below.