Education Indicators

Indicators in Education

"Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write." ~ H. G. Wells (1)

The popular media routinely provide extensive coverage of education indicators. A sampling of “findings” that have been reported by major media outlets within the past few years include:

• Only 33% of fourth graders were proficient in reading.

• For the class of 2001, 72% of White students graduated from high school, but only half the Black, Hispanic, and American Indian children left with diplomas.

• About half of all new teachers leave the profession after just 5 years on the job.

These statements are very direct, concrete, and convincing, and in some cases present dire situations for our children and the system in which they are educated. But upon what evidence are these statements based? Given the large emphasis that society places on using indicators as measures of performance and descriptions of status and change, the rarity with which we exercise any critical assessment of these measures is surprising. This is, in part, due to a lack of practical guidance to the audience on how to evaluate and assess indicator quality. Consumers are largely left on their own to navigate through this information and determine the legitimacy of the claims they encounter. Some may muddle through the voluminous technical reports that often accompany a substantive publication, but this documentation, if it is published at all, is often pedantic, arcane, and impossible to follow without an advanced degree in statistics or research methodology. As a result, it is not uncommon for individuals to develop an attitude that all numbers are of equal quality, are easy to manipulate, and offer little value to the issues at hand. A main purpose of this book is to dispel, or discourage the formation of, such attitudes. Numbers and indicators are not of equal quality, and the high-quality ones can be immensely valuable when considering important policy issues. The trick is to identify which indicators are of high quality and add value to policy debates.

Underlying individuals’ distrust and pessimism toward indicators—and numbers more generally—is the fact that they often seem to be developed in mystical ways. The process through which these indicators are created is often less than transparent. How are high school dropout rates measured? What does proficiency in mathematics mean? What constitutes a violent crime at school? These seemingly straightforward issues can quickly become complicated once one considers the wide variety of ways these questions can be answered. While these issues certainly present difficulties, they can be overcome with careful consideration and evaluation.

We have all heard the quote often credited to Mark Twain: “There are three kinds of lies: lies, damned lies, and statistics.” Twain is certainly exaggerating when he refers to all statistics as gross distortions of the truth.

In our opinion, statistics—or more specifically, indicators—based on sound scientific procedures can be powerful means for description, analysis, and monitoring change. Indicators of poor quality certainly distort and misguide decision making and policy. We believe that high-quality indicators are the overwhelming norm, but recognize that lies and distortions can be difficult for an uninformed audience to discern.

The purpose of this book is a modest one. It is to provide users with a framework and tools for understanding and evaluating the quality and value of education indicators. We provide readers with an overview of how indicators are generated and used in education. The reader is presented with the process of conceptualization, measurement, and dissemination. The seemingly straightforward process is, in actuality, rather complex, but it is our hope that this complexity becomes transparent and the reader gains an appreciation for the value and limitations associated with construction of education indicators. This is an effort to create a healthy skeptic and an informed citizen.

**The book is not intended to provide the reader with a guide as to what should be measured in education. We leave these decisions- whether to measure graduation rates, mathematics achievement, truancy rates, etc.- up to the community, principals, school boards, researchers, governement agencies, and other policymakers. A list of organizations that have taken a focused look at what should be measured is provided below.

(1) This quote, commonly attributed to Wells, is actually a paraphrase by Samuel S. Wilks (1951) of the following sentence in Mankind in the Making (1904, p. 192): “The great body of physical science, a great deal of the essential fact of financial science, and endless social and political problems are only accessible and only thinkable to those who have had a sound training in mathematical analysis, and the time may not be very remote when it will be understood that for complete initiation as an efficient citizen . . . it is as necessary to be able to compute, to think in averages and maxima, as it is now to be able to read and write.”

Links to organizations and publications dealing with education indicators:

U.S. Department of Education, NCES

http://nces.ed.gov/programs/coe/

NCES Forum Guide to Education Indicators (2005)

http://nces.ed.gov/pubs2005/2005802.pdf

ChildTrends

http://www.childtrends.org/index.cfm

OECD Education at a Glance

www.oecd.org/edu/eag2009

Council of the Chief State School Officers

http://www.ccsso.org/

Childstats