Overview
From test scores to teacher salaries, from graduation rates to grade-point averages, the education world is full of data. The federal No Child Left Behind law, signed in 2002, created an unprecedented demand for detailed information about students and schools. No longer are public schools judged simply by average test scores for all students. The law requires states, school districts and campuses to break out (“disaggregate,” in education-speak) test scores by race, gender, English proficiency, socioeconomic status and more. Students in every group must meet the same academic standards. At the same time, people want to know which individual teachers have the most success working with various groups of students.
The demand for data is growing in higher education, too. Parents, policymakers and taxpayers want to know which college freshmen need remedial education. They want to know how many students graduate on time, and whether graduates find good jobs. In both K-12 and higher education, some political leaders and others want to track academic performance with spending to see which schools and colleges seem to provide the best bang for the academic buck. This section of Story Starters examines what this proliferation of data means for education reporters and offers links to key data resources.
This national push for accountability in education means that schools and colleges must first collect lots of useful, timely and precise data. Then, they must analyze and use that data -- that is, make good decisions based on evidence -- all with an eye toward improving student success.
Here’s how Jeffrey Wayman, assistant professor of education at the University of Texas at Austin, describes the goal on his Data Use website: “If offered in a useful form, such data can help teachers, principals and other educational personnel learn more about their students, improve their teaching craft, and ultimately impact a variety of educational outcomes.”
When the Data Quality Campaign, a national coalition advocating for better data, formed in 2005, no states had extensive longitudinal databases for education. The campaign created “10 essential elements” it deemed the state data systems should have. By 2011, the group reported that 49 states (all but Montana) have eight or more of the elements. Now the group is focusing on what states should do with all of that data.
Aimee Guidera, the campaign’s executive director, says: “The need is urgent: state policymakers need to allocate scarce resources based on what works to help students, and they cannot do that without data.”
Sources of data
Federal, state and local agencies now keep a wealth of education data. At the federal level, one of the best sources is the National Center for Education Statistics, part of the U.S. Department of Education. The center publishes annual reports (namely the Condition of Education and the Digest of Education Statistics) with state and national education trends on student enrollment and demographics, school staffing, education funding, graduation and dropout rates, and much more. The center also keeps searchable and downloadable databases, such as the Common Core of Data (for K-12 education) and the Integrated Postsecondary Education Data System (for higher ed). While the information at NCES is comprehensive, it’s also often dated, lagging two years behind.
State education agencies also offer a trove of data on student achievement (test scores from state exams, SATs, ACTs, Advanced Placement), graduation and dropout rates, student demographics, school funding, teacher salaries and staffing levels, and more. Some states put many databases online so they’re easily downloaded, while other states may not readily post such information -- and that means reporters will need to specifically request it.
School districts also keep data on student test scores, employee salaries, budgets and the like. Again, the type and availability of information vary by district.
Keep in mind that federal privacy laws, namely the Family Educational Rights & Privacy Act (FERPA), restricts disclosure of individual student education records. In practical terms, that means a journalist can’t ask a school district or state agency for a list of student names with their race, gender and test scores. But a journalist can ask for student-level data with all identifying information (names, social security numbers, etc.) removed. Depending on the state, education agencies may even redact information in particular for small subgroups of students (for instance, test scores for the only four white, female students in a school, under the theory someone could still deduce the individual students’ scores).
Controversies, challenges and caveats
As schools and education agencies are able to collect powerful data, and they’re able to analyze it with more sophisticated tools, controversies have risen. Some school systems around the country rate teachers based on how much academic growth students made while in their classrooms. These “value-added” models look at which teachers are the most (and least) effective with their students by these measures. Some education leaders want to tie teacher pay or raises to their students’ academic growth, instead of compensating them solely on their education credentials and years of experience. Skeptics say the value-added models have too much statistical error to rate individual teachers accurately.
There can be other problems with education data. If students or their teachers cheat on state exams, the test scores (and everything they’re based on, such as state academic ratings) become meaningless. If schools don’t accurately report the reasons students leave school, then dropout and graduation rates aren’t accurate. Some colleges have been accused of reporting false or exaggerated data to U.S. News & World Report so they do better in the magazine’s annual college rankings. Journalists, and anyone else who uses data, need to ask questions about data when they see red flags.
With so much emphasis on data, some experts advise that journalists, policy makers and others should also examine things that can’t be easily measured -- classroom observations of students and teachers, portfolios of student work, parent involvement, and school culture. And we don’t have sufficient data for all students, such as those who are exempt from state exams, or part-time college students who don’t count in the official graduation rates.
In the classroom, meanwhile, educators face the challenge of taking all of the data on students and using it well. The research isn’t yet extensive on the use of data to improve instruction or make decisions about teachers or students, but the research that is available shows that districts need to offer extensive training to principals and teachers to make the data useful. If educators don’t receive the training, they won’t use the data.
“The greatest perceived area of need among districts is for models of how to connect student data to instructional practice,” a U.S. Department of Education study found in 2010. “Districts want examples of how to identify which practices work best for which students and how to adapt instructional strategies to meet the needs of individual students.” — Holly Hacker, June 2012
Key Coverage
Highlighted journalism and reports for this topic
-
Relax, It’s Only a Test
February 7, 2013In the 12 years since the passage of the No Child Left Behind Act, frequent high-stakes exams have become the norm at every public school in every state in the country. Standardized testing programs cost states a total of $1.7 billion yearly, according to a recent report from the Brookings Institution. Poor performances on these exams can have severe consequences: students with low scores can be held back, teachers whose students do poorly can be fired, and schools with below-average overall results can be closed entirely. "Schools and teachers are under a lot of pressure to meet standards, and that pressure gets passed on to students," says Nathaniel von der Embse, a psychologist at East Carolina University who studies tests and their stresses. (Time Magazine)
Read More » -
Teachers’ Ratings Still High Despite New Measures
February 5, 2013Changes to evaluation systems yield only subtle differences. (Education Week)
Read More » -
The Opportunity Gap
January 24, 2013This database includes all public schools in districts with more than 3,000 students from the 2009-2010 school year -- about three-quarters of all such students in the country. Use it to find out how well your state provides poor and wealthier schools equal access to advanced classes that researchers say will help them later in life. (ProPublica)
Read More » -
Colleges Overproducing Elementary Teachers, Data Find
January 23, 2013Data, while imprecise, suggest that some states are producing far more new teachers at the elementary level than will be able to find jobs in their respective states—even as districts struggle to find enough recruits in other certification fields.
Read More »
For some observers, the imbalances reflect a failure of teacher colleges—by far, the largest source of new teachers—and their regulatory agencies to cap the number of entrants.(Education Week) -
The history of school closings in Chicago 2002-12
January 16, 2013WBEZ plotted annual school closings and schools "turned around" since the 2001-02 school year when CPS began shuttering schools as a reform strategy.
This sortable chart and map shows where schools have been closed or turned around (where the staff is completely replaced but students remain), what’s become of the old buildings and how well the new schools in those buildings are performing. The chart includes updated performance data from the 2011-12 school year.
Read More » -
Louisiana’s educators enter a new world with evaluations and their consequences
January 7, 2013Teachers in Louisiana have all but lost the tenure rules that once protected their jobs. Beginning this year, all 50,000 of them will be evaluated and ranked on an annual basis, often with test scores factoring in heavily. Soon, consistently "ineffective" teachers will no longer be welcome in the classroom.
This, depending on one's point of view, is either the latest assault on Louisiana's educators or an urgent step toward modernizing the teaching profession and lifting the state out of academic mediocrity. (Times-Picayune)
Read More » -
Vocabulary Test Results Show Top U.S. Students Losing Ground, Others Stagnate
December 6, 2012If you can identify the meaning of the word "prospered" within a passage, chances are you know more vocabulary than most American high school seniors.
The results of the national standardized vocabulary tests are in, and the scores are troubling -- but not unexpected -- experts say. Average performance on the U.S. Education Department's national exams was mostly stagnant at low levels between 2009 and 2011, and the highest performers lost ground during that time. (The Huffington Post)
Read More » -
School Testing In U.S. Costs $1.7 Billion, But That May Not Be Enough: Report
November 29, 2012Matt Chingos has an idea that will likely roil the scores of parents and teachers who think the U.S. tests its students too much: we might actually spend too little on standardized testing.
[...]
In a report released Thursday titled "State Spending on K-12 Assessments," Chingos, a fellow at the Brookings Institute, tallied up the cost of standardized testing, a subject that has fueled much debate and speculation. After sending out countless Freedom of Information Act requests and rummaging through boxes of documents, he arrived at an estimate of $1.7 billion. (The Huffington Post)
Read More » -
State Falls Short on School Desegregation Requirements
November 15, 2012Connecticut has run out of time to comply with a court order to reduce the inequities caused by the segregation of Hartford's largely black and Hispanic school population.
The state Department of Education on Thursday afternoon reported that 37 percent of Hartford students are now attending integrated schools -- 4 percent shy of the number the state agreed to reach in a settlement five years ago. (Connecticut Mirror)
Read More » -
Mississippi School Funding Shortfalls Could Trigger Even Lower Target
November 6, 2012Based on the expenditures in the mid-range districts, an average cost per student is developed. Local school districts statewide are supposed to receive that average cost per student multiplied by their average daily attendance.
Because of an unprecedented drop in state revenue collections in 2009-2010, the Adequate Education Program has been underfunded a total of about $980 million since the 2007-08 school year. The 2007-08 school year was the high water mark for education funding. Since then, education funding was reduced every year until the most recent 2012 session where it was increased about $20 million. (Northeast Mississippi News Daily Journal)
Read More » -
San Jose Unified, teachers reach breakthrough evaluation, pay plan
October 31, 2012The superintendent of San Jose Unified and leaders of the district’s teachers union have agreed on an innovative evaluation and compensation system that, if implemented, would be significantly different from any in California. With education groups in Sacramento and legislators still bruised over a grueling, failed effort to revise the state’s teacher evaluation law last summer, the San Jose plan offers hope that a progressive compromise on divisive issues is possible.(EdSource)
Read More » -
More Cheating Scandals Inevitable, as States Can’t Ensure Test Integrity
September 29, 2012The AJC’s survey of the 50 state education departments found that many states do not use basic test security measures designed to stop cheating on tests. And most states make almost no attempt to screen test results for irregularities
[...]
The survey reveals other wildly inconsistent practices around the country: Some states require outside investigations of cheating in school districts, but most states permit districts to investigate themselves; some states look for unusual increases in test scores, but most don’t; about half send out independent monitors to oversee testing, about half do not. (Atlanta Journal Constitution)
Read More » -
State Reported Inflated Rate of Teachers Lacking Credentials
September 28, 2012The percentage of teachers and other certificated staff lacking proper credentials was actually 29 percent, not the 58 percent the state reported for the 2005-06 school year. The revelation, sparked by errors in state data identified by California Watch, means the state has been using an incorrect baseline as it measures progress at its lowest-performing schools. (California Watch)
Read More » -
Why Kids Should Grade Teachers
September 20, 2012A decade ago, an economist at Harvard, Ronald Ferguson, wondered what would happen if teachers were evaluated by the people who see them every day—their students. The idea—as simple as it sounds, and as familiar as it is on college campuses—was revolutionary. And the results seemed to be, too: remarkable consistency from grade to grade, and across racial divides. Even among kindergarten students. A growing number of school systems are administering the surveys—and might be able to overcome teacher resistance in order to link results to salaries and promotions. (The Atlantic)
Read More » -
Everything You’ve Heard About Failing Schools Is Wrong
September/October 2012Attendance: up. Dropout rates: plummeting. College acceptance: through the roof. My mind-blowing year inside a "low-performing" school. (Mother Jones)
Read More » -
OECD Releases Report on Global Education Trends
September 11, 2012The Organization for Economic Cooperation and Development released its annual Education at a Glance report today, a 565-pager with statistics on a wide range of education topics, from early childhood to higher education.
Read More » -
Teaching the Teachers
Summer 2012Data-driven instruction may be great in theory, but the challenge is having the right tools to put it in practice. That includes training the teachers, according to an article in Education Next that looks at the efforts by the Achievement Network. (Education Next)
Read More » -
Districts Use Data to Sharpen Focus in Class
May 22, 2012Omaha Public Schools spent millions launching a computer system to help teachers provide data-driven instruction. But some teachers haven’t embraced it, and others say they aren’t fully trained in how to use it. And still other educators say that while they’re able to diagnose where students need help, they need more assistance in developing alternative strategies. (Omaha World Herald)
Read More » -
Can Education Data Build the Perfect Teacher?
February 29, 2012Can administrators, policymakers and educators use objective data to create the perfect learning experience? (Governing Magazine)
Read More » -
Statistical Significance
Winter 2012Data have been increasingly incorporated into education practice and the increasing reliance on standardized tests is increasing the amount of data for educators to analyze. But the overwhelming data are leaving many unable to figure out how to use them. (Ed./Harvard Graduate School of Education)
Read More » -
The Class of 2020
December 1, 2011EWA 2012 National Reporting Contest winner. Longitudinal studies tend to tell us the most about student progress, but the reports can be dense and difficult to access. This series takes on the task of following the same group of students for 13 years until they graduate from high school. (NBC News/TODAY)
Read More » -
Can Education Be ‘Moneyball’-ed?
October 14, 2011Data analysis is so trendy these days that Brad Pitt is getting millions of people to sit through a movie about quantitative methodology called Moneyball, writes Eduwonk blogger Andy Rotherham. A lot of education reformers are calling for a similar approach to improve student performance but there are some significant strikes against a Moneyball approach to education, he says. (Time)
Read More » -
Spotlight on Data-Driven Decision Making
June 30, 2011Education Week focused on data-driven decision making in nine articles over 2010 and 2011, looking at issues around mining the data to improve instruction, better use of data to prevent dropouts, implementing the technology and managing student privacy, among other topics.
Read More » -
Data-Driven Instruction and the Practice of Teaching
May 12, 2011Prominent education researcher Larry Cuban writes a column offering a skeptical view of the move toward data-driven instruction. (Larry Cuban on School Reform and Classroom Practice)
Read More » -
Is High School Making Your Kids College-Ready?
November 12, 2010EWA 2010 National Reporting Contest winner. Even the most posh suburban high schools in the Chicago area missed the mark on preparing college-bound students for the rigors of academics in higher education. Using never-before-seen ACT data accessed through a Freedom of Information Act request to the state, the paper learned that only 20 percent of high school juniors in Illinois scored high enough on the four benchmarks the test making agency deems are most essential in determining whether students will succeed in college. (Chicago Tribune)
Read More » -
Building a Better Teacher
November 6, 2010EWA 2010 National Reporting Contest winner. From the series description: “In a series appearing over eight Sundays, ‘Building a Better Teacher’ looked at challenges to the way teachers are trained, evaluated, paid, promoted and dismissed - and how all of it comes to bear on student success.” (Milwaukee Journal Sentinel)
Read More » -
The WRITE Stuff
June 1, 2010Grant School District 3 in Oregon took full advantage of a new data-driven training program to improve its students’ test scores by using data to focus on trouble spots. Spelling errors by students shrank substantially thanks to the effort. (Blue Mountain Eagle)
Read More » -
Untapped Resource: State Unlocks Data Storehouse for Teachers
April 4, 2010Tennessee has kept detailed measurements of student achievement for nearly two decades but the data were off-limits to teachers, who still don't know exactly how to use it to improve instruction. (Chattanooga Times Free Press)
Read More » -
The ABCs of Ditching School
March 3, 2010EWA 2010 National Reporting Contest winner. Studies increasingly show chronic truancy is a telltale sign that a student is on the road to dropping out of school. So what can schools do about it? This feature examines the sleight of hand students come up with to play hooky, and the steps schools could take to combat the high rates of ditching class. (San Francisco Weekly)
Read More » -
Culture of Data Evolves in Fulton Schools
September 1, 2009The Fulton County, Ga., public school system is well-known for its data-driven decision making and management system. In fact, it’s been lauded by experts for its techniques. It took the district 10 years to get there. (Education Week Digital Directions)
Read More »
Sources
Click to View (opens in new tab)Reports & Data
Notable research on this topic
-
One-Stop Shop to Learn About Education Data
January 25, 2013Data First was created with the idea that data matters. Education data, used well, can help school board members and everyone else who cares about education to make good decisions – ones based, not on the loudest voices or the latest theories, but on the facts about what students need and how they are currently doing. The Data First site is designed to link visitors to data they can use about schools, and to teach them how to use it better. (Center for Public Education)
Read More » -
Ensuring Fair and Reliable Measures of Effective Teaching
January 9, 2013This non-technical research brief for policymakers and practitioners summarizes recent analyses from the Measures of Effective Teaching (MET) project on identifying effective teaching while accounting for differences among teachers’ students, on combining measures into composites, and on assuring reliable classroom observations. (Editor's note: The study was part of a three-year, $50 million project funded by the Bill & Melinda Gates Foundation that included dozens of researchers and over 3,000 teachers who volunteered. It concluded the best approach for assessing the strength of a teacher is relying on a mix of classroom observations with at least two observers, student feedback, and value-added measurements.)
Read More » -
Strength in Numbers: State Spending on K-12 Assessment Systems
November 29, 2012The report identifies state collaboration on assessments as a clear strategy for achieving cost savings without compromising test quality. For example, a state with 100,000 students that joins a consortium of states containing one million students is predicted to save 37 percent, or $1.4 million per year; a state of 500,000 students saves an estimated 25 percent, or $3.9 million, by joining the same consortium.
Collaborating to form assessment consortia is the strategy being pursued by nearly all of the states that have adopted the Common Core standards. But it is not yet clear how these common assessments will be sustained after federal funding for their development ends in 2014, months before the tests are fully implemented. (The Brookings Institution)
Read More » -
Education at a Glance 2012: OECD Indicators
September 11, 2012The 2012 edition of Education at a Glance enables countries to see themselves in the light of other countries’ educational performance. (OECD)
Read More » -
Impact of Data-Driven Reform on Mathematics and Reading Achievement
September 2011This study may be the first large-scale randomized assignment study looking at data-driven instruction and its effect on student learning. The study looked at 500 schools in 59 districts to estimate the effects of a project by the Johns Hopkins Center for Data-Driven Reform in Education. The study looks at the first year. (University of Wisconsin-Madison)
Read More » -
Organizational Considerations in Educational Data Use
April 2011Effective data use by U.S. schools is proving to be a vexing problem. Researchers in this paper suggest ways that schools and districts can use data for educational improvement. They discuss three organizational areas in which these districts may improve: establishing common understandings, professional learning for using data, and computer data systems. (University of Texas)
Read More » -
Using Student Data to Support Instructional Decision-Making
September 2009The Institute for Educational Sciences provided a guide for educators on the best research available on data-driven decision-making and instruction. The report highlights findings with strong evidence as well as findings where the evidence is weak.
Read More » -
Beyond Test Scores: Leading Indicators for Education
January 2008“Leading indicators” in education — as in economics — can provide early signs of progress toward academic achievement and thus help district leaders and other stakeholders make informed decisions about efforts to improve student learning. (Annenberg Institute)
Read More » -
Cutting Through the “Data-Driven” Mantra: Different Conceptions of Data-Driven Decision Making
April 2007The paper, a chapter in the Yearbook of the National Society for the Study of Education, summarizes two RAND studies that address this question: What are the different ways educators use data to make decisions about teaching and learning? The study outlines factors that enabled – or inhibited – various types of data-based decision-making. (The RAND Institute)
Read More » -
Making Sense of Data-Driven Decision Making in Education
2006RAND provides an “occasional paper” reviewing its research on data-driven decision making and clarifying what conclusions can be drawn. There remain many unanswered questions about the interpretation and use of data to inform decisions and the ultimate effect of those decisions. (The RAND Institute)
Read More »
Five Questions to Ask
- What data do your state and school districts collect?
- What do your state and school districts do with their data? Do they collect the data and warehouse them?
- What mechanisms are in place for states and school districts to share the data with schools, principals and teachers?
- What kind of training are your school districts offering school administrators and teachers in using the data? The limited amount of research on data use shows that if teachers and administrators don’t receive training on how to use data, they won’t use it consistently.
- How transparent is the state’s data system? Journalists should be able to request student-level data as long as students are not easily identifiable.
Organizations
The Center for Data-Driven Reform in Education is based at Johns Hopkins University and conducts training as well as provides a model, Raising the Bar, for data-driven instruction. Note that the organization is affiliated with the Center for Research and Reform in Education and Robert Slavin.
Consortium for School Networking’s Data-Driven Decision Making (3D) Initiative is a national effort to help school district technology leaders build and sustain a data culture within their districts. It is designed to provide tools and resources to help districts implement and sustain data usage while providing a national forum on how data are being used to individualize the learning process.
Data Quality Campaign is a national coalition that has pushed for better data in education. It keeps a list of “10 essential elements” that every state education data system should have -- including student-level information on test scores and demographics (such as race, gender and socioeconomic status); a unique identifier assigned to students so they can be followed over time; and the ability to match individual teachers to their students.
DATA Use is a new website created by University of Texas professor Jeffrey C. Wayman, who researches data-driven decision making by school districts and what it takes to make districts effective users of data. His new project, “the Data Informed District,” is examining the work of three districts in central Texas. He will develop a framework based on the research this year (2012).
Data Use for Improving Learning is a website operated by UCLA’s National Center for Research on Evaluation, Standards and Student Testing (CRESST) to look at the effective use of data to improve learning. The site offers current research and guidelines for educators.
Data Wise Project is an effort based at the Harvard Graduate School of Education that helps develop resources for educators on how to effectively use data. Data Wise provides online training as well as an annual training summit at the school.
Michael & Susan Dell Foundation focuses on performance-driven education as one of its key goals and has funded efforts in school districts and charter management organizations across the country. As part of that, the foundation commissioned several case studies, including one on the Denver Public Schools District and the other on Charlotte Mecklenburg.
Multimedia
'Check In’ on Education Inequality allows ProPublica readers to link their Foursquare account with the news outlet's "Opportunity Gap" project. Users can get stats about a school when they check into it with their mobile device's Foursquare app.
Suggest a Change
If you'd like to suggest an addition or change to this section, send an email to EWA Project Director Kenneth Terrell.