A snapshot of data-driven decision making

Educational professionals wishing to use technology to make effective decisions for school improvement should heed what Arne Duncan’s Department of Education communicated last summer. The 124-page National Educational Technology Plan (NETP 2010) points out how technology produces data, the life-blood of school improvement. The smart use of technology-based data improves “entire education system[s], improve[s] student learning,” and “accelerate[s] and scale[s] up the adoption of effective practices.” The key to much of the improvement is making the individual classroom teacher into a data analyst, able to monitor and analyze data, collaborate and share with his colleagues, and, as the NETP 2010 puts it, work together for “continuous improvement.”  My K-12 district’s recent has recently taken some bold steps down the path of technology based, data-driven decision making, and I hope it is willing to take the next.
a “pie chart” of data
In 2009, a program was begun to identify and make interventions for a group of under-performing middle-school students.  District administrators acting as a “data team”–in this case the building principal and English department chair–were “startled” by evidence from the 2007 National  Assessment of Educational Progress (NAEP) showing that  69% of 8th graders were scoring at basic or below proficiency level. They noted with alarm that 25% of students were not meeting basic standards at precisely the time the NCLB law demanded they be increasingly proficient. The administrators explained the negative trend in the 2010 School Improvement Plan, noting “clear evidence in our performance data that there is a trend, or a ‘cohort’ of students who enter high school with reading and math gaps and continue not meeting standards when they take the PSAE.” By addressing this group’s needs at the freshman level, administrators were doing what they could to reverse the students’ deficits and eventually get under-performing in 8th graders “the skills and strategies necessary to achieve and exceed college-readiness standards while in high school, as well as pursue entrance to the university of his or her choice.”  The cohort might have been identified earlier in their K-12 career, before AYP standards made it a necessity, but ultimately, because they used technology to analyze data, the administrators were able to offer help to these 8th graders. The team used the computer-based NWEA MAP, as well as the standardized ACT Explore Reading,  and R-CBM to identify the “cohort.”
When they began analyzing the student performance data in 2009, our district administrators performed just as McIntire (2005) predicted they should. That is, they analyzed the data and found ways to use their analysis to improve efficiency.  After collecting, organizing, and analyzing them, the administrators–importantly–agreed on what the data had to say. They then developed an action plan in response to their findings. Our district’s administrative “data team” acted correctly, according to the authors of Data Wise, Boudett, City, and Murname (2008). But the authors of the book, whose subtitle is “using assessment results to improve teaching and learning,” would point out that until now, our data-driven administrators have only taken preliminary steps. The largest gains in learning efficiency go to those districts that–over a number of years–transform themselves into “culture[s] of inquiry,” or a “data culture[s] (12).”  Such districts share student performance data with all teachers, which, in turn,  according to Boudett, et al, “inspire[s]” … and “increase[s] staff capacity to …understand and carry out school improvement work . Sharing data analysis and evaluation is supposed to have a “multiplier effect” on school improvement.  
Other researchers agree that the best way for data to drive good educational decisions is for the process to be systematically democratized–made the responsibility of the entire faculty. Popham (2009) maintains if the school has any chance of improving its performance, it must enlist its teachers as data-literate participants in the analysis and evaluation of student and school performance data . Dartnow, Polk, and Kennedy (2008) maintain there is no reason to wonder about the effectiveness of making schools into “data-cultures”. The jury, they wrote in 2008, is in:
…an increasingly clear and persuasive body of research is demonstrating … that high-performing schools and school systems use student data in all facets of their work to continuously inform and improve their instruction. ….several pioneering secondary schools—and the school systems of which they are part—are making significant inroads in using data to improve instruction and hence to improve student outcomes.
McIntire’s second two phases of developing into a “data-culture”–using data to maximize educational efficiency throughout the building and reorganizing the school and curriculum for sustained improvement–have yet to arise in our district, but there has been a good start in stage one. The classroom goals and cooperative work that can bring about further development through school-wide goals and professional development could build on the work of the “cohort,” the district’s major data-driven decision in the last two years. And although only one semester’s worth of assessments exist, the data-team, which has expanded to include the dozen or so teachers involved in the “cohort,” is already doing what NETP 2010 described as “using the full flexibility and power of technology to design, develop, and validate new assessment materials and processes for both formative and summative uses.” No evaluative reports have been issued yet, but in their work so far, the team appears to be “finding new and better ways to assess what matters, doing assessment in the course of learning when there is still time to improve student performance, and involving multiple stakeholders [at least within their team] in the process of designing, conducting, and using assessment.” The “data team” can function in our district as Boudett, City, and Murname (2008) maintain it should:  as a “bridge” between the confusing array of performance data and the rest of the district. These are the people who are “in a good position to help … faculty develop assessment literacy” (30), a pre-condition to system-wide transformation. In other words, this could be the start of something very big in the life of the district, but we are only in stage one.
Once the transformation of stages two and three has been brought about, the scenario that McIntyre (2005) describes can be seen, in which schools “use data to squeeze the most out of their students’ performance and thereby get as many as possible over the next [proficiency standard] bar. they do everything they can to maximze the performance of students on the margin of moving to the next performance level. Schools on the kinfe-edge of AYP must get results now, and therefore the focus is on short-term actions plans. Which ‘bubble students’ need help on this week’s mathematics topics? What was the impact of last week’s Saturday School lessons?  How many in the ELL subgroup reached the threshold on this month’s practice test?”  In such a scenario, data-analysis has dramatically transformed the monitoring and reporting capabilities of teachers, as well as their reflection and planning. One hopes that the district’s hopeful steps into data-driven decision making described here are just the opening steps in a long, successful dance.
Boudett, Kathryn Parker, Elizabeth A. City and Richard J. Murnane. (2008) Data Wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, Massachusetts:  Harvard Education Press 2008.
Datnow, A., Park, V., & Kennedy, B. (2008). Acting on data: How urban high schools use data to improve instruction. USC Rossier School of Education: Center on Educational Governance.
McIntire, Todd (2005). “DATA:  Maximize your mining, part one.” Tech & Learning. April 15, 2005. accessed on 22 January 2011 at http://www.techlearning.com/article/13816
Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory Into Practice, 48, 4-11.
US Department of Education, Office of Educational Technology. (2010). National Educational Technology Plan. “Transforming American Education:   Learning Powered by Technology.”  Accessed 10 December 2010 at http://www.ed.gov/technology/netp-2010
visual by AtomicShed, via search.creativecommons.org

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: