EDT 6050: an untraditional course reviewed

The name of the CUC class is “Using technology to make data-driven decisions,” or something like that. It doesn’t matter because it’s the way the class happens that interests me.

It is not at all a traditional course, although its subject matter is pretty standard. Yes, like the other courses in this Educational Technology masters cohort I have been taking, it is blended (three classes in person, five via the Internet). No, what makes this such an odd course is that in our sessions together and online, the instructor does almost zero instruction. Instead, she acts as a caretaker of someone else’s estate might–“Here is the English garden,” “Here is the ballroom,” “Please don’t take any flowers.” She is a quiet attendant of the course, not ready to tell anyone how to do anything, and this is not at all to the bad.

One of the consequences of her entirely backgrounded presence is that it puts the onus for learning entirely on my cohort classmates and me. If we don’t write good responses and discussion tabs, if we don’t provide thoughtful and well-researched case study analyses, we will not learn.

So when we do the Excel exercises in class, and the “instructor” sits at her desk and does something on her computer, the cohort starts to respond–the genius of non-instruction begins: Groups help within and across the classroom. One student who has been struggling with the text-heavy classes is a positive maven with Excel, and heroically assists anyone with their technical difficulty. Small groups given projects to accomplish by certain dates organically find the best tools for doing the collaborative jobs and figure out their problems themselves, probably better than they would have had the instructor told them what to do more prescriptively. By sitting quietly behind the desk, this “non-instructor” got out of the way of the learning. Makes me wonder how much I do with my high schoolers, and how much more I should.

In a bit of serendipity, I spoke this week with Rusty Jones, who served as the late George Shearing‘s drummer in his quintet for many years. Jones verified my perception that George was a man who loved what he was doing–his happiness as he played was so clear to the observer.

And at this same jazz performance, I spoke with the great trumpet/cornet player George Bean. Mr. Bean teaches jazz playing to aspiring artists and said that the hardest thing to teach is the most essential and simple. He said, “there’s five words that, if they do them, they’ll play great. But it is so hard to teach them.”

“What are those five words, George?”

“Just ‘get out of the way.’ If they try and play a bunch of notes and show off all their intelligence, the music will stink. But if they understand the song, and the audience, and forget about themselves, great things happen!”

So there’s this week’s wikiness wisdom, reader: “Just get out of the way!” and good things can happen.
________________________________________________________________________________
Below is a good example of what transpires when the teacher gets out of the way. First, the case study background:

Background Allison Massey has been teaching 6th grade math at Shark Middle School for the past 15 years. This is her only position since graduating with a bachelor’s degree in math from IU. She loves the kids and for the most part the parents but keeping up with the demands of increasingly informed parents is becoming difficult.

The Internet and media attention creates an environment that challenges her ability to stay one step ahead of the parents. They have access to almost every piece of information she does but they only need to know about their child and she teaches over 100! Allison also realizes that information and technology available today can greatly enhance her ability to target her instruction to the unique needs of her students. This pressure means she has to make some tough decisions about the use of her time. What efforts will provide her with most value for the time expended??

And here is what our group–with practically no instruction–came up with. It’s pretty decent, if I do say so myself.

1. Identify two response systems available to classroom teachers. How do they work and what is the application(s) in her math class?

There are two very well know options for Allison, but they will come at a cost for the whole class. The first system is the TurningPoint System from Turning Technologies. These clickers can be used in conjunction with PowerPoint and the company states that the system “provides advanced capabilities such as cell-phone style text entry for short answer and essay questions” as well as a “self-paced test mode for individual summative assessment that works in conjunction with the newly released TurningKey software” (2011). With great ease, Allison can monitor each student because they each have a specific ID number and collect data to show progress in a very easy way that does not take as long as a pen and paper assessment or necessarily be part of the students’ grades.

Another excellent instructional and assessment tool is made by SMART Technologies. The instructor does not necessarily need a SMART interactive whiteboard to use this technology as long as the program is installed on a computer with a projector. This system is perfect for secondary students because it creates a collaborative, interactive environment and at the same time can give a teacher insight into a student’s progress in the class. Questions can be asked spontaneously during the class or programed with multiple responses or show different ways of solving problems. It is a complex system that gives both the teacher and students feedback. The SMART Response XE interactive response system has a Teacher Tools features used to create class lists, and view and manage all student assessment data in one location with a “built-in gradebook to record test scores, track performance, evaluate trends, create reports or export results into third-party gradebooks for further analysis” (SMART, 2011).

2. Identify two adaptive testing programs. Compare the value of a response system to the adaptive testing programs. How often would testing be practical? How difficult is it to utilize the information generated? Do they test information needed to meet your curriculum requirements?

Measures of Academic Progress

Measures of Academic Progress (MAP) testing is a computer-adaptive assessment program that is “used in more than 10 percent of K-12 school districts nationwide and in more than a third of districts in the Midwest” (Brandt, 2010). Tests in reading, language arts, math and science are available for the suggested levels of grades 3-10, though they could be administered to students above and below those suggested levels. As students take these tests, “MAP dynamically adapts to student’s responses” which “narrows in on a student’s learning level” (Northwest Evaluation Association, 2011). The Northwest Evaluation Association, a not-for-profit organization and MAP testing developer, recommend that a school/district administer MAP tests 3 to 4 times a year (fall, winter, spring and summer), though the actual amount of testing done is up to each individual school/district.

Each MAP test is scored on the Rasch unit (RIT) scale that is independent of the grade-level of the student taking the test (RIT charts can be accessed at http://www.nwea.org/support/article/1140/rit-charts-map) and is correlated with “assessments in all 50 states and the District of Columbia” (Brandt, 2010). Each student’s overall RIT score is available immediately after test completion, with more detailed teacher reports available within 24-72 hours after every student has completed the testing cycle. These more detailed teacher reports break down the overall RIT score into its component parts. For example, a language arts RIT score is broken into individual scores for “word reading/fluency/vocab”, “reading comprehension”, and “literary response and analysis”. These RIT range scores allow a teacher to focus in on specific areas in need of improvement. One could target an area of concern for a single student or group students together to receive more intensive academic interventions. This data could be used to differentiate instruction for struggling and gifted students alike. (A sample teacher report with detailed explanations can be accessed at http://www.nwea.org/sites/sitereview.nwea.org/files/resources/Class_Report.pdf.)

Unlike the two response systems discussed above, MAP testing will not provide the frequent, immediate results that can be obtained with a class quiz, test, or poll. The information obtained from MAP testing can primarily be used for the following: determining grade-level trends, individual goal-setting and placement, and as a predictor for performance on state standardized tests. Because RIT scores are independent of grade-level and MAP tests are computer adaptive, administrators and staff can use student and grade-level data to draw general conclusions concerning year-to-year academic trends. If there is a noticeable, consistent trend found in the scores, teachers should adjust their curriculum accordingly to address gaps/needs in the student population. Not only can the RIT scores help identify larger school/district-wide trends, but the individual scores and their component parts can be used for individual need analysis, as discussed earlier, and goal setting for the next round of testing. By comparing year-to-year scores (fall to fall or spring to spring) students and teachers should be able to see growth in RIT scores. Administrators and teachers could also use the individual RIT scores to group/track students for more specific instruction. Finally, individual and group RIT scores could be used as a general predictor for performance on state standardized tests since they are correlated with state proficiency standards.

Scantron Performance Series ™

An alternate adaptive testing program is provided by Scantron, the company whose main product has been part of objective testing for decades. Since job one of effectively using assessment is to create in schools a “data culture,” and since the “short weekly quizzes and instant response system” that Allison is using are not sufficiently informing parents or her instruction, she should think about an Internet-based, adaptive testing program such as the Scantron Performance Series(tm). If Allison were to leverage the lab she has at school with the instantaneous reporting function of this program, parents could be made part of the “data culture” and make interventions at home where important learning is often foregone, for lack of meaningful data upon which to make valid educational decisions. Allison and parents would get “real-time,” current data from the Scantron Performance Series(tm) for each individual student. Together, parent and teacher would be able to assess the student’s performance and work together to make interventions.

According to Steele and King (2005), educators need to consider three criteria when shopping for an adaptive testing program. They write, “…it is important to keep in mind the amount of effort needed to administer the assessment…, the speed with which results can be obtained, and the richness of the information they provide” (144). According to its users, the Scantron Performance Series(tm) fulfills the criteria, being easy to use, instantaneous in its results, and tied in with rigorous common core standards. The results are allegedly full of rich “in-depth knowledge” of each child’s instructional needs. If her school district can afford it, the Scantron program would appear to be a good solution to her problem.

3. Is the cost of either system justified based on the value added to the instructional program?

Scantron Performance Series and Measure of Academic Progress (MAP) both provide excellent results and feedback from students. Both testing systems offer a wide range of progress monitoring for many of the different types of learners within our classrooms. Testing itself can only help students in so many ways. Teachers need to be able and willing to use the test results in the classroom. The most effective way to increase student achievement involves improving classroom instruction and student support services, not the use of high-stakes testing. The curriculum should draw its breath from a range of social, cultural, and academic sources, not solely from student assessment procedures. We must consider the wide range of influences that shape academic settings and prepare students by equipping them with proper tools to meet tomorrow’s challenges.

Teachers are under tremendous pressure to make sure students succeed on standardized tests though. Scantron Performance Based Series is estimated to cost around $2.85 per student while MAP averages $13.00 per student. Both systems cost several thousands of dollars for any district, but if these tests prove to provide teachers with the resources the students need in the classroom then the cost is worth the instructional time dedicated to analyzing and improving classroom instruction. Investment in the education of children has been played on system accountability with a focal point on testing. There is no objection to accountability and testing itself because appropriate assessment practices are critical to accountability and improved teaching practices. Opposition, however, is specific to mandates associated with single measures of assessment though standardized tests. The two systems mentioned above provide questions based on state standards and challenge students according to their answers for more reliable scores reported to the schools. Students are then placed in appropriate group settings that meet their needs, which in turn provides a more concentrated curriculum for each student. All students can succeed when provided with the tools meeting their educational needs.

Teachers are mediators between the world of students and the world of state-mandated standards of education. If assessments are used for the purpose of guiding curriculum decisions, students have a greater opportunity for innovative instruction and intervention. But if assessments are used solely to determine the order, rank, and promotion of students, the teacher is placed in the position of having to resolve the conflicts that arise in the process. The tough reality is that stakes for many students are quite high. Misunderstanding and confusion between what is expected of students and what is expected of educators may create additional tension in the classroom. The dual responsibility of the teacher, to those who conceive standards, and to the students, increases the stress that exists in any educational situation. Teachers must be given the tools to mediate effectively between the state-mandated curricula and the students.

References

Brandt, W. C. (2011). The impact of the measures of academic progress on differentiated instruction and student achievement. Retrieved from http://ies.ed.gov/ncee/edlabs/projects/rct_245.asp?section=ALL.

Nothwest Evaluation Association. (2011). MAP: Measures of academic progress. Retrieved from http://www.nwea.org/products-services/computer-based-adaptive-assessments/map.

Steele, J. L. and J.E. King (2005). “Planning to assess progress,” Chapter 7 in Boudett, Kathryn Parker, Elizabeth A. City and Richard J. Murnane. (2008) Data Wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, Massachusetts: Harvard Education Press 2008.

The Wikiness