Showing posts with label data teams. Show all posts
Showing posts with label data teams. Show all posts

Monday, December 8, 2014

Data Teams and the Debate Over the College Football Playoffs

On Saturday, as the college football regular season was winding down and Twitter was flooded with arguments about which teams should and should not be selected for the first NCAA FBS playoffs, I sent out the following tweet:


It turns out that the selection committee for the 4-team playoff does have a protocol for choosing teams, and it's been public since before the season started. It's not exactly a rubric aligned to standards, but it does contain specific criteria for committee members to consider. From what I can see, the committee followed their own rules perfectly when they selected Alabama, Oregon, Florida State, and Ohio State to compete for the national championship. 

Source: www.sbnation.com

The most controversy came from the fourth seed, Ohio State. Many college football fans, especially those living in the state of Texas, felt the final spot in the playoffs should have gone to either Baylor or Texas Christian University. They pointed to Ohio State's early season loss to an average Virginia Tech team, and to its relatively weak strength of schedule. They also argued that only a week before, the selection committee had ranked TCU as the third seed, and that dropping the Horned Frogs to number six after a 55-3 win over Iowa State didn't make any sense. Baylor fans pointed out that all of this was true, but added that Baylor had beaten TCU during the regular season, so Baylor deserved the spot over either Ohio State or TCU.

In the day since the selections were announced, I've heard all sorts of sports commentators give their reasons about why they do or do not agree with the selection committee. What I haven't heard from anyone in sports journalism or the Twitterverse is any reference to the actual selection committee protocol. You can read the entire document on the College Football Playoff website, but for the purpose of this blogpost, here is the tie-breaker section:


With these four criteria, Ohio State, who won the Big Ten championship game 59-0 over Wisconsin, comes out ahead of either Baylor or TCU, who tied for the Big 12 regular season championship. (The 10-team Big 12 does not have enough teams to have a championship game.) According to the highly-respected Sagarin rankings, TCU's strength of schedule was ranked 42nd, Ohio State's was 52nd, and Baylor's was 56th. The only head to head competition among the three teams occurred when Baylor defeated TCU 61-58 on October 11. Finally, Ohio State did not share any common opponents with either of the other teams, but Baylor and TCU both played every other Big 12 team, as well as non-conference opponent Southern Methodist. Against their eight common opponents, Baylor's record was 7-1 (including a loss to West Virginia), while TCU's record was a perfect 8-0. 

For the selection committee, Baylor's head-to-head victory put them ahead of their Big 12 co-champion TCU. Unfortunately for Baylor, Ohio State was the outright Big Ten champion, and the Buckeyes' strength of schedule was ranked higher than Baylor's. An objective evaluation of the three teams using the selection committee's protocol makes Ohio State the logical choice for the final playoff spot.

Like the College Football Playoffs selection protocol, the data team process helps us look objectively at student progress at Southeast Polk. In Data Director, there is a massive amount of assessment data. Many people outside of education believe that our use of this data means we see our students merely as numbers that need to be moved from one column on a spreadsheet to another. While it's certainly true that we want our non-proficient students to become proficient on any given standard, that view fails to recognize what happens when a group of educators collaboratively combine assessment data with their own expertise that comes from daily interaction with students. The collaborative data team process helps humanize the data and keep the focus on students instead of mere numbers. On the other hand, the same process helps us see beyond our biases to make informed decisions that are more likely to benefit students in the long run.

Saturday, November 22, 2014

The Signal and the Noise

The signal is the truth. The noise is what distracts us from the truth.
--Nate Silver, The Signal and the Noise

The Signal and the Noise is my favorite book about statistical analysis. (Yes, I have a favorite book about statistical analysis. Don't judge.) The book's author Nate Silver runs a website called FiveThirtyEight.com, a name derived from the number of electoral votes in United States presidential elections. Silver is most famous for correctly predicting the electoral results of 49 out of 50 states in the 2008 presidential election, and then topping that feat by correctly predicting all 50 in 2012.

In the book, Silver discusses how we humans naturally observe and seek out patterns, and how we often fail to successfully make accurate predictions based on those observations. This is in part because there is so much information out there, and much of the information is noise. It distracts us from what we really need to know to accurately analyze the information available to us.

Thanks to unit pre and post tests in math and ELA (as well as those that are now being developed in other subject areas), we have a wealth of data about our students and their proficiency on a wide range of Iowa Core standards. As Southeast Polk's assessment coordinator, I have been looking for ways to share more and more data with teachers, coaches, and administrators to help facilitate our common goal of raising student achievement. Using Data Director, Infinite Campus, and Excel, I've been able to develop reports not only on overall test data, but also on subgroups based on IEP, ELL, TAG, and gender. I've gone back and found historical data in these areas as well. And I think this is probably just the tip of the iceberg in terms of what sort of data could be available to everyone.

The problem becomes how to deal with all of that data. How do we decide what is important right now, what needs to be observed for future patterns, and what can be dismissed? How do we separate the signal from the noise?

The answer lies in the data team process. A strong data team has the ability to hone in on the signal, on the important truths that our unit assessment data can tell us about what our students are doing well, and in what areas they need more help. I've been privileged enough to attend a few data team meetings since the start of the school year, and I'm looking forward to hopefully making them a more regular part of my schedule as the year progresses.

If you're interested in reading The Signal and the Noise, it's available at Amazon.com and many other retailers. I also have a copy of it in my office that I would love to loan out. Send me an email if you want to borrow it!