"Figures don't lie, but liars figure."
-Mark Twain
"I shall try not to use statistics as a drunken man uses lampposts, for support rather than illumination."
-Andrew Lang
Do you remember Shane Battier? Unless you're a passionate basketball historian and/or a statistics geek, probably not. Battier played basketball at Duke University and went on to play for several NBA teams, but he was never a leader in scoring, or rebounding, or assists-- in fact, none of his statistics were all that impressive. Still, Duke won a national championship with Battier. The Miami Heat won a championship with Battier. Twice. In fact, Battier is the only player in NBA history to be part of two 20-game winning streaks, on two different teams (the Heat and the Houston Rockets).
So how come Shane Battier was not considered a superstar? Battier himself said, "They (other players) think of me as some chump." According to a 2009 article in the New York Times, "Here we have a mystery: a player is widely regarded inside the NBA as, at best, a replaceable cog in a machine driven by superstars. And yet every team he has ever played on has acquired some magical ability to win."
What's the deal with this guy?
It turns out that Battier did a lot of things that are essential in basketball. He instinctively ran to the empty spaces to balance the court and get high-percentage shots. He almost always got a hand up in a shooter's face. He stripped the ball from a shooter's hands on the way up for a shot. When shots went up, he boxed out the other team's best rebounder -- even when he wasn't guarding that player.
The question we should be asking is: Why didn't more people immediately recognize and reward Battier's actions? What's the deal with basketball statistics? Everything Battier did was visible (if you knew what to look for), describable, and measurable -- so why didn't they count in the box score?
Many students do things every day that don't get recorded in their academic records. You help a classmate with his homework. You ask the question in class that everyone secretly wanted to ask because the teacher's explanation made no sense. These actions are important-- they help you learn and they help the people around you learn. People see your efforts and benefit from them. But it still doesn't help your G.P.A.
We tend count the things that are easiest to describe and measure. It's easy to count a player's point total or how many free throws she attempted. But nowhere in a traditional box score will you see a statistic for diving out of bounds, or separating your team captain from the referee just before she gets a technical foul, or a thousand other discrete actions that contribute to a team's success.
This has changed in recent years. Many professional and amateur sports have developed different analytics to better understand their games and make decisions that support success. Baseball made such a science and an art out of analyzing data that there's even a movie about it starring Jonah Hill and Brad Pitt. As an early innovator in "moneyball", baseball executive Theo Epstein was credited with using data and evidence-based analytics, including lots and lots of computer-crunched statistics, to guide the Boston Red Sox to a World Series Championship in 2004 (their first since 1918) and the Chicago Cubs to a World Series Championship in 2016 (their first since 1908). Baseball has gone so far to adopt new categories of statistics to analyze player performance. New stats such as Wins Above Replacement, On-Base Plus Slugging, and a heap of sabermetrics help managers and team executives decide everything from which players they need to where to place their fielders for a specific pitch.
Analyzing data can certainly give us insight. However, numbers -- no matter how cleverly they are arranged and described -- don't do a very good job of helping us understand intangible things like curiosity, or creativity, or passion, or resilience. After a graduation speech he gave at Yale, Theo Epstein himself put it this way:
"One of the great ironies of the digital information age is
there is so much information out there, so much data, so many
statistics, that it's easy to attempt to precisely quantify a player's
contribution. But you can never really quantify a human being, can't
really quantify character, and that stuff does matter, especially in a
group situation where players really do have an impact on one another. …
I still think data is important; it can give you some empirical facts
about a player. Objectivity is important, but you have to combine it
with an understanding of the player as a human being. Chemistry is
really hard to pinpoint. It's really hard to discern the magic formula.
So what do the lessons of Shane Battier and Theo Epstein have to do with how we evaluate the learning performance of K-12 students?
In school, we tend to count the things that are easy to count. Usually this boils down to completed work and fractions consisting of right answers over total answers that we can convert into percentages for the sake of grading and comparing people. It's easy to count the number of paragraphs a student creates and measure that against a five-paragraph essay assignment, but apart from following instructions, this indicator is so superficial that it's nearly meaningless.
This is a problem. Statistics -- in schools, this means test scores -- are poorly understood and frequently used to persuade unsuspecting customers, voters, and others. The things that are easiest to count, such as a baseball player's batting average or the number of points a basketball player scores, or even a student's answers on a test, is a poor indicator of that person's talent, effort, character, future performance, or fit with a organization's culture or "team chemistry."
The number of items a student answers correctly on a test tells us nothing about how she thinks, or how well she's learning, or how well she can apply the knowledge from the test to something meaningful in her life, or how well she can solve problems, or see opportunities, or... you get the idea.
It's worth taking a moment to ask: What can we tell from a correct multiple choice answer on a test?
Only that the student colored in a particular bubble or made a circle around a particular letter.
We have no idea WHY the student answered this way. Maybe the student knew the correct answer and selected it with confidence. Maybe the student had a pretty good idea and guessed correctly. Maybe the student had no clue and got lucky. Maybe the student read everything wrong and selected the right answer for the wrong reasons. Maybe the student was bored out of her mind and was taking a mental vacation on a beach somewhere while she doodled the same answer for every question down the whole column on the answer sheet.
Most tests don't offer much in the way of insight or progress over time because they are summative, i.e., students can't take them again with the benefit of additional instruction or practice. Putting these sorts of test scores together is just a string of moments without any thread to connect them. This practice makes it easy to create a semester grade online, but it seems woefully inadequate to authentically describe anything about learners or the work we do to improve.
That's the problem with statistics and learning. Here's the solution. On the first day, our learning community decided to adopt Open-Source Learning principles to guide our practices this year. We decided to run the course openly and curate our learning journeys online.
Since we Open-Source Learners use a variety of digital media to tell our stories, we create a mountain of data in the process. This includes both quantitative data (things we can count, like how much we post/write/comment) and qualitative data (things that are not numerically describable, like how well Jeronimo recited "Richard Cory" from memory in class). Let's begin thinking about elements of reading, writing, literature, Mental Fitness, Physical Fitness, Civic Fitness, Spiritual Fitness, and Technical Fitness are important enough for us to evaluate (both quantitatively and qualitatively) as we move forward.
To start the conversation, here are some numbers from our first week. Have a close look and please let me know what (if anything) you think they mean by commenting to this post. What matters in evaluating performance? What numbers should we look at, in momentary snapshots and/or over time? What qualitative data should we consider? Looking forward to your thoughts.
Course blogs: 2
Members with blogs listed on the Member Blog pages: 167
Members without blogs listed on the Member Blog pages: 10
Members without blogs listed on the Member Blogs pages who haven't been absent at least 2 days: 2
(Can't wait for Monday to ask Maira Gonzalez & Jesse Ruiz how I can help them.)
Total (attending & non-attending) student participation: 94%
Total (attending) student participation: 99%
Course blog page views during first week: 3436
Course blog posts: 33
Course blog comments: 78
Course blog followers: 40
No comments:
Post a Comment