A compelling vision of the data-driven future of K-12 schooling? Or a chilling description of a brave new educational world in which even students' smallest actions are converted to digital data and used to build permanent "learner profiles"?
A new report from London- and New York City-based educational publishing powerhouse Pearson is likely to generate both reactions, depending on whom you ask.
Released this week, "Impacts of the Digital Ocean on Education" is intended as "an aspirational vision of what success might look like" in the rapidly changing world of "big" educational data and personalized learning.
Report authors Kristen DiCerbo and John Behrens of Pearson's Center for Digital Data, Analytics, and Adaptive Learning sketch out a vision in which end-of-year, summative tests of narrowly defined skills and content knowledge are replaced by a constant stream of digital data generated by "in vivo naturalistic tasks" that thoroughly blur the line between assessment and instruction.
Such data—generated from a variety of sources and activities, and focusing on students' social connections and interactions, rather than just their isolated individual experiences—would be constantly tracked and used to update profiles that follow each student across classrooms, grades, and schools, helping facilitate more customized learning experiences for each.
"The devices and digital environments with which we interact are designed to record and store experiences, thereby creating a slowly rising ocean of digital data," DiCerbo and Behrens write in the report. "We believe the ability to capture data from everyday formal and informal learning activity should fundamentally change how we think about education."
Just as big data and analytics have transformed finance, insurance, retail, and professional sports, the report says, they will change education. Until recently, the authors write, data collection and storage was expensive, limited, and isolated, and students' educational records were not portable, easy to share, or able to be quickly analyzed.
But the digital revolution has changed that reality, DiCerbo and Behrens contend. They argue that the abundance of increasingly fine-grained data available to educators "can help pinpoint the moments when learning occurs or a learner's approach to a problem changes" and then be used to help tailor suggestions or recommendations to help each student's learning continue.
'Ocean' of Digital Data to Reshape Education, Pearson Report Predicts
By Benjamin Herold on March 19, 2014 11:24 AM
A tidal wave of data could personalize learning
Pages
▼
12 months ago, you could raise funding for your startup overnight if you were doing "big data" for education (even though the data was quite paltry in size. The future hope of a Diamond-Age like mechanism to have kids worked working in flipped classrooms at their ZPD all without teachers needing to do anything to differentiate instruction was the holy grail.
ReplyDelete6 months ago, the perfect storm of Anti-Common Core backlash, NSA domestic spying, and IRS abuse of 501c4 citizen groups meant Bog Data for Ed wasin serious jeopardy.
I first realized this at a talk I was giving at a school that had chosen in May of 2013 to improve their math program: a goal of Singapore Primary Math in K-6, Shoseki in 7, Dolciani in 8, complete with real math training for all teachers and a FT BS math degreed teacher for 6-8. Oh, and state standrads-based standardized tests for the first time ever to measure student progress relative to their peers and state requirements.
The parents were on board in May.I gave the same talk in September, and the climate had changed.
Parents were so anti-data that they wanted *no standardized test*. Never mind that there was no way to measure student progress without a benchmark.
In order to make some basic determination about where these students were mathematically that first May, MSMI had devised a test based on state sample item. In September, I actually had a parent say to me "I know you took data on my child."
Another parent who stood by as I spoke to a family asking for how I could help their child get a bit of higher content in their math class said "I heard you ask that family for their child's name and grade."
The paranoia was rampant. The push to "opt out" so great that we were in serious jeopardy of having any way of measuring if we;d done any good, let alone if kids had learn a year's math in a year's time.
Parents no longer trust their schools. This is perhaps the biggest legacy of the politicization of civic institutions. It is earned in many cases,but probably is worse in result than even lousy curricula. It will take a long long time to fix.
Puts a whole new spin on the threat of "it'll go on your Permanent Record" ~
ReplyDelete"Parents no longer trust their schools."
ReplyDeleteAbout time. Maybe parents are getting some of that "grit".
I wonder what the response would be if parents and students started collected data like that on teachers, administrators and so on up the line.
BL
Has anyone proven the use of "big data" increases student learning? On tests not written by the "big data" salespeople?
ReplyDeleteI do believe the companies have a product to sell. I haven't seen anyone produce any proof that it works. At all.
There can be "cargo cult" thinking on all sides, so how do you get people to really think and dig deeper? What framework can be used to allow good ideas to flourish and lead the way, unencumbered by others? There could be good Core Knowledge solutions, good Harkness Table solutions, and good unschooling solutions. Some schools will push and some not. Who gets to decide?
ReplyDeleteIn K-6, most schools hate tracking and separating kids by willingness and ability. This is a problem of philosophy (pushing or not), and a problem of whether there are many track-jumping paths. Many prefer that the pushing (help and tutoring at home) stay hidden. Tracking happens, however, but the option is not available for many students. The inability of K-6 public education to deal with this issue is the cause of so many problems. This problem goes away in high school, but for many students, it's too late. Worse yet, people will only see an IQ problem. My son is bright, but I had to ensure that he mastered all sorts of skills in K-6.
K-6 pedagogues try to have it all. They want a natural, full inclusion, differentiated instruction process that brings all kids up to their best academic level, while ignoring parents who constantly tell them what they have to do at home. Ironically, on top of that, they ask parents to practice math facts at home. It's all quite incredible.
The number one parent priority here is that children graduate with their age-mates. Number two is that the right kids are in the honors classes. With data, the GOB applecart is in danger of being upset, just as it was with nclb when staff heads rolled and students were forced in to remedial.
ReplyDeleteNiels - NO KIDDING!
ReplyDeleteThe amazing thing to me, here in Irvingtonland, is that "Permanent Record" seems to mean nothing to our super & board re: their own permanent records.
I've taken to pointing out that the super has just said thus-and-such *on camera,* and it has no effect whatsoever.
Meanwhile, I watched a BOE meeting in which I'm on camera for a minute or so, as a member of the audience, obsessively playing with my hair, and that section of the tape is damned embarrassing.
ReplyDeleteBL wrote: "I wonder what the response would be if parents and students started collected data like that on teachers, administrators and so on up the line."
ReplyDeleteToo bad you weren't around for the early days of ktm, when I used to post "Problems of the Week" from Chris's then teacher.
Here's what happens when parents collect data on teachers & name names: the union publicly threatened to sue me in a board meeting.
The PTSA turned out to assure the teachers that 'we love them.' (I think those were the words--)
I know this because an anonymous well-wisher in the PTSA sent me the email correspondence with the then-superintendent.
BL wrote: "I wonder what the response would be if parents and students started collected data like that on teachers, administrators and so on up the line."
ReplyDeleteToo bad you weren't around for the early days of ktm, when I used to post "Problems of the Week" from Chris's then teacher.
Here's what happens when parents collect data on teachers & name names: the union publicly threatened to sue me in a board meeting.
The PTSA turned out to assure the teachers that 'we love them.' (I think those were the words--)
I know this because an anonymous well-wisher in the PTSA sent me the email correspondence with the then-superintendent.
cranberry - exactly!
ReplyDeleteI am very skeptical of the big data movement.
Siegfried Engelmann is quite clear on the assessment issue; as I recall he says you can get a crystal clear sense of where the kids are with a few questions asked in-class.
Morningside Academy, which has to be the ultimate in use of individual data to 'drive instruction' (such awful terminology!) does **not** use big data.
At Morningside Academy everyone uses "celeration charts," written in pencil & by hand, and the kids do their own charting with adults checking the charts from time to time to make sure the kids are doing it right.
For the record, the grownups don't check to see whether kids are 'cheating' or anything of that nature; there's nothing at Morningside to cheat FOR.
The grownups just check to make sure the kids are recording their times properly.
Morningside is built on fantastically good curricula they know works.
ReplyDeleteBig data is built on the idea that if you constant assessment and reteaching equals individualized instruction.
Catherine, Morningside does use norm-referenced standardized testing on a regular basis, I gathered less to "drive instruction" than to monitor overall achievement by grade, subject, cohort, etc. They use one of the basic standardized achievement tests (can't remember if it's the SAT-11, the Iowa or what), as well as the Woodcock-Johnson achievement battery. Kent Johnson said they go over the results as a staff to make decision about curriculum, texts, etc. They embraced Bob Dixon's Reading Success based on what they saw in test results, for example.
ReplyDeleteThey make use of standardized (norm-referenced) tests in the partner schools they work with, as well.
But as you pointed out, to determine ongoing instruction, they use the data from the celeration chart.
Right - they use the ITBS, and they base their tuition refunds on the ITBS.
ReplyDeleteWhat's being proposed re the "ocean of data" is completely and totally different from what Morningside does.
I'm seeing Matrix-like fantasies of perfect and perfectly pervasive data at all time. I've even seen references to data being able to tell teachers the exact moment in which learning occurred inside one student's brain.
"Big data" has nothing to do with standardized tests in the sense that we know standardized tests.
Conventional standardized tests are the form of data 'Big data' hopes to replace.
Catherine, honestly, you don't know enough about so called Big Data to make that determination one way or another. And I can make neither heads nor tails of whether you support or disapprove of standard sized tests
ReplyDeleteBoth my husband and I were trained in and did research in CS n what is now called Big Data. My first real job was waiting algorithms to find explosives in cargo (this was pre Sept 11) and I've done all kinds of data mining and computational learning theory and machine vision etc etc. Since; my husband has worked for over ten years in big data on retail customer data writing tools and gathering patents in how statisticians use their models to get results.
Data analytics works well enough to predict who you are and most everything about your preferences. The negative reaction to this is not because the analytics don't work it is Because they Do. And parents are scared of their child being so utterly well defined and that becoming known. If it didn't work no one would care.
There are numerous ways that data analytics could be used to wonderful effect in education. There are numerous ways it could drive instruction. To declare no one anywhere is doing them is absurd.
You want distributed practice? Then data analytics tell you which items you need to keep practicing frequently and which you can cycle through less often.
You want gifted kids getting something out of their time in a flipped classroom? These tools do make that more likely.
NWEA uses at least some adaptive testing. Their MAP scores are trusted in countless districts now for helping show what that child's progress on a year could be above and beyond one calendar for everyone.
Even the simplest collaborative filtering model would help find kids books they like at their level based on what other kids at their level like in schools lacking ft librarians.
This isn't worthless.
Itbs and other achievement tests aren't normed against standards just against others. Helpful for a school trying to get their kids to the mean. Pretty darn useless for seeing how far that kid can go relative to their capabilities or compared to how far they should get in a year's time.
The privacy issues are real. Dismissing data analytics in education out of hand rather than addressing them just means the privacy issues won't be addressed.
"waiting algorithms"-->writing algorithms. sorry, aggressive spell check.
ReplyDeleteSo if it's so wonderful, there's nothing wrong with proving it. See if it works with real students, real teachers, real administrators, real politics.
ReplyDeleteBecause as far as I can see, there's a huge resistance in public education to the implications of the data they do have. Creating a clearer pattern doesn't help things. They already know who's doing well in school--and there's no desire to increase the performance of the top end. Rather the opposite.
So far, my children's educational outcomes could have been predicted at about the age of 7 or 8. The IQ tests administered at that time provided results which really mean there are no surprises.
At one point, schools did administer IQ tests, and they did use them to track students. They don't do that now. The SAT has been purposefully moved away from measuring IQ. Because the answers aren't comfortable.
If the public schools do use data analytics, they'll use them to discover which kids they can ignore. Because, you know, "they'll do fine."
Yes, Cranberry: the main issue for many districts is finding things in the data that people don't want to know.
ReplyDeleteIt isn't that data analytics doesn't work; it is that it does.
But schools exist that deal with reality, and they use data. Whether private or charter, they exist. Good schools want to know.
In our physical environment, we have widely accepted patterns that apply – for instance defining which spaces are more private than others – and we’re more comfortable when these patterns are followed. (For anyone who's read it, I’m thinking here of Alexander’s “A Pattern Language”, and specifically Intimacy Gradient.) For instance, a bedroom is more private than a living room, which is more private than a front porch.
ReplyDeletePerhaps the discomfort about data collection and use in education has to do with which data we, or some group of us, think is at the right intimacy level for a public institution like a school, or the government. There isn't agreement on what is appropriate yet, and excessive intrusion seems likely thanks to some of the things that have been done.
Personally, I like NWEA’s MAP tests – I thought it was interesting data to have when my kids were in schools where those tests were given. I wished that the teachers used it. Weight tracking? That seems relevant at the doctor’s office but not really appropriate for a school to collect.
"the main issue for many districts is finding things in the data that people don't want to know."
ReplyDeleteThat works for schools too if they continue to not ask parents what they do at home.
Our high school just sent out a link to a technology questionnaire for students, parents, and teachers. Here is one prompt followed by several questions they ask. You are supposed to answer between 1 (not at all) and 5 (significantly)
"When technology is successfully integrated into classrooms, students…"
"are more organized"
"are more engaged in class assignments"
"develop into lifelong learners"
"improve their academic achievement"
"are more likely to engage in global issues"
They have quite a few of these questions where there is no button for: "That's a stupid question"
When our K-6 schools collected reams of data from our current NCLB state test, they found one time that their "problem solving" score went down." Their solution was to increase teacher work on problem solving. It really was "big data", but they weren't testing and collecting the right data.
There is nothing wrong with lots of data. It just depends on what that data is and what you do with it. They could collect data on IQ. It's not the data that bothers me, but the people behind the data. They usually have an agenda to push.
There is also the statistical problem of big data. Problems are all defined by trends and standard deviations. However, you really don't know a lot of what is going on unless you analyze specific, individual problems. Good results don't necessarily tell you why they are good, but studying a single bad result or failure can uncover a problem that really applies to many more students. Big data assumes that you know what all of the problems could be beforehand.
In programming, a single error is golden. You have a tangible mistake to analyze and trace back to the problem. It could be that the problem really is only a special case, but often it leads to an understanding that would not have been predicted. Big data questions won't find many of those things because you don't know what to ask.
You have to look at the code line by line and trace back to the error. That's not done statistically. Big data might find some things you don't expect, but more often it's used to validate one's expectations.
--here is also the statistical problem of big data. Problems are all defined by trends and standard deviations.
ReplyDeleteHm...the law of large numbers applies here. st deviations get small, outliers clustered in known dimensions.
--However, you really don't know a lot of what is going on unless you analyze specific, individual problems... Big data assumes that you know what all of the problems could be beforehand.
This is not what people doing data analytics assume--they are doing more interesting things than this.
They really do ask very specific questions, like what is the likelihood that a woman of age 24-39 is going to enter a Target store on Thursday given she has a $5 gift card in her hand, vs. she doesn't have it? What is her expected market basket worth? etc.
Here is a nice article on just how predictive analytics are.
http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=1&_r=2&hp&
The issue here in math ed is: what *are* the interesting questions?
i can tell you one thing. Right now, we have nowhere near the depth of understanding of the key components of numeracy the way we do literacy. We have almost no diagnostic tests to say when a student isn't solving math problems, which component skills are the issue. Real data for cog sci folks really would help, as would well defined samples in order to create such diagnostic assessments.
For example, what rate of oral or written math facts are needed in multiplication for a student to be proficient? Can we determine a student's ability to do long division just by measuring that rate? What component skills are needed for a student to see immediately that the problem
(8.6 - 2.04) / .02
is the same as
(8.6 - 2.04) x 100 /2?
What about recognizing commutativity--can a student's ability to master commutativity be measured? is it predictive?
etc. I am sure there are many many more.
I don't doubt that you can come up with useful data, but the big data we get has to do with "problem solving", not whether a student can add any two fractions. Besides, big data solutions are too late. Teachers test in class and they should know whether Jimmy can add two fractions or not. They are the ones who have to solve the problem right then and there. They can't just spiral kids along and wait for statistical big data to tell them what they should already know - that Jimmy (or many in the class) can't do fractions.
ReplyDeleteIt's not so much whether state or national tests collect the right data, it's just that it's too late and only of use for low cutoffs. It won't help Jimmy. However, if you help teachers help Jimmy, then you won't need big data.
I helped my son at home. I don't care one bit about NCLB or CC for him. His high school honors and AP teachers aren't sitting around waiting for big data feedback. They know what's what - or should.
"Big data" in the industry is terabytes a day at minimum. Usually peta. We aren't talking about 500 questions total where 100 kids max see the same set. We are talking about tens of thousands of questions and hundreds of thousand kids...preferably weekly or daily.
ReplyDeleteOf course there is good reason to be skeptical. But it helps to understand what the serious people in the field mean and then evaluate whether anything serious can happen in education. It's not big data that is likely to be the source of failure here.
_Scored_ by Lauren McLaughlin. Good read to see where this imaginary future goes.
ReplyDeleteOf course, as many point out, the fundamental problem is one of competence. We will never see accurate big data in the educational industry. We'll never see A/B testing to determine optimal teaching strategy. We'll see more tests, more sales, more theories, and more ed school clowns in the middle of it mucking it all up.
What, exactly, is the problem? Well, that's one of the problems. There is no one problem and no one fix. Just for K-6 we have:
ReplyDeleteLow expectation problems
Pedagogy problems
The problem of social promotion or whatever they want to call it.
The problem that no school ensures mastery on a grade-by-grade basis
The low teacher math knowledge problem.
The problem of not knowing what parents do at home
The problem of not separating the willing and able from those who are not
Big Data assumes that people want to solve problems, but just can't see the problem tree in the forest. But Big Data won't solve any of the problems above, and they are the major ones. Fix those and the need for Big Data almost goes away. People are looking for a process that hides the need to confront very obvious and fundamental problems.
Big Data, typically collected yearly, can only be used as part of a global feedback loop. It won't help little Jimmy who can't do fractions, but those are the problems that schools need to fix. Fixing individual problems will fix global problems, but a global process will almost always hide or avoid problems that are painfully obvious when studied on an individual basis. Where is the local feedback loop? There isn't any, by definition. What we need is Little Data - knowlege of how to create proper classroom tests. We need teachers who know math and are are willing and able to fix problems and not let them spiral on for years.
When you have a sprialing, social promotion philosophy that assumes that Jimmy will learn when he is ready, then no amount of Big Data analysis will determine if Jimmy is being served correctly, unless Big Data also collects data on IQ and how much help other students are getting at home or with tutors. Can Big Data ever change fundamental assumptions and pedagogy? Can science and data ever fix problems that are based on belief?
Sheldon Cooper: This is my home now. Thanks to you, my career is over and I'll spend the rest of my life here in Texas, trying to teach evolution to creationists.
ReplyDeleteMary: You watch your mouth, Shelly. Everyone's entitled to their opinion.
Sheldon Cooper: Evolution isn't an opinion, it's fact.
Mary: And that is your opinion.
Sheldon Cooper: [to the others] I forgive you. Let's go home.
[leaves]
Mary: Don't tell me prayer doesn't work.
StevenH has a point. Project Follow Through was the big data of its era, but it disagreed with the consensus of ed theory, so it was ignored and denigrated.
ReplyDeleteI see no reason why modern Big Data will have a different fate. Data that supports mainstream thought will be embraced. Data that doesn't will be ignored.
-- Auntie Ann