kitchen table math, the sequel: Wisconsin introduces competency exams!

Friday, January 25, 2013

Wisconsin introduces competency exams!

From College Degree, No Class Time Required:
David Lando plans to start working toward a diploma from the University of Wisconsin this fall, but he doesn't intend to set foot on campus or even take a single online course offered by the school's well-regarded faculty.

Instead, he will sit through hours of testing at his home computer in Milwaukee under a new program that promises to award a bachelor's degree based on knowledge—not just class time or credits.

"I have all kinds of credits all over God's green earth, but I'm using this to finish it all off," said the 41-year-old computer consultant, who has an associate degree in information technology but never finished his bachelor's in psychology.

Colleges and universities are rushing to offer free online classes known as "massive open online courses," or MOOCs. But so far, no one has figured out a way to stitch these classes together into a bachelor's degree.

Now, educators in Wisconsin are offering a possible solution by decoupling the learning part of education from student assessment and degree-granting.

Wisconsin officials tout the UW Flexible Option as the first to offer multiple, competency-based bachelor's degrees from a public university system.

By CAROLINE PORTER
January 24, 2013, 6:32 p.m. ET
Wall Street Journal
Wow.

I'm thinking....all those disruptive technology people may turn out to be right for the wrong reason.

The coming Disruption won't be courses going online.

The coming Disruption will be courses going away altogether. (Some of them, anyway.)

The tests go online, not the courses.

I don't believe in making predictions, but even so I have simply never been able to see how MOOCs save the day. I can't stand online courses myself, my students don't like them, and I don't know any grownups who like them, either. The people touting them (and investing in them) don't seem to have taken any coursework via MOOCs themselves, as far as I can tell.

So while the logic of the MOOC -- put the best professors online so thousands can learn! -- often seems unassailable, I just don't see it. Put it this way: I don't believe in making predictions, and I continue to wonder what I'm missing, but I will not be investing my pennies in Udacity, Coursera, or edX.

I have always thought, though, that there is a major use for the internet when it comes to assessment.

Maybe college competency exams are the missing piece?

Such a scenario -- MOOCs fail as a means of making college affordable, but competency exams succeed -- strikes me as possible.

First, according to the Bain/Sterling Partners report, one-third of U.S. colleges and universities are in financial trouble.

Second, the economy continues to be depressed and, absent "regime change" at the Federal Reserve,  will remain depressed.*

Given a depressed economy, I assume some colleges will close.

College closings will put pressure on state universities to expand, but state colleges are also in financial stress and have been raising tuition. They are in no position to grow.

At some point, it seems to me, political pressure will build on state legislatures to find another way to provide college diplomas.

In short, I can imagine Wisconsin's move attracting a lot of imitators, and sooner rather than later. I can also imagine a circular effect, with college closings leading to the introduction of competency exams,  and the introduction of competency exams then leading to more college closings.

That's disruption.

We'll see.

If competency exams begin to take hold, I can imagine a number of other developments that might be very interesting.


* The economy is growing but is not going back to trend as it always has done in the past, including the Great Depression. I know people hold out hope that a housing recovery will lead to a real recovery, but since I am persuaded by Scott Sumner's analysis, I don't see housing as the white horse.

chart from: historinhas
and see:
Lawyers without Law School
proposal for a national baccalaureate
why college costs so much
US News: 2-year law degrees


37 comments:

Anonymous said...

I am feeling very smug and want to brag a bit.

I sorta called this a year and a half ago :-)

At Joanne Jacobs site I wrote:

"... But ... I’d be delighted to have the DoE offer $1B per year to UMich and Cal to do the same thing [Mandate that the five service academies design degree programs which students can complete entirely by exam and license Sylvan Learning Centers and the University of Phoenix to administer the exams]. The money to be withheld if they screw around by making the on-line degrees harder than the on-campus ones. And they can’t distinguish between an on-line degree and an on-campus one (this keeps them honest about trying to make the on-line ones "real". If they make the degrees bogus, then they screw up their own brand).

Do this for ten years and see what the results are. It isn’t like the DoE would be spending the money well anyhow ...

The state college systems are in financial trouble, for the most part (not, it appears, UMich ...). Eventually, one of them is going to crack and offer an on-line (or on-line mostly) degree for ~5K/year or something like that. They will do it because creating and administering the tests is a lot cheaper than $5K/year and they will *REALLY* want/need the money.

It won’t take off until one of the "good" schools does this ... but there are lots of pretty good state schools in financial trouble. One of them will crack.

We can actually already see it starting. Michigan Tech offers a number of on-line, pretty much exam-only courses already. Right now they charge full price for these, but if we’ve already established the principle, then all we need to do is negotiate price

http://www.joannejacobs.com/2011/03/rethinking-pell-grants/
"

It looks like I got the motivation wrong, but other than that.

University of Wisconsin -- Madison ranks 41st on the USNWR rankings (which I think are bogus, but probably roughly correct). In any event, a degree from there is "real".

:-)

-Mark Roulo

Jen said...

At his home computer, huh? It sounds like there may soon be some really lucrative opportunities out there for good test-takers.

What's to prevent someone from hiring someone to take their tests and get a degree? It would certainly be cheaper to pay someone, say 10K to take your tests and get you certified. If that person did one round of testing per month, well heck, that's not a bad take, eh?

I dunno -- even if you didn't have someone else take the whole test, what's to stop you from having someone emailing you answers? Putting questions up on those sites and getting people to answer them for $1 each?

Heh, just looked down and saw the
"Please prove you're not a robot"

Please prove you actually know this content...

Anonymous said...

The test taking will very rapidly turn into something taken at a testing center.

-Mark Roulo

Auntie Ann said...

When getting my masters, I had to take x number of "masters credits." These weren't tied to any specific class, but were essentially the payment for the degree. Somewhere along the line, the Universities will have to bilk people out of that kind of money, or this won't fly. As soon as the online exams are cheaper and faster (and faster *is* cheaper) than spending 4-6 years on campus, the campuses might as well close up shop. Voila, end of the university.

Count on this: somewhere along the line, they will insist on government protection to make sure that the cost of an exam-only degree is comparable to a brick-and-mortar university. Without it, they can kiss their nice, cushy tenure positions good-bye.

Anonymous said...

"somewhere along the line, they will insist on government protection to make sure that the cost of an exam-only degree is comparable to a brick-and-mortar university"

The universities will *TRY*, but it only takes one state with a good public university system (and we have several states that meet that ...) and a governor and/or legislature that wants a reasonably priced degree.

*Plus* any university offering this to out of state kids is mostly just picking up new customers.

They would all have to collude together ... and I don't think that this is going to happen. *Someone* will crack ...

-Mark Roulo

MagisterGreen said...

IMO in the end it won't be the universities, or any school, that make something like this happen, it'll be businesses and private industry. If private industry can get what it wants from people with these "exam-only" degrees, the jig will be up. Colleges today are little more than job training centers and higher education in this country has, for decades, sold itself on the idea that you'll never get a "good" job without it. That veil is already fraying at the edges and once a tear opens up, stand aside.

Jen said...

Except that I think we will quickly find that there is a big difference between "cheaper and faster" and the actual knowledge and abilities that the new "graduates" will come out with.

Certainly, there will be a huge growth in for-profit test prep centers.

Now kids won't go to college, they'll go to test prep.

Huh. The testing companies could make a killing twice over -- charging for writing/administering/standardizing the tests and then for prepping you for them.

It would also seemingly lead to a "common core" -esque college knowledge system.

When my oldest was visiting colleges, one university explained the leanings and interests of their philosophy department and contrasted it with the different flavors at two other universities, one local, one in NYC. They strongly recommended taking at least one or two courses from the other local school or trying to do a semester or summer in NYC.

That sort of thing would pretty much vanish, though, right? If there is ONE big test, then that's what you need to know.

Jen said...

Ohhh, okay, I just thought right now of what I wouldn't mind seeing (since I've been pretty gloomy about the outcome of all of this in my other two comments.)

How about a competency exam as a mid-point in college. You could study on your own, take free online courses, whatever.

Passing your chosen school's competency exams would fulfill your basic requirements and taking a more comprehensive one would enable you to major in a subject.

You could choose to take classes at the school to achieve this competency or to do it on your own. But you'd still have the experience at some point of actually rubbing elbows with professionals in the field and your future colleagues. You'd still be able to choose universities by their faculty and specialties, but might be doing so once you've actually gotten some learning under your belt.

Anonymous said...

There's no way a "degree by test" will ever be seen as as good as a traditional degree. It'll be like the UPhoenix division of a stat Uni, without the same weight. Really it isn't the universities themselves who get to decide what their degrees are worth - it's the rest of us. Special wording will evolve on resumes to make it clear the applicant has a traditional degree rather than a "degree by test." You'll be able to get your resume past the first cut, but not the second. This will limit the market to mid-career professionals who screwed up when they were kids and just need the rubber stamp.

Anonymous said...

Excelsior College (formerly Regents College), Charter Oak State College, and Thomas Edison State College have been doing this sort of thing for years.

Grace said...

My armchair prediction is that many second-tier colleges will be closing, and online courses will still be part of the mix.

Here's the thing - many of these traditional colleges are delivering a lousy education at exorbitant prices. Isn't that what Academically Adrift was about? So it may not be considered such a loss for a switch to a certification system of competency exams.

Grace said...

In related news, New York’s SUNY system is pushing three-year bachelor degrees with the use of online courses and credit for experiential education

Catherine Johnson said...

Mark - good for you!

Except: licensing Huntington & Sylvan is a VERY BAD IDEA.

A true competency exam should be a competency exam: there should be no 'licensing' of teachers & test prep companies involved.

That's the way it always was with law.

You studied the law in whatever way you wished, then you took the test.

The profession didn't concern itself with how you got the knowledge.

Only with whether you did indeed possess the knowledge.

Catherine Johnson said...

At his home computer, huh? It sounds like there may soon be some really lucrative opportunities out there for good test-takers.

Uhhhh.....

Good point.

(In truth, what I have "always thought" wasn't that you could use the internet to score competency exams, but that you could use the internet to score HW.)

Catherine Johnson said...

Jen wrote: At his home computer, huh? It sounds like there may soon be some really lucrative opportunities out there for good test-takers.

Uhhhh.....

Good point.

In truth, what I have "always thought" wasn't that you could use the internet to score competency exams, but that you could use the internet to score HW.

That's the one aspect of Khan that I unreservedly LOVE: the automated scoring.

Catherine Johnson said...

When getting my masters, I had to take x number of "masters credits." These weren't tied to any specific class, but were essentially the payment for the degree.

right --- and this is the kind of thing that drives parents living in a depressed economy INSANE

HOWEVER, on the issue of kissing cushy tenure positions goodbye: there are no tenure positions.

Remember the chart in the other post?? Only 10% of faculty today are tenure track.

My guess is that adjuncts will take the hit.

Catherine Johnson said...

Mark wrote: The universities will *TRY*, but it only takes one state with a good public university system (and we have several states that meet that ...) and a governor and/or legislature that wants a reasonably priced degree.

As I said, I have NO business making predictions....but I really think Mark is right on this one-----

"Disruption" does occur in industries (granting that colleges & universities aren't precisely 'industries')---and this sure sounds like a No Turning Back kind of thing.

Catherine Johnson said...

Grace - thanks!

YUCK.

That's my worst fear, I would say: public colleges & universities continuing to charge what they're charging now but watering down the degree & moving everything (or as much as possible) online.

At the moment, that is the danger I see in terms of whether competency exams can or cannot get off the ground.

btw, I believe in competency exams, period --- not just as a way of saving money.

I hate credentialism.

Under a competency exam system, the vast majority of students would still want to "have the college experience" (or have two years of the college experience, as Jen suggests...)

I do think that test prep centers will become widespread BUT, on the other hand, you will also see (I think) "private practitioners" who teach their subject to groups of students and attract students by publishing their results.

Catherine Johnson said...

The big growth area for MOOCs is colleges REQUIRING their students to take MOOCs instead of classroom courses.

You have to take one of the Os out of the acronym.

What I see happening, and what I expect to see more of, is MOCs, not MOOCs.

Catherine Johnson said...

My armchair prediction is that many second-tier colleges will be closing, and online courses will still be part of the mix.

Grace - that's sure what I think.

froggiemama said...

Second tier colleges are big producers of the "professional" degrees - teachers, school psychologists, speech therapists, pharmacists, nurses, and so on. Those fields are highly regulated and very hands-on. I predict that won't change - it is very hard to train nurses via MOOCs.

The big losers are going to be those companies that charge an arm and a leg for "training" courses aimed at industry professionals. You know, the ones that charge thousands of dollars to send 10 developers to onsite training on SharePoint or Oracle. I also think this could hit the for-profit schools hard- why spend a fortune on an online degree from Joe Blow U that everyone will laugh at when you can do a MOOC instead.

Anonymous said...

"licensing Huntington & Sylvan is a VERY BAD IDEA"

I was talking about licensing these folks as testing centers. I don't think letting folks take test from home is going to work (certainly not long term ... the *obvious* cheating problem is too large to ignore). So we need somewhere "verifiably okay" to administer the tests. Not teach the material. And not design the tests. Just to give them.

It doesn't have to by Sylvan and Co. It might even be a local high school or JC.

In *MY* ideal world, the testing center would be shielded by/with/in a faraday cage so that you couldn't cheat via internet or cell access. There would be some sort of search to ensure that folks didn't bring in cell phones loaded up with answers. And I'd have each test filmed (webcams are cheap) so that if there was any question of cheating we could review the tape record later.

I don't think we are going to see anything like this because almost no one actually *CARES* if the kids cheat or not. But taking the test at home from your own computer isn't going to fly long term.

-Mark Roulo

Anonymous said...

"A true competency exam should be a competency exam: there should be no 'licensing' of teachers & test prep companies involved.

That's the way it always was with law."


Your use of the past tense here is important. Each state has different rules, but Wikipedia lists only four states where one can take the bar exam without attending law school :-(

-Mark Roulo

Glen said...

Like MagisterGreen, I've always said that this will be determined by employers. Except for technical majors, neither students nor employers are paying for what the university actually teaches. The ability to analyze a customer invoice from a marxist-feminist perspective isn't worth a fig, much less $200,000.

Employers pay big bucks for convincing evidence of a candidate's ability and willingness to jump through whatever hoops an organization tells him to jump through. Any degree from any university serves that purpose, though the more selective the university and the more demanding the degree, the better evidence it is, so the higher the premium. Even for many technical majors, it's not the technical skills but proof of the ability to get through hard technical classes that is being paid for. It's not the value of the education itself but the premium paid by employers that determines what students are willing to pay (and do) for the credential.

If employers can get their evidence from alternative credentials, they'll pay a premium for those, too, corresponding to the quality of the evidence. Students will then compare the premium paid for a credential to the cost of obtaining it, and make a purchase decision.

Off campus credentials don't have to be worth as much as traditional degrees to change everything. If off-campus credentials are only worth half the premium to employers but cost less than 5% of the traditional degree, so many people will opt for the off-campus option that colleges will lose their monopoly pricing power and their bubble will pop.

In the wake of such poppage, many universities might be forced to unbundle their courses and sell them a la carte or in smaller sets (ex: the first four semesters of physics), discovering in the process what the free market thinks each is worth on its own.

It's not likely to be good news for many Potemkin departments.

ChemProf said...

"Off campus credentials don't have to be worth as much as traditional degrees to change everything. If off-campus credentials are only worth half the premium to employers but cost less than 5% of the traditional degree, so many people will opt for the off-campus option that colleges will lose their monopoly pricing power and their bubble will pop. "

Exactly! I hate it when my colleagues make quality arguments without pricing arguments. If the alternative is cheap enough for the quality, then it will make more sense for many students.

And many of us in academia have not done ourselves any favors. As an example, in my own field, a lot of schools did away with Analytical Chemistry. It isn't very theoretical and it is not much fun to teach, but that also leads to chem majors who can't make their own solutions, prepare a calibration curve, or titrate accurately. And those are still the skills that a lot of employers are looking for!

Independent George said...

This excites me on a personal level.

About five years ago, I began a master's program in Software Engineering. 6 credits and over $5,000 later, I decided it wasn't worth the time or money, and began looking for lower-cost alternatives. I didn't find any.

I need some time to research it, but I might well be posting some first-hand reports here come the fall.

Catherine Johnson said...

I predict that won't change - it is very hard to train nurses via MOOCs.

No!

Not true!

My sister-in-law is a professor of nursing & author of a nursing textbook.

She teaches all of her courses online as far as I know.

I think it's exactly the opposite: professional degrees can be taught online because students are motivated, focused, etc. & will stick with it.

Also, there is a defined body of knowledge that really does have to be committed to memory, and while I don't think MOOCs are fabulous even for that purpose, they aren't ***obviously*** inferior to brick-and-mortar classrooms.

Catherine Johnson said...

Independent George - EXACTLY! I'm with you.

chemprof & Glen - In the wake of such poppage, many universities might be forced to unbundle their courses and sell them a la carte or in smaller sets (ex: the first four semesters of physics), discovering in the process what the free market thinks each is worth on its own.

Right --- if we really do see 'poppage,' I think all kinds of interesting things could happen.

For one thing, English might become English again.

People do want to take courses about English literature, and people do want to learn to write. Unbundling might reveal, fairly quickly, a market for survey courses and real writing courses.

I suspect that unbundling would quickly reveal a distinct lack of enthusiasm for 'interdisciplinary' courses, too.

Catherine Johnson said...

Colleges today are little more than job training centers and higher education in this country has, for decades, sold itself on the idea that you'll never get a "good" job without it.

MagisterGreen is completely right ---- I've forgotten the statistics now, but the liberal arts are a tiny part of what is going on inside colleges at this point.

Catherine Johnson said...

I was talking about licensing these folks as testing centers

Oh, sorry --- yes, I agree.

Of course, this point completely contradicts my post!

I guess now I've arrived at the point of projecting that Technology will Disrupt bricks-and-mortar instruction, leading to....bricks-and-mortar testing centers.

That's Really disruptive!

froggiemama said...

Yeah, some nursing courses can be taught online, but not the labs, and not the hands on stuff. I would not want to be in a hospital with a nurse who had been trained entirely via MOOC. Shudder.

I work a lot with a pharmacy program. Those students do massive amounts of hands on work, from traditional lab courses, through a simulated clinical practice,and then supervised training. A lot of the time is spent bringing students into the community of practice, which is really hard to do online.

The other thing is that these majors are highly regulated, and I don't see that changing anytime soon.

That being said, I have a friend who did one of those entirely online "teacher-training-in-a-year" programs, in Texas, aimed at people who already had bachelors degrees. She said it was a joke, and that she was utterly unprepared to enter a classroom when she got her first job.

Catherine Johnson said...

She said it was a joke, and that she was utterly unprepared to enter a classroom when she got her first job.

Gosh - that sounds like a conversation I had not too long ago ---- or was it something someone said here?

Definitely agree you can't do labs & clinical training online -----

I don't think you can do much of anything online -- although (n of 1) I did have a student who said he had learned a lot of grammar in a required online grammar course he took the summer before he enrolled in my class.

I believe him....he was way ahead of the other students AND had a kind of enthusiasm they didn't have, which seemed to come straight out of the knowledge he'd gained.

SO....never say never.

Hainish said...

"She said it was a joke, and that she was utterly unprepared to enter a classroom when she got her first job."

Um, I'd say the same about the time-and-money-sink alternative cert program I completed at a bricks-and-mortar university.

GoogleMaster said...

The tech industry has had certification-via-testing for about 20 years now. Think "Microsoft Certified (whatever)" or "A+ Certified (whatever)", and years ago we had "Certified Novell (whatever)". This led to a huge demand for "brain dumps", followed by a large population of "certified" developers/network engineers/whatever who had crammed on the brain dumps enough to pass the exams to get certified, but when you got them into the workplace, they were worse than useless.

I shudder to think of opening this can of worms even wider.

Anonymous said...

Yet the Cisco Certified Engineer (or whatever it is called) is held in high regard.

It is possible to create worthless certificates. But this isn't required.

Note that UW-M has its reputation riding on this, so they are incentivized to get it right ...

-Mark Roulo

Anonymous said...

The foreign testing institutions’ battle to combat Chinese students from cheating has been going on for years now.

In the quest to go abroad, cheaters have not hesitated to hire substitute test takers, also known as “sharpshooters”, to take tests such as the TOEFL and IELTS on their behalf. Some have even gone as far as to forge degrees, resumes, etc. Such tactics have forced foreign testing institutions to implement tedious measures and protocols. However, in spite of new measures, the “Substitute Test Taker” industry still thrives.

Foreigners Struggle to Combat Chinese Cheaters

Many Chinese college applicants are forging fake recommendation letters, personal essays, or otherwise gaming the admissions process, said Tom Melcher, chairman of Zinch China, a consulting company that advises American colleges and universities about mainland China.

To understand the depth of the problem, Zinch informally surveyed about 250 Chinese high-school students, and also spoke with some parents and college-recruiting agents.

Based on those interviews, Zinch estimates that 90 percent of Chinese applicants submit false recommendations; 70 percent have other people write their personal essays; 50 percent forge their high-school transcripts; 30 percent lie on financial-aid forms; and 10 percent list academic awards and other achievements they did not receive.

Falsified Applications Are Common Among Chinese Students Seeking to Go Abroad, Consultant Says [Chronicle of Higher Education]

When 21-year-old Zhang, an average student in college, got set for the Graduate Record Exam (GRE) in Beijing this year, she felt so unprepared that she skipped the exam entirely.

Forty days later, she flew to Vietnam and nailed a near-perfect score in the test, which is taken by candidates applying to graduate school in the United States.

Gaming the GRE test in China, with a little online help [Reuters]

Patricia J. Parker, assistant director of admissions at Iowa State, which enrolls more than 1,200 Chinese undergraduates, says students have proudly told her about memorizing thousands of vocabulary words, studying scripted responses to verbal questions and learning shortcuts that help them guess correct answers.

She has seen conditionally admitted students increase their Toefl scores by 30 or 40 points, out of a possible 120, after a summer break, despite no significant improvement in their ability to speak English. Her students, she says, don’t see this intense test-prepping as problematic: “They think the goal is to pass the test. They’re studying for the test, not studying English.”

The China Conundrum [NY Times]

momof4 said...

In the financial world, the CFA (certified financial analyst), for which one studies on one's own (CFA's own prep materials) and takes the test at a testing center (specific dates for each level)is very highly regarded and required for many jobs - either instead of or in addition to a MBA. There are three levels, and the failure rate is very high (IIRC over half for both level 1 and level 2), with few failures at level 2 ever obtaining certification. The same company also offers a CFP (cert. finan. planner) program.