Overcoming the legacy of prior education

Published March 20, 2013 Free online courses 25 Comments
Tags: , , , , , , ,

A real-time chronicle of a seasoned professor who is giving his second massively open online course.

We’re now into the third week of the course. The numbers are down on the first edition, almost certainly because the six months that have passed have seen the appearance of hundreds of other MOOCs students have to choose from. But the numbers are still huge. As of today:

Total registration: 27,014

Active students last week: 9,608

Total number of streaming views of lectures: 120,925

Total number of lecture downloads: 35,888

Number of unique videos watched: 87,155

Number of students submitting homework assignments: 5,552

Based on what we (my TA, Paul, and I) learned when I gave the course the first time last fall, I made some changes this time round. Paul and I discussed those changes in a video-recorded discussion we had with media host Angie Coiro just before edition 2 launched, that I referred to in my last blog.

Although the overall numbers are down by about 60%, the profile of the class activity is very similar. The most obvious one, the huge drop in numbers from the total number of enrollments to the number who are still active in week three, has been discussed ad infinitum, often being referred to as “a big problem with MOOCs.” As I observed in a recent blog in the Huffington Post, I don’t think there is a problem at all. The drop off is just a feature of what is a very new form of human experience. Old metrics are simply not appropriate, “retention rate” being one such. (Unless you pay attention to the base for the retention computation, in which case MOOC “retention” is not that different from retention in traditional college education.)

Some of the early research into MOOC participants that has been carried out by my colleagues at Stanford (including studies of my first MOOC) has already demonstrated what we suspected about why so many drop out of MOOCs: many people who register for a MOOC never have any intention of completing the course, or even getting beyond sampling one or two lectures and perhaps attempting one or two of the assignments. Some are motivated by pure curiosity into this new phenomenon, others just want to get a flavor of a particular discipline or topic, and doubtless others have different reasons.

For example, one reason some students enroll that I had not anticipated, reflects the fact that a MOOC offers a large number of eyeballs to be accessed. A very  small number of students enrolled for my course in order to advertise products. (At least, that was one reason they enrolled; they may also have wanted to learn how to think mathematically!) In the long run, this may or may not turn out to be a positive thing. Certainly, the products advertised in the discussion forums for my course (at least the ones I saw) were all education related and free. (Moreover, I also included my own course-related textbook in my short list of suggested – but not required – resources.)

Still, the very wide reach of MOOCs means we are likely to see new kinds of activities emerge, some of them purely commercial. The example I cite above, though right now a very isolated one, may be a sign of big things to come – which is why I mention it. There is, after all, a familiar pattern. The Internet, on which MOOCs live, began as a military and educational network, but now it is a major economic platform. And textbooks grew from being a valuable educational support to the present-day mega-profit industry that has effectively killed US K-12 education.

Talking of which (and this brings me to my main focus in this post), the death – or at least the dearth – of good K-12 mathematics education becomes clear when you look through the forum posts in a MOOC such as mine, which assumes only high school knowledge of mathematics.

To be sure, generalizing is always dangerous, particularly so when based on comments in an online forum, which always attracts people with something to complain about. (Case in point: See my Twitter feed when it comes to banks, United Airlines, and bigoted politicians.) But with that caveat in mind, some themes become clear.

First, many forum posters  seem to view education as something done to them, by other people who are in control. This is completely wrong, and is the opposite of what you will find in a good university (and a very small number of excellent K-12 schools).  ”To learn” is an active verb. The focus should be creating an environment where the student can learn, wants to learn, and can obtain the support required to do so. There is no other way, and anyone who claims to do anything more than help you to learn is trying to extract money from you.

Second, there is a common view of education as being primarily about getting grades on tests – generally by the most efficient means (which usually means by-passing real learning). In education, tests are metrics to help the student and the instructor gauge progress. That does not prevent tests being used to assess achievement and provide credentials, but that is something you do after an educational experience is completed. Their use within the learning process is different, and everyone involved in education – students, instructors, parents, bureaucrats, and politicians – needs to be aware of the distinction.

Even worse, is the belief that a test grade of less than 90% is an indication of failure, often compounded by the hopeless misconception that activities like mathematics depend mostly on innate talent, rather than the hours of effort that those of us in the business know is the key. (Check out Carol Dweck’s Mindset research or read Malcolm Gladwell’s book Blink. Better still, read both.)

This is compounded by the expectation that a grade of 90% is possible within just a few days of meeting something new. For example, here is one (slightly edited) forum post from a student in my class:

Right now I want to quit this class. I don’t understand ANY of it. Hell I don’t understand anything regarding to math except basic equations and those barely. When asked to give a theorem on why something (let’s say a right angle) is that way my answer always was “it is because it is”). So now I don’t know what to do. I got 14 out of 40 … 14, and the perfectionist in me is saying might as well give up … you gave it a shot … there is no way to catch up now. The person in me who wants to learn is saying to keep trying you never know what will happen. And the pessimist in me says it doesn’t matter – I dumb and will always be dumb and by continuing I am just showing how dumb I am.

In this case, I looked at other posts from this student and as far as I can tell (this is hard when done remotely over the Internet) she is smart and shows every indication she can do fine in mathematics. In which case, I take her comment as an indication of the total, dismal failure of the education system she has hitherto been subjected to. No first-line education system should ever produce a graduate who feels like that.

Certainly, in learning something new and challenging, getting over 30% in the first test, less than a week after meeting it for the first time, is good. In fact, if you are in a course where you get much more than that so quickly, you are clearly in the wrong course – unless you signed up in order to fine-tune something you had already learned. Learning is a long, hard process that involves repeated “failure”. And (to repeat a point I made earlier) anyone who says otherwise is trying to extract money from you.

Turning to the third theme emerging on the course forums, there is a perception that the most efficient way to learn is to break everything down into the smallest possible morsels. While an important component of learning – if the breaking down is done by, and not for, the student – it is just the first part of a two-part process. The second part, which is by far the most important, and is in fact where the actual learning takes place, is putting it back together into a coherent whole. Textbooks and YouTube videos can provide morselized edubits (I just made that word up), and they do so by the bucketload. What they cannot do, is deliver real learning.

Suitably designed, I see no reason why MOOCs cannot be made to provide good learning, at least up to sophomore college level in many, if not most, disciplines. But a key to doing that is to leverage the power, not of machines, but of people. For fairly well understood evolutionary reasons, human learning is a social activity. We learn best from and with other people. That is how we are built!

Part of the benefit from learning in a social context is that it can offer the learner not just feedback, but also the – at a fundamental level, more important – human support that people need to succeed in education. You can find both of these in a MOOC. Within a short time of the student above posting her feelings, another student responded with this:

Hi. Don’t be discouraged. This course will give you the opportunity to think in a different way. I took the course last year and struggled with most of it. I am taking the course again as I find the subject of mathematical thinking fascinating. My scores this time round are better than the last time which indicates that given enough time even the most mathematically challenged can improve! Only have one caveat for you. If you don’t enjoy the struggle in trying to comprehend and feel that it is not worth the effort then maybe this course is not for you.

With that comment we can see one huge benefit of MOOCs. (At least, all the time they are free.) You can take them as many times as you need or want.

The one essential ingredient in order to take advantage of the huge opportunity MOOCs offer, is knowing how to learn. That should be the main ability graduates of the K-12 system get from their education. Unfortunately, with the current US (and elsewhere) system built around “being taught” and “being tested,” only a few students emerge with that crucial ability, and the ones who do usually say it is in spite of their school education.

The problem, by the way, is not the teachers. Certainly, most of the ones I meet agree with me, and are very clear as to what the problem is: a system that simply does not give them the freedom and support that is necessary for them to really help students learn. (See Jo Boaler’s excellent, well researched book What’s Math Got To Do With It? for a distressing account of how the current, overly micro-regulated system fails our students in the case of mathematics.)

Okay, that’s enough ranting for one post. Let me finish with a couple of examples where MOOCs are already working well. One student in my MOOC posted the following comment:

I have taken this course on a whim to get myself back in gear to return to school in the fall. I always despised the math classes that I was forced to attend in high school and early college. I was frustrated with the endless formulas and cookie cutter style problem solving. If you can solve one you can solve them all so being forced to endlessly solve these equations and proofs over and over seemed to be a futile act of nonsense.

Heading into week three three of this class, my mind has been completely changed. I not only enjoy this more logic based math, but have, in the course of some personal reading and problem solving, discovered i have a knack for it. I have found the challenge of solving more and more difficult problems from a few books i have purchased much more gratifying and interesting than any other area of previous study.

I would like you know that I now plan to switch majors to mathematics. I would like to thank you and your team for an eye-opening experience.

Oh, all right, I admit that included more ranting about US K-12 education. But, heavens, it is bad, and it is likely to remain so all the time that real, knowledgable educators are not part of the conversation, with all the important decision being made by people whose primary interests are profits or political career advancement. (BTW, I have nothing against the profit motive. Heavens, I have two for profit companies of my own and am talking with colleagues about launching a third. But financial ROI is not the same as educational ROI – and again, anyone claiming otherwise, as one head of a major textbook publisher did not long ago, is motivated by the former. I do have something against many politicians, but then I am an American citizen, so after what we have experienced in the past four years, I would.*)

Here’s the other example, this one sent to me in an email, rather than posted on the course discussion forum.

I am enrolled in your course “Introduction to Mathematical Thinking.” It is incredible. You have alleviated my fears that my college professors will have the same attitude towards mathematics that my high school teachers do. Mathematics is beautiful and certainly emotional. I am surrounded at school by people who believe mathematics is systematic. Through all of the videos you have posted so far and your archived NPR clips, I am now confident that mathematics is the direction I want to pursue. I am excitedly awaiting next week’s lectures. 

With tears in my eyes and more gratitude than I know how to express,

It’s that kind of feedback that makes teaching one of the most rewarding professions in the world. It’s why people become teachers. If society would just get off teachers’ backs and let them get on with what they were trained to do, what they know how to do,  and what they want to do, we’d all be a lot better off. (Check out Finland.)

To be continued …

*ADDED LATER IN RESPONSE TO A QUERY FROM AN OVERSEAS READER: The problem is the complete refusal of the Republican Party to cooperate with a now twice-elected President of the US, in governing the country as they are all elected and paid from public funds to do, choosing instead to drive the country, and with it most of the world, to the brink of financial and thence  social disaster.

 

MOOCs are So Back to the Future

Published March 1, 2013 Free online courses 2 Comments

 A real-time chronicle of a seasoned professor who is about to give his second massively open online course.

With the second edition of my Stanford MOOC Introduction to Mathematical Thinking starting this weekend on Coursera, I have once again been wrestling with the question of the degree to which good, effective mathematics learning can be achieved at scale, over the Internet.

I describe some of my reflections in my latest post in my monthly Devlin’s Angle column for the MAA — a column aimed primarily at college mathematics faculty, which makes up most of the MAA’s membership.

When I started to plan the first iteration of the course last spring, my main goal was to be able to walk away alive in order to try again. I stayed as close as I could to the way I had taught such a course in a traditional classroom setting, since I knew how to make that work (in the traditional classroom). This time round, armed with what I learned from that first attempt, I am making a number of changes.

My course TA (last time and this) is Paul Franz, a doctoral student in Stanford’s Graduate School of Education, and the two of us went into the Stanford TV studio recently to talk about the new course with broadcaster Angie Coiro, currently host of the syndicated radio and television interview show In Deep. You can find the first clip (10 minutes) from that hour-long interview here.  I’ll release further clips in future posts to this blog.

One change I’ve made to the course is to stretch it from seven weeks (five weeks of lectures followed by two weeks of examination work) to ten weeks (eight plus two). As we observe in that video clip, that change was a direct result of the information we collected from giving the course the first time.

Students in a MOOC exhibit a very different — and far more varied — profile from the traditional university cohort. Not just in age and backgrounds, but also in their reasons for enrolling in a MOOC. For instance, many people enroll in a MOOC with no intention of completing the course. They simply want to get a sense of the topic or subject.

But there is another group that wants to complete the course,  and come in prepared to work very hard to do so. They want the course to be as close as possible to a regular university course — essentially the classroom course I have been giving off and on at a number of elite colleges and universities since the late 1970s, most recently at Stanford — not a watered down version. But as the course went on, a substantial number of them submitted forum posts and emailed me to say that the pressures of their professional lives occasionally made it impossible to keep up. With instructional videos being released three times a week, on Monday’s Wednesdays, and Fridays, if a business trip caused them to miss a couple of days, they were never able to recover, and eventually had to drop out.

So I have reorganized the course so it runs slightly longer, but with instructional videos coming out only twice a week (Mondays and Wednesdays). That still maintains the pressure that is a major component of my course (and primarily, I see it as a course, for reasons I have articulated in several earlier posts in this blog), but provides what I hope is sufficient flexibility for busy people to cope.

The adoption of a different schedule is almost certainly the most obvious change I have made. But that one is purely logistic. Far more significant, and to me (and my education graduate student TA) more interesting, are the pedagogic changes I have implemented.

As the first course progressed, I gradually came to realize that the underlying pedagogical model I had adopted enabled me to make much more extensive and aggressive use of a number of educational devices I had used only minimally the first time round, namely:

  1. machine-graded, multiple-choice pop quizzes
  2. machine-graded, multiple-choice (substantive) problem sets
  3. student evaluation/grading of work.

I have been strongly opposed to the first two (as are most of my colleagues, and for good reason) for my entire career in university education, and had never seen the need for the third (though I am familiar with the research that shows the beneficial effects on student learning of being asked to evaluate and grade the work of their peers). In a MOOC, where there are thousands of students, all three seem unavoidable. And so I used them all. But I did so as little as possible.

This next time round, all three play a much more prevalent role. And they do so because of that recognition that my underlying pedagogic model eliminated many of the objections and hestitations I had to those devices.

What is that pedagogic model? One-on-one teaching/learning, the kind of learning experience that in the traditional academy is reserved only for doctoral students. For inescapable personnel reasons — sheer numbers — it is not possible to provide one-on-one learning experiences for undergraduates or masters students at a traditional university.

But surely, isn’t it even more problematic in an online course with tens of thousands of students? Strange though it may seem, the answer is no.

The reason is that a MOOC is, in many ways, like radio or TV — and not just because MOOCs make use of video-recorded lectures. Of far more educational significance, though TV and radio are both referred to as “mass media,” they are in fact highly individual. The newsreader on radio or TV is not addressing a large audience; she or he is talking to millions of single individuals.
The secret to being good on the radio or TV is to forget the millions and think of just one (generic) person. After all, the listener or viewer is not in a room with millions of other people; in fact, if the broadcast is successful, that listener or viewer is cognitively in a room with just the presenter. The really successful radio and TV newsreaders and presenters are the ones who can do that really well. They create that sense that they are talking just to You. And the same is true for a MOOC.
When your voice, with or without your face, is in someone’s living room, especially if on a regular basis, there is a direct human connection that in important ways is far more intimate than is possible in a lecture hall filled with anything more than a handful of students.
Once you realize this feature of the MOOC medium, the one-on-one pedagogic model is obvious. I used it extensively in the first version of the course, going to considerable lengths to create a sense of the student sitting next to me at my desk as we worked through the material. (You can see a low resolution example here. The Coursera videos on their site are larger and of much higher resolution.)
Although the entire course was planned meticulously in advance — for such a complex system, with so many moving parts, it has to be — when I sat down to record an instructional session, everything was recorded as live, without notes. I simply plonked a piece of paper down on my desk, beneath an overhead camcorder, and began talking to an imaginary (single!) individual student sitting alongside me.
The only editing done to what was captured by the camcorder was to speed up some of the handwriting to match my voice. All my false starts and my inevitable writing and speaking errors, together with my moments of indecision, made it into the video that was released. For the fact is, the focus of the course, namely mathematical thinking, is an error-prone, messy, human activity,  that often proceeds at a pedestrian pace, punctuated with uncertainties, and that is exactly what I wanted to convey.
To make the result as realistic as possible, I made a point of not thinking in advance about the topic or problem I would discuss — and certainly never “rehearsed.” After all, none of those would have been possible if the student had simply knocked on my office door and said, “Professor Devlin, do you have a moment to explain something to me?”
True, this does not make for riveting, slick television. It’s not meant to. It’s WYSIWOSG teaching: “what you see is what our students get.”
In other words, MOOCs enable us to go back to the oldest, and to this day by far the most effective method of education the world has ever known: the one-on-one apprenticeship system. Once you realize that, many things become possible that can’t be effectively used in traditional undergraduate education.
That includes the use of pop quizzes and multiple-choice questions! For when you think about it for a moment, PhD advisers use both all the time. True, they don’t think of them in those terms, and they don’t implement them with machine grading, but they do use both techniques.
Of course, a MOOC still does not allow the apprentice to talk back to the instructor — though in reality most apprentice learners feel pretty timid next to the expert, and rarely do that. Moreover, the instructor is not able to view and comment on the learner’s work. This is where, in a MOOC, crowd sourcing (something not available in a traditional apprenticeship situation) must be brought to bear. Now we are into something new!
I’ll continue this thread in future posts.
Since most of the instructional materials in a MOOC have to be created and assembled before the course begins, once it gets underway you are pretty well locked into the approach you start with, and any changes have to wait until you give the course again. So this second version of the course is my first opportunity to implement the changes I would have liked to make last time, and to make adjustments based on looking at the course data after the first edition finished.
I’ll be blogging about those changes in real time as the course proceeds. Will they make things better, and if so how? You will know soon after I do.
To be continued …

How are MOOCs organized?

Published February 24, 2013 Free online courses 12 Comments
Tags: , ,

A real-time chronicle of a seasoned professor who is about to give his second massively open online course.

With exactly one week to go before the second edition of my MOOC Introduction to Mathematical Thinking goes live, my TA and I have been working feverishly to get everything ready — a task far more complex and time consuming than preparing for a traditional (physical) course. (If you have been following this blog since I launched it last summer, when I started to plan my first edition of the course, you likely have some idea of the complexities involved.)

MOOCs continue to be in the news. Just last week, NBC-tv used my course as an illustration in a news story (4 min 21 secs) they ran about the American Council on Education’s recommendation that some Coursera MOOCs be considered eligible to receive college credit.

But what exactly is a MOOC and how are they organized? The easiest way to find out is to simply sign up for one or more and take a look. They are all free (at least, all the ones everyone is talking about are free), and there is no requirement to do any more than hang around online and see what is going on. If you do that, you’ll find that they all exhibit some differences from one another, as well as many similarities. Moreover, almost everyone giving a MOOC approaches it as an experiment, so they often change from one edition to the next.

Taking my own MOOC as an illustration, when the course website opens to registered students next weekend (Saturday March 2), they will initially find themselves in a website populated with several pages of information about the course structure, together with a bit of background information relevant to the course content, but none of the lectures, assignments, quizzes, problem sets, or tutorials will be available. Those are released at specified times throughout the ten weeks the course will run, starting with Lecture 1 on March 4.

For a sample of a lecture, see this short clip (7min 16 sec) from Lecture 1 on YouTube. (But note that Coursera videos are much higher resolution than YouTube, so the YouTube video is hard to follow — it’s purely an illustration of the overall format of the lectures.)

One of the main informational pages the students will see describes the various components of the course. Here, verbatim, are the contents of that page.

Basic elements of the course

Consult the Daily timetable (see link on left) on a regular basis to see what is due at any one time.

1. Lectures – videos presented by the instructor.
2. In-lecture quizzes – simple multiple-choice questions that stop the lecture, designed to assist you in pacing and monitoring your progress.
3. Assignment sheets (one for each lecture) – downloadable PDF files to work through in your own time at your own pace, ideally in collaboration with other students. Not graded.
4. Problem sets (one a week for weeks 1 through 8) – in-depth problems like those on the assignment, but with a deadline for submitting your answers (in a multiple choice format). Machine graded.
5. Tutorial sessions – the instructor provides (video) comments and answers to some of the previous week’s assignment problems.
6. Reading assignments – downloadable PDFs files providing important background information.
7. Final exam – a downloadable PDF file that you will have one week to complete before participating in a peer review process. Required to be eligible for a grade of completion with distinction.

Lectures

Lecture videos are released at 10:00AM US-PDT on Wednesdays. (Weeks 1 and 2 are slightly different, with lectures released on Monday and Wednesday.) Each lecture comprises one or two videos, with each video of length 25 to 35 minutes if played straight through. Completing the embedded progress quizzes will extend the total duration of a video-play by a few minutes, and you will likely want to stop the playback several times for reflection, and sometimes you will want to repeat a section, perhaps more than once. So you can expect to spend between one and two hours going through each lecture, occasionally perhaps more.

The lecture videos are not carefully crafted, heavily edited productions. If you want a polished presentation of the course material, you can read the course textbook. My goal with the lectures is to provide as best I can the experience of sitting alongside me as we work through material together. And, guess what, I often make mistakes, and sometimes mis-speak. I want to dispel any misconception that mathematicians are people who generate perfect logical arguments all the time. We’re not. We just keep going until we get it right.

In-lecture quizzes (Ungraded)

Each lecture is broken up by short multiple-choice “progress quizzes”. The vast majority of these in-lecture quizzes are essentially punctuation, providing a means for you to check that you are sufficiently engaged with the material.

Slightly modified versions of the quizzes will also be released as standalones at the same time as the lecture goes live, so if you do not have a good broadband connection and have to download the lecture videos to watch offline, you can still take the quizzes. In which case, you should do so as close in time to viewing the lecture as possible, to ensure gaining maximim benefit from the quizzes in monitoring your progress. The standalone quizzes are grouped according to lecture.

Completion of all the quizzes is a requirement (along with watching all the lectures) for official completion of the course, but we do not record your quiz scores, so quiz performance does not directly affect your final grade. If you complete the quizzes while watching the lecture (the strongly preferred method, as it helps you monitor your progress in mastering the material), you do not need to complete the standalone versions.

BTW, you may notice that it is possible to speed up video replay up to a factor of double speed. This can be a useful device when watching a video a second or third time. Going beyond 1.50 speed, however, can sometimes lead to problems with the display of the quizzes (besides making me sound like a chipmunk (though some may find that an enhancement).

Course assignments (Self graded)

An assignment will be released at the end of each lecture, as a downloadable PDF file. The assignment is intended to guide understanding of what has been learned. Worked solutions to problems from the assignments will be demonstrated (video) or distributed (PDFs) in a tutorial session released the Monday following the lecture (so in Weeks 2 through 9). The tutorial sessions will be released at 10:00AM US-PDT.

Working on these assignment problems forms the heart of the learning process in this course. You are strongly urged to form or join a study group, discuss the assignment problems with others in the group, and share your work with them. You should also arrange to assess one another’s answers. A structured form of peer review will be used for the final exam, when you will be graded by, and grade the work of, other students, randomly (and blindly) assigned, so it will help to familiarize yourself beforehand with the process of examining the work of others and providing (constructive) feedback.

Problem Sets (Machine graded)

Each Wednesday (in weeks 1 thtough 8), following the lecture, a for-credit Problem Set will be posted, with submission due by 9:00AM US-PDT the following Monday. The scores on these problem sets will count toward the course grade. Though the Problem Set has a multiple-choice quiz format, these questions are not the kind you can answer on the spot (unlike most of the in-lecture quizzes). You will need to spend some time working on them before entering your answers.

Though you are strongly encouraged to work with others on understanding the lecture material and attempting the regular assignments, the intention is that you work alone on the Problem Sets, which are designed to give you and us feedback on how you are progressing.

Tutorial sessions

The tutorial sessions are more than mere presentations of solutions to the previous week’s assignments and problem session. They are really lectures based on problems that the student has already attempted. You can expect to expand your knowledge of the course material beyond the lectures. Not all questions on the assignments sheets and problem set will be considered in the tutorial session.

Final exam (Peer graded)

Though the lectures end after week 8 (apart from a tutorial on the final assignment), the final two weeks are intended to be highly active ones for any students seeking a grade of distinction, with considerable activity online in the various forums and discussion groups. This is when you are supposed to help one another make sense of everything.

At the start of week 9, an open-book exam will be released, to be completed by the end of the week. Completed exams will have to be uploaded as either images (or scanned PDFs) though students sufficiently familiar with TeX have an option of keyboard entry on the site. The exam will be graded during week 10 by a calibrated peer review system. The exam will be based on material covered in the entire course.

As with the weekly Problem Sets, the intention is that you work alone in completing the final exam.

NOTE: The process of peer reviewing the work of others (throughout the course, not just in the final exam) is intended to be a significant part of the learning experience and participating in the formal peer review procedure for the final exam is a requirement for getting a grade of distinction. In principle, it is during week 10 that stronger students will make cognitive breakthroughs. (Many of today’s professors really started to understand mathematics when, as graduate student TAs, they first helped others learn it!)

Course completion and final grade

There are two final course grades: “completion” and “completion with distinction”. Completion requires viewing all the lectures and completing all the (in-lecture) quizzes and the weekly problem sets. Distinction depends on the scores in the problem sets and the result of the final exam.

Pacing

The pacing of the lecture releases is designed to help you maintain a steady pace. At high school, you probably learned that success in mathematics comes from working quickly (and alone) and getting to the right answer as efficiently as possible. This course is about learning to think a certain way – the focus is on the process not the product. You will need time to understand and assimilate new ideas. Particularly if you were a whiz at high-school math, you will need to slow down, and to learn to think and reflect (and ideally discuss with others) before jumping in and doing. A steady pace involving some period of time each day is far better than an all-nighter just before a Problem Set is due.

Keeping track

Consult the Daily timetable on the website on a regular basis to see what is due.

SO NOW YOU KNOW!

Here we go again

Published February 20, 2013 Free online courses 17 Comments
Tags: , , , ,

A real-time chronicle of a seasoned professor who is about to give his second massively open online course.

The second offering of my MOOC Introduction to Mathematical Thinking begins on March 4 on Coursera. (The site actually opens on March 2, so students can familiarize themselves with its structure and start to make contact with other students before the first lecture.) So far, 13,000 students have registered. Last time I got 65,000, but back then there was the novelty factor. I’m expecting about 35,000 this time round.

For a quick overview of my current thoughts on MOOCs, see this 13 minute TV interview I did at Tallinn University of Technology in Estonia last November. (As the home of Skype, global-tech-hub Tallinn is particularly interested in MOOCs, of course.)

It’s been almost four months since my first foray into the chaotic new world of MOOCs came to an end, and ten weeks since I posted my last entry on this blog. I have decided that giving a MOOC falls into the same category as running a marathon (I’ve done maybe two dozen), completing the Death Ride (three), and – I am told – having a baby (I played a decidedly minor role in two). At the time you wonder why you are putting yourself through such stress, and that feeling continues for a while after the event is over. But then the strain of it all fades and you are left with feelings of pleasure, accomplishment, and satisfaction. And with that comes the desire to do it all again – better in the case of running, cycling, and MOOCing.

Coursera, we have a problem

It’s important to remember that genuinely massive MOOCs are a mere eighteen months old, and each one is very much a startup operation — as are the various platform providers such as Udacity, edX, Coursera, Venture Labs. and Class2Go (all except edX coming out of Global Startup Central, i.e., Stanford). One of the features of any startup operation is that there will be plenty of missteps along the way. Given the complexity of designing  and delivering a university course in real time to tens of thousands of students around the world, it’s amazing that to date there have been just two missteps. The first, when the instructor had to pull the plug on a MOOC on designing online courses (yes, a particularly poignant topic as it turned out) and then more recently when the instructor pulled out, leaving the course to be run by the support staff.

Notice that I did not refer to either as a “failure.” Anyone who views such outcomes as failures has clearly never tried to do anything new and challenging, where you have to make up some of the rules as you go on. We are less than two years into this whole MOOC thing, so it’s worth reminding ourselves what it took (VIDEO) the USA to put a man on the Moon and bring him back alive, and to go on and build the Space Shuttle. The pedagogic fundamental that we gain confidence from our successes but learn from our mistakes, is as true for MOOC platform builders and MOOC instructors as it is for MOOC students.

Fortunately, I survived my first test flight relatively unscathed. I may not be so lucky second time round. I’ve made some changes that are intended to make the course better, but won’t know if they do until the course is underway.

Perhaps the most obvious change is to stretch the course from seven weeks (five weeks of lectures followed by two weeks of final exam work) to ten (8 + 2). Many students in my first course told me that the “standard university pace” with which I covered the curriculum was simply too much for online students who were fitting the course around busy professional and family schedules. I doubt that change will have any negative consequences.

More uncertain in their outcome are the changes I have made to the peer review process, that forms a major component of the course for students who are taking it for a Certificate of Completion (particularly Completion with Distinction).

Give credit where credit is due? Maybe

Talking of which, the issue of credentialing continues to generate a lot of discussion. My course does not offer College Credit (and it is not clear any Stanford MOOC ever will), but just recently, the American Council on Education’s College Credit Recommendation Service (ACE CREDIT)  has evaluated and recommended college credit be given for five MOOCs currently offered (by other universities) on Coursera. (Starting this March, it will be possible to take an enhanced version of my MOOC given by Stanford Online High School, for which a credential is awarded, but that course, aimed at high flying high school juniors and seniors, has a restricted enrollment and carries a fee, so it is not a MOOC, rather a course with tutors and assessment, built around my MOOC.)

But I digress. As I observed on a number of occasions in this blog and my MAA blog Devlin’s Angle, I see group work and peer evaluation as the key to making quality mathematics education available in a MOOC. So students who took the first version of my course and are planning on enrolling again (and I know many are) will see some changes there. Not huge ones. Like NASA’s first fumbling steps into space, I think it is prudent to make small changes that have a good chance of being for the better. But I learned a lot from my first trip into MOOC-space, and I expect to learn more, and make further changes, on my second flight.

Finally, if you want to learn more about my reflections on my first MOOC and MOOCs in general, and have a two hour car drive during which you would find listening to a podcast about MOOCs marginally better than searching through an endless cycle of crackly Country and Western radio stations, download the two podcast files from Wild About Math, where host Sol Lederman grills me about MOOCs.

MOOC reflections – and Coursera’s Business Model

Published December 5, 2012 Free online courses 6 Comments
Tags: , , , , , ,

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

Two of my most recent reflections on MOOCs were in many ways reflections about mathematics education in general, so instead of burying them here, where only the MOOC-curious would see them, I submitted them to the Mathematical Association of America as articles in my monthly series “Devlin’s Angle”, and that’s where you will find them.

The first, titled MOOC lessons, focused on the kind of learning that can take place in a MOOC.

In the second, The Darwinization of Higher Education, I looked at the likely effect of MOOCs on the higher education landscape. I originally submitted a shorter, “less-personal-blog” version as a post to my blog in The Huffington Post (Education Section), but much to my surprise, after sitting on it for over a week, they rejected it. Either I am way ahead of the HuffPost’s education editors or else they think I’m off my rocker (maybe both). It probably reflects on the massive uncertainty about where MOOCs are going, that I think there is a 5% chance I am off my rocker on this issue. A week ago I would have put that figure at 10%, but just this morning (a scant week after I had submitted my thoughts to HuffPost, and a mere 24 hours after I had sent the piece instead to the MAA) I received an email Coursera sent out to all past and present students.

It began thus:

Career Services: Finding great job opportunities

Coursera has begun Career Services with the goal of helping Coursera students find great jobs! Meeting great companies just got easier. Just go to <Coursera web page URL> and fill in your profile to opt-in to the service. After you opt-in, we will share your resume and other information you provide with selected partner companies who will introduce themselves if there’s a match.
We’re excited to connect you with great companies and new opportunities! Complete your profile here!

Remember, you read it first here. (As they say.)

Is this really Coursera’s business model (in the sense of the business model)? I am in no position to know. I suspect they don’t yet know either, how (and maybe if) they, or any other MOOC platform, will eventually make sufficient revenue to sustain their activities. Calling it “Coursera’s business model” in my title indicates only that it is a business model that the company has now announced. From my second article listed above, you will gather I think it is a smart move on their part. On the other hand, I can think of at least half a dozen others ways to monetize MOOCs, and at least as many ways for others to build businesses around the MOOC phenomenon.

I know many of my academic colleagues feel uneasy when education is discussed as a for-profit enterprise, but it has never been anything else. Someone has to pay. Usually, it is the student or the student’s family, either directly or indirectly. The novel aspect of the Ivy League MOOCs that I hope those colleagues see as positive is that the one person who does not pay is the student — at least on entry, which means that MOOC education is entirely free.

While on the topic of MOOC upsides, I had lunch recently with three of my fellow pioneer MOOC-instructors, and one substantial student demographic we all noticed was moms with young children (in many cases single moms, without the means to afford child care while they study). Hard to fault that.

We are entering a very different world in terms of access to higher education.

To be continued …

Coming up for air (and spouting off)

Published November 26, 2012 Free online courses 11 Comments
Tags: , , , , , , , , , , , , , , , ,

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

Almost a month has passed since I last posted to this blog. Keeping my MOOC running took up so much time that, once it was over, I was faced with a huge backlog of other tasks to complete. Taking a good look at the mass of data from the course is just one of several post-MOOC activities that will have to wait until the New Year. So readers looking for statistics, analyses, and conclusions about my MOOC will, I am afraid, have to wait a little bit longer. Like most others giving these early MOOCs, we are doing so on the top of our existing duties; the time involved has yet to be figured into university workloads.

One issue that came up recently was when I put on my “NPR Math Guy” hat and talked with Weekend Edition host Scott Simon about my MOOC experience.

In the interview, I remarked that MOOCs owed more to Facebook than to YouTube. This observation has been questioned by some people, who believe Kahn Academy’s use of YouTube was the major inspiration. In making this comment, they are echoing the statement made by former Stanford Computer Science professor Sebastian Thrun when he announced the formation of Udacity.

In fact, I made my comment to Scott with my own MOOC (and many like it) in mind. Though I have noted in earlier posts to this blog how I studied Sal Khan’s approach in designing my own, having now completed my first MOOC, I am now even more convinced than previously that the eventual (we hope) success of MOOCs will be a consequence of Facebook (or social media in general) rather than of Internet video streaming.

The reason why I felt sure this would be the case is that (in most disciplines) the key to real learning has always been bi-directional human-human interaction (even better in some cases, multi-directional, multi-person interaction), not unidirectional instruction.

What got the entire discussion about MOOCs off in the wrong direction – and with it the public perception of what they are – is the circumstance of their birth, or more accurately, of their hugely accelerated growth when a couple of American Ivy League universities (one of them mine) got in on the act.

But it’s important to note that the first major-league MOOCs all came out of Stanford’s Computer Science Department, as did the two spinoff MOOC platforms, Udacity and Coursera. When MIT teamed up with Harvard to launch their edX platform a few months later, it too came from their Computer Science Department.

And there’s the rub. Computer Science is an atypical case when it comes to online learning. Although many aspects of computer science involve qualitative judgments and conceptual reasoning, the core parts of the subject are highly procedural, and lend themselves to instruction-based learning and to machine evaluation and grading. (“Is that piece of code correct?” Let the computer run it and see if it performs as intended.)

Instructional courses that teach students how to carry out various procedures, which can be assessed to a large degree by automatic grading (often multiple choice questions) are the low hanging fruit for online education. But what about the Humanities, the Arts, and much of Science, where instruction is only a small part of the learning process, and a decidedly unimportant part at that, and where machine assessment of student work is at best a goal in the far distant future, if indeed it is achievable at all?

In the case of my MOOC, “Introduction to Mathematical Thinking,” the focus was the creative/analytic mathematical thinking process and the notion of proof. But you can’t learn how to think a certain way or how prove something by being told or shown how to do it any more than you can learn how to ride a bike by being told or shown. You have to try for yourself, and keep trying, and falling, until it finally clicks. Moreover, apart from some very special, and atypical, simple cases, neither thinking nor proofs can be machine graded. Proofs are more like essays than calculations. Indeed, one of the things I told my students in my MOOC was that a good proof is a story, that explains why something is the case.

For the vast majority of students, discussion with (and getting feedback from) professors, TAs, and other students struggling to acquire problem solving ability and master abstract concepts and proofs, is an essential part of learning. For those purposes, the online version does not find its inspiration in Khan Academy as it did for Thrun, but in Facebook, which showed how social interaction could live on the Internet.

When the online version of Thrun’s Stanford AI class attracted 160,000 students, he did not start a potential revolution in global higher education, but two revolutions, only the first of which he was directly involved in. The first one is relatively easy to recognize and understand, especially for Americans, who for the most part have never experienced anything other than instruction-based education.

For courses where the goal is for the student to achieve mastery of a set of procedures (which is true of many courses in computer science and in mathematics), MOOCs almost certainly will change the face of higher education. Existing institutions that provide little more than basic, how-to instruction have a great deal to fear from MOOCs. They will have to adapt (and there is a clear way to do so) or go out of business.

If I want to learn about AI, I would prefer to do so from an expert such as Sebastian Thrun. (In fact, when I have time, I plan on taking his Udacity course on the subject!) So too will most students. Why pay money to attend a local college and be taught by a (hopefully competent) instructor of less stature when you can learn from Thrun for free?

True, Computer Science courses are not just about mastery of procedures. There is a lot to be learned from the emphases and nuances provided by a true expert, and that’s why, finances aside, I would choose Thrun’s course. But at the end of the day, it’s the procedural mastery that is the main goal. And that’s why that first collection of Computer Science MOOCs has created the popular public image of the MOOC student as someone watching canned instructional videos (generally of short duration and broken up by quizzes), typing in answers to questions to be evaluated by the system.

But this kind of course occupies the space in the overall educational landscape that McDonalds does in the restaurant business. (As someone who makes regular use of fast food restaurants, this is most emphatically not intended as a denigratory observation. But seeing utility and value in fast food does not mean I confuse a Big Mac with quality nutrition.)

Things are very, very different in the Humanities, Arts, and most of Science (and some parts of Computer Science), including all of mathematics beyond basic skills mastery – something that many people erroneously think is an essential prerequisite for learning how to do math, all evidence from people who really do learn how to do math to the contrary.

[Ask the expert. We don’t master the basic skills; we don’t need them because, early on in our mathematic learning, we acquired one – yes, just one – fundamental ability: mathematical thinking. That’s why the one or two kids in the class who seem to find math easy seem so different. In general, they don’t find math easy, but they are doing something very different from everyone else. Not because they are born with a “math gene”. Rather, instead of wasting their time mastering basic skills, they spent that time learning how to think a certain way. It’s just a matter of how you devote your learning time. It doesn’t help matters that some people managed to become qualified math teachers and professors seemingly without figuring out that far more efficient path, and hence add their own voice to those who keep calling for “more emphasis on basic skills” as being an essential prerequisite to mathematical power.]

But I digress. To get back to my point, while the popular image of a MOOC centers on lecture-videos and multiple-choice quizzes, what Humanities, Arts, and Science MOOCs (including mine) are about is community building and social interaction. For the instructor (and the very word “instructor” is hopelessly off target in this context), the goal in such a course is to create a learning community.  To create an online experience in which thousands of self-motivated individuals from around the world can come together for a predetermined period of intense, human–human interaction, focused on a clearly stated common goal.

We know that this can be done at scale, without the requirement that the participants are physically co-located or even that they know one another. NASA used this approach to put a man on the moon. MMOs (massively multiplayer online games – from which acronym MOOCs got their name) showed that the system works when the shared goal is success in a fantasy game world.

Whether the same approach works for higher education remains an open question. And, for those of us in higher education, what a question! A question that, in my case at least, has proved irresistible.

This, then, is the second MOOC revolution. The social MOOC. It’s outcome is far less evident than the first.

The evidence I have gathered from my first attempt at one of these second kinds of MOOC is encouraging, or at least, I find it so. But there is a long way to go to make my course work in a fashion that even begins to approach what can be achieved in a traditional classroom.

I’ll pursue these thoughts in future posts to this blog — and in future versions of my Mathematical Thinking MOOC, of which I hope to offer two variants in 2013.

Meanwhile, let me direct you to a recent article that speaks to some of the issues I raised above. It is from my legendary colleague in Stanford’s Graduate School of Education, Larry Cuban, where he expresses his skepticism that MOOCs will prove to be an acceptable replacement for much of higher education.

To be continued …

Peer grading: inventing the light bulb

Published October 31, 2012 Free online courses 29 Comments
Tags: , , , , , , , , ,

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

With the deadline for submitting the final exam in my MOOC having now passed, the students are engaging in the Peer Evaluation process. I know of just two cases where this has been tried in a genuine MOOC (where the M means what it says), one in Computer Science, the other in Humanities, and both encountered enormous difficulties, and as a result a lot of student frustration. My case was no different.

Anticipating problems, I had given the class a much simplified version of the process – with no grade points at stake – at the end of Week 4, so they could familiarize themselves with the process and the platform mechanics before they had to do it for real. That might have helped, but the real difficulties only emerged when 1,520 exam scripts started to make their way through the system.

By then the instructional part of the course was over. The class had seen and worked through all the material in the curriculum, and had completed five machine-graded problem sets. Consequently, there were enough data in the system to award certificates fairly if we had to abandon the peer evaluation process as a grading device, as happened for that humanities MOOC I mentioned, where the professor decided on the fly to make that part of the exam optional. So I was able to sleep at night. But only just.

With over 1,000 of the students now engaged in the peer review process, and three days left to the deadline for completing grading, I am inclined to see the whole thing through to the (bitter) end. We need the data that this first trial will produce so we can figure out how to make it work better next time.

Long before the course launched, I felt sure that there were two things we would need to accomplish, and accomplish well, in order to make a (conceptual, proof-oriented) advanced math MOOC work: the establishment (and data gathering from) small study groups in which students could help one another, and the provision of a crowd-sourced evaluation and grading system.

When I put my course together, the Coursera platform supported neither. They were working on a calibrated peer review module, but implementing the group interaction side was still in the future. (The user-base growth of Coursera has been so phenomenal, it’s a wonder they can keep the system running at all!)

Thus, when my course launched, there was no grouping system, nor indeed any social media functionality other than the common discussion forums. So the students had to form their own groups using whatever media they could: Facebook, Skype, Google Groups, Google Docs, or even the local pub, bar, or coffee shop for co-located groups. Those probably worked out fine, but since they were outside our platform, we had no way to monitor the activity – an essential functionality if we are to turn this initial, experimental phase of MOOCs  into something robust and useful in the long term.

Coursera had built a beta-release, peer evaluation system for a course on Human Computer Interaction, given by a Stanford colleague of mine. But his needs were different from mine, so the platform module needed more work – more work than there was really time for! In my last post, I described some of the things I had to cope with to get my exam up and running. (To be honest, I like the atmosphere of working in startup mode, but even in Silicon Valley there are still only 24 hours in a day.)

It’s important to remember that the first wave of MOOCs in the current, explosive, growth period all came out of computer science departments, first at Stanford, then at MIT. But CS is an atypical case when it comes to online learning. Although many aspects of computer science involve qualitative judgments and conceptual reasoning, the core parts of the subject are highly procedural, and lend themselves to instruction-based learning and to machine evaluation and grading. (“Is that piece of code correct?” Just see if it runs as intended.)

The core notion in university level mathematics, however, is the proof. But you can’t learn how to prove something by being told or shown how to do it any more than you can learn how to ride a bike by being told or shown. You have to try for yourself, and keep trying, and falling, until it finally clicks. Moreover, apart from some very special, and atypical, simple cases, proofs cannot be machine graded. In that regard, they are more like essays than calculations. Indeed, one of the things I told my students was that a good proof is a story, that explains why something is the case.

Feedback from others struggling to master abstract concepts and proofs can help enormously. Study groups can provide that, along with the psychological stimulus of knowing that others are having just as much difficulty as you are. Since companies like Facebook have shown us how to build platforms that support the creation of groups, that part can be provided online. And when Coursera is able to devote resources to doing it, I know it will work just fine. (If they want to, they can simply hire some engineers from Facebook, which is little more than a mile away. I gather that, like Google before it, the fun period there has long since passed and fully vested employees are looking to move.)

The other issue, that of evaluation and grading, is more tricky. The traditional solution is for the professor to evaluate and grade the class, perhaps assisted by one or more TAs (Teaching Assistants). But for classes that number in the tens of thousands, that is clearly out of the question. Though it’s tempting to dream about building a Wikipedia-like community of dedicated, math-PhD-bearing volunteers, who will participate in a mathematical MOOC whenever it is offered – indeed I do dream about it – it would take time to build up such a community, and what’s more, it’s hard to see there being enough qualified volunteers to handle the many different math MOOCs that will soon be offered by different instructors. (In contrast, there is just one Wikipedia, of course.)

That leaves just one solution: peer grading, where all the students in the class, or at least a significant portion thereof, are given the task of grading the work of their peers. In other words, we have to make this work. And to do that, we have to take the first step. I just did.

Knowing just how many unknowns we were dealing with, my expectations were not high, and I tried to prepare the students for what could well turn out to be chaos. (It did.) The website description of the exam grading system was littered with my cautions and references to being “live beta”. On October 15, when the test run without the grading part was about to launch, I posted yet one more cautionary note on the main course announcements page:

… using the Calibrated Peer Review System for a course like this is, I believe, new. (It’s certainly new to me and my assistants!) So this is all very much experimental. Please approach it in that spirit!

Even so, many of the students were taken aback by just how clunky and buggy the thing was, and the forums sprung alive with exasperated flames. I took solace in the recent release of Apple Maps on the iPhone, which showed that even with the resources and expert personnel available to one of the world’s wealthiest companies, product launches can go badly wrong – and we were just one guy and two part-time, volunteer student assistants, working on a platform being built under us by a small startup company sustained on free Coke and stock options. (I’m guessing the part about the Coke and the options, but that is the prevalent Silicon Valley model.)

At which point, one of those oh-so-timely events occurred that are often described as “Acts of God.” Just when I worried that I was about to witness, and be responsible for starting, the first global, massive open online riot (MOOR) in a math class, Hurricane Sandy struck the Eastern Seaboard, reminding everyone that a clunky system for grading math exams is not the worst thing in the world. Calm, reasoned, steadying, constructive posts started to appear on the forum.  I was getting my feedback after all. The world was a good place once again.

Failure (meaning things don’t go smoothly, or maybe don’t work at all) doesn’t bother me. If it did, I’d never have become a mathematician, a profession in which the failure rate in first attempts to solve a problem is somewhere north of 95%. The important thing is to get enough data to increase the chances of getting it right – or far more likely, just getting it better – the second time round. Give me enough feedback, and I count that “failure” as a success.

As Edison is said to have replied to a young reporter about his many failed attempts to construct a light bulb, “Why would I ever give up? I now know definitively over 9,000 ways that an electric light bulb will not work. Success is almost in my grasp.” (Edison supposedly failed a further 1,000 times before he got it right. Please don’t tell my students that. We are just at failure 1.)

If there were one piece of advice I’d give to anyone about to give their first MOOC, it’s this: remember Edison.

To be continued …

I'm Dr. Keith Devlin, a mathematician at Stanford University. In fall 2012, I gave my first free, open, online math course and this spring I am giving my second. This blog chronicles my experiences as they happen.

Twitter Updates

New Book 2012

New book 2011

New e-book 2011

New book 2011

May 2013
M T W T F S S
« Mar    
 12345
6789101112
13141516171819
20212223242526
2728293031  
Follow

Get every new post delivered to your Inbox.

Join 442 other followers

%d bloggers like this: