Archive for October, 2012

Peer grading: inventing the light bulb

Published October 31, 2012 Free online courses 29 Comments
Tags: , , , , , , , , ,

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

With the deadline for submitting the final exam in my MOOC having now passed, the students are engaging in the Peer Evaluation process. I know of just two cases where this has been tried in a genuine MOOC (where the M means what it says), one in Computer Science, the other in Humanities, and both encountered enormous difficulties, and as a result a lot of student frustration. My case was no different.

Anticipating problems, I had given the class a much simplified version of the process – with no grade points at stake – at the end of Week 4, so they could familiarize themselves with the process and the platform mechanics before they had to do it for real. That might have helped, but the real difficulties only emerged when 1,520 exam scripts started to make their way through the system.

By then the instructional part of the course was over. The class had seen and worked through all the material in the curriculum, and had completed five machine-graded problem sets. Consequently, there were enough data in the system to award certificates fairly if we had to abandon the peer evaluation process as a grading device, as happened for that humanities MOOC I mentioned, where the professor decided on the fly to make that part of the exam optional. So I was able to sleep at night. But only just.

With over 1,000 of the students now engaged in the peer review process, and three days left to the deadline for completing grading, I am inclined to see the whole thing through to the (bitter) end. We need the data that this first trial will produce so we can figure out how to make it work better next time.

Long before the course launched, I felt sure that there were two things we would need to accomplish, and accomplish well, in order to make a (conceptual, proof-oriented) advanced math MOOC work: the establishment (and data gathering from) small study groups in which students could help one another, and the provision of a crowd-sourced evaluation and grading system.

When I put my course together, the Coursera platform supported neither. They were working on a calibrated peer review module, but implementing the group interaction side was still in the future. (The user-base growth of Coursera has been so phenomenal, it’s a wonder they can keep the system running at all!)

Thus, when my course launched, there was no grouping system, nor indeed any social media functionality other than the common discussion forums. So the students had to form their own groups using whatever media they could: Facebook, Skype, Google Groups, Google Docs, or even the local pub, bar, or coffee shop for co-located groups. Those probably worked out fine, but since they were outside our platform, we had no way to monitor the activity – an essential functionality if we are to turn this initial, experimental phase of MOOCs  into something robust and useful in the long term.

Coursera had built a beta-release, peer evaluation system for a course on Human Computer Interaction, given by a Stanford colleague of mine. But his needs were different from mine, so the platform module needed more work – more work than there was really time for! In my last post, I described some of the things I had to cope with to get my exam up and running. (To be honest, I like the atmosphere of working in startup mode, but even in Silicon Valley there are still only 24 hours in a day.)

It’s important to remember that the first wave of MOOCs in the current, explosive, growth period all came out of computer science departments, first at Stanford, then at MIT. But CS is an atypical case when it comes to online learning. Although many aspects of computer science involve qualitative judgments and conceptual reasoning, the core parts of the subject are highly procedural, and lend themselves to instruction-based learning and to machine evaluation and grading. (“Is that piece of code correct?” Just see if it runs as intended.)

The core notion in university level mathematics, however, is the proof. But you can’t learn how to prove something by being told or shown how to do it any more than you can learn how to ride a bike by being told or shown. You have to try for yourself, and keep trying, and falling, until it finally clicks. Moreover, apart from some very special, and atypical, simple cases, proofs cannot be machine graded. In that regard, they are more like essays than calculations. Indeed, one of the things I told my students was that a good proof is a story, that explains why something is the case.

Feedback from others struggling to master abstract concepts and proofs can help enormously. Study groups can provide that, along with the psychological stimulus of knowing that others are having just as much difficulty as you are. Since companies like Facebook have shown us how to build platforms that support the creation of groups, that part can be provided online. And when Coursera is able to devote resources to doing it, I know it will work just fine. (If they want to, they can simply hire some engineers from Facebook, which is little more than a mile away. I gather that, like Google before it, the fun period there has long since passed and fully vested employees are looking to move.)

The other issue, that of evaluation and grading, is more tricky. The traditional solution is for the professor to evaluate and grade the class, perhaps assisted by one or more TAs (Teaching Assistants). But for classes that number in the tens of thousands, that is clearly out of the question. Though it’s tempting to dream about building a Wikipedia-like community of dedicated, math-PhD-bearing volunteers, who will participate in a mathematical MOOC whenever it is offered – indeed I do dream about it – it would take time to build up such a community, and what’s more, it’s hard to see there being enough qualified volunteers to handle the many different math MOOCs that will soon be offered by different instructors. (In contrast, there is just one Wikipedia, of course.)

That leaves just one solution: peer grading, where all the students in the class, or at least a significant portion thereof, are given the task of grading the work of their peers. In other words, we have to make this work. And to do that, we have to take the first step. I just did.

Knowing just how many unknowns we were dealing with, my expectations were not high, and I tried to prepare the students for what could well turn out to be chaos. (It did.) The website description of the exam grading system was littered with my cautions and references to being “live beta”. On October 15, when the test run without the grading part was about to launch, I posted yet one more cautionary note on the main course announcements page:

… using the Calibrated Peer Review System for a course like this is, I believe, new. (It’s certainly new to me and my assistants!) So this is all very much experimental. Please approach it in that spirit!

Even so, many of the students were taken aback by just how clunky and buggy the thing was, and the forums sprung alive with exasperated flames. I took solace in the recent release of Apple Maps on the iPhone, which showed that even with the resources and expert personnel available to one of the world’s wealthiest companies, product launches can go badly wrong – and we were just one guy and two part-time, volunteer student assistants, working on a platform being built under us by a small startup company sustained on free Coke and stock options. (I’m guessing the part about the Coke and the options, but that is the prevalent Silicon Valley model.)

At which point, one of those oh-so-timely events occurred that are often described as “Acts of God.” Just when I worried that I was about to witness, and be responsible for starting, the first global, massive open online riot (MOOR) in a math class, Hurricane Sandy struck the Eastern Seaboard, reminding everyone that a clunky system for grading math exams is not the worst thing in the world. Calm, reasoned, steadying, constructive posts started to appear on the forum.  I was getting my feedback after all. The world was a good place once again.

Failure (meaning things don’t go smoothly, or maybe don’t work at all) doesn’t bother me. If it did, I’d never have become a mathematician, a profession in which the failure rate in first attempts to solve a problem is somewhere north of 95%. The important thing is to get enough data to increase the chances of getting it right – or far more likely, just getting it better – the second time round. Give me enough feedback, and I count that “failure” as a success.

As Edison is said to have replied to a young reporter about his many failed attempts to construct a light bulb, “Why would I ever give up? I now know definitively over 9,000 ways that an electric light bulb will not work. Success is almost in my grasp.” (Edison supposedly failed a further 1,000 times before he got it right. Please don’t tell my students that. We are just at failure 1.)

If there were one piece of advice I’d give to anyone about to give their first MOOC, it’s this: remember Edison.

To be continued …

Advertisement

Answering the 64,000-Students Questions

Published October 26, 2012 Free online courses 14 Comments
Tags: , , , , , , , ,

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

With the “instructional” part of the course finished and the remaining students working on the Final Exam (it will be peer graded next week), at last I can sit back and take a short breather. The next step will be to debrief and reflect with my two course assistants (both PhD students in the Stanford Graduate School of Education) and decide where to ride the MOOC beast next.

For sure I’ll offer another version of this course next year, with changes based on the huge amounts of data you get with a global online class of 64,000 students. Despite the enormous effort in designing, preparing, and running such a massive enterprise, there are three very good reasons to pursue this.

First, and this I believe is one of the main reasons why Stanford is supporting the development of MOOCs (I am not part of the central, policy-making administration), designing, running, and analyzing the learning outcomes of MOOCs is a tremendous research opportunity that will almost certainly result in new understandings of how people learn, and as a result very likely will enable the university to improve the learning experience of our regular on-campus students. After just five weeks, my two graduate assistants have enough data to write several dissertations, in addition to the one they need to get their doctorates.

Second, there is a huge, overall, feel-good factor for those of us involved, knowing that we can help to provide life-changing opportunities for people around the world who would otherwise have no access to quality higher education. Is what they get as good as being at Stanford? I very much doubt it, though the scientist in me says we should keep an open mind into the eventual outcomes of what is at present a very novel phenomenon. But if you compare a Stanford MOOC with the alternative of nothing at all, then already you have an excellent reason to continue.

Third, and this is something that anyone in education will acknowledge makes up for our earning a much lower salary than our (often less formally qualified) friends in the business and financial worlds, there is the pleasure of hearing first-hand from some of our more satisfied customers. The following is one of many appreciative emails and forum posts I have received as my course came to and end:

Mr. Devlin and all members of the Introduction To Mathematical Thinking team, I just wanted to say Thank You for everything that you have done to share your knowledge and giving your time and great effort to help others learn. I imagine that this is not an easy project to lead and sustain on a continuous basis. However, you have done a wonderful job in relaying your message. Through your efforts, you have helped many people in the process; especially me. Until this class, I hated math. I hated the idea of learning math or thinking in mathematically analogous methods that are applicable to real world situations. I just didn’t get it. I’m still a little confused about why I am able to comprehend your lessons as effectively as I am (which is saying a lot considering how much I hated math) when I have not been able to do so in the past. Now, I find myself looking forward to your classes everyday! I look forward to using what I have learned from the last video lectures or assignments and using those lessons in situations I did not think possible. And now, I love math! Your instruction has helped me to think more logically and to draw more concise conclusions with issues that I am trying to handle. This is indeed a skill. This is also a skill that you can build upon throughout your lifetime if one chooses to do so. Though I may not be at the level of learning that I should be at, I have learned more in the past three weeks than I have learned throughout my life; and I will continue to learn. I am very serious about this statement. So, thank you All. Thank you, Mr. Devlin. Great Job and Cheers!

Nice!

To be sure, there were trolls on the course discussion forum, for whom nothing we did was right. But one of the benefits of having tens of thousand of students is that within at most an hour of a flame post appearing, tens of others jumped on the offending individual, and within a short while all that was left was a “This comment has been deleted” notice. As the course wore on, the trolls simply dropped away.

Though there was the one individual who, in week four, posted a comment that he hated my teaching style and was learning nothing. Given that this was a free course that no one was under any compulsion to take, and for which no official credential was awarded, one wonders why this person stuck around for so long!

That example provided no more than an amusing anecdote to tell when I start to give talks on “What’s it like to teach 64,000 students?” (Invitations are already coming in.) But there is a somewhat closely related issue that I find far more significant.

Like almost all current MOOCs, there was no real credentialing in my course, so the focus was entirely on learning for its own sake. (As a lifelong math professor, used to teaching classes where many of the students were there because they needed to fulfill a mathematics requirement, having a class of students who were there purely voluntarily added appeal to my giving a MOOC.) To be sure, there were in-lecture quizzes, machine-graded assignments, and a peer evaluated final exam, but the only people who had access to any student’s results were myself, my two course assistants, and the student. Moreover, there was no official certification to back up a good result (the course offered two levels, Completion and Completion with Distinction), and turn it into a form of credential.

Yet many students had an ongoing obsession with their grades, and indeed pleaded with me from time to time to re-grade their work. (Clearly not possible in a 64,000 student MOOC. Besides, I never saw their work. How could I?) As a competitive person myself, I can appreciate the desire to do well. But with literally nothing at stake, I was at first surprised by the degree to which it bothered some of them. When I figured out what was probably going on, I found something that bothered me.

Unlike most MOOCs, mine, being at first-year university level, can be taken by high school students. Indeed, since my primary target audience comprised students entering or about to enter university to study mathematics or a math-related subject, I expected to get high school seniors, and designed my course as much as possible to accommodate them.

I’m guessing that the majority of students who were obsessed with grades were still at high school – indeed, most likely a US high school. That grade obsession I observed is, I suspect, simply a learned behavior that reflects the way our K-12 system turns the learning of a fascinating subject – one of humankind’s most amazing, creative, intellectual achievements – into a seemingly endless sequence of bite-sized pieces that are fed to the student in a mandated hamster-wheel.

No wonder they could not relax and enjoy learning for its own sake. Any natural curiosity and desire to learn – something all humans are born with – had been driven out of them by the very institution that is supposed to encourage and develop that trait. In its place was mere grade hunting.

Do I know this for a fact? No. That’s why I used those hedging words “guess” and “suspect”. But something has to explain that grade obsession in my course, and it certainly brought to mind Paul Lockhart’s wonderful essay A Mathematician’s Lament, which I had the privilege to bring to a wider audience some years ago.

But now I digress. Time to wrap up and check the dashboard on the course website see how many students have submitted the Final Exam so far.

Though this post has dropped the title “MOOC Planning”, I am going to keep posting here, as the project goes forward. Stay tuned.

To be continued …

It’s About Time (in Part): MOOC Planning – Part 10

Published October 22, 2012 Free online courses 6 Comments
Tags: , , , , , , , , , ,

 A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, lectures have ended and the course has now switched gears. For those still left in the course (17% of the final enrollment total of 64,045), the next two weeks are focused on trying to make sense of everything they have learned, and working on the final exam — which in the case of my course involves peer evaluation.

Calibrated Peer Review is not new. A study of its use in the high school system by Sadler and Good, published in 2006, has become compulsory reading for those of us planning and giving MOOCs that cover material that cannot be machine graded. [If you want to see how I am using it, just enroll in the class and read the description of the “Peer Review system”. There is no obligation to do anything more than browse around the site! No one will know you are not simply a dog that can use a computer.]

As I was working on my course, Coursera was still frantically building out their platform to support peer evaluation. There was a lot of just-in-time construction. It’s been a long time since I’ve had to go behind a user-friendly interface and dig into the underlying code to do something on a computer, and the programming languages have all changed since I last did that.

One thing I had to learn was one of the ways networked computers keep time. I now know that at the time of writing these words, 7:00AM Pacific Daylight Time on October 22, 2012,  exactly 1,350,914,400 seconds have elapsed since the first second of January 1st, 1970, Eastern Standard Time. That was the start of Unix Time.

I needed to learn to work in Unix Time in order to set the various opening times and completion deadlines for the exam process. I expect that by the time the next instructor puts together a MOOC, she or he will be greeted by a nice, friendly Coursera interface with pulldown menus and boxes to tick — which probably will come as a great relief to any humanities professors reading this, who don’t have any programming in their background.

[By coincidence, Unix was the last programming language I had any proficiency in, but I did not need to know Unix to use Unix Time. I just used an online converter. Unix was developed in 1969 at AT&T Bell Laboratories in New Jersey. Hence the 1970 EST baseline.]

In fact, time conversion issues in general turned out to be a  continuing, major headache in a course with students all over the world. One thing we will not do again is have 12:00PM Stanford Time, aka Coursera Time (i.e., PDT), as any of the course deadlines. It might seem a nice clean stopping point, and there are all those memories of Gary Cooper’s deadline in the classic Western movie High Noon, but many students missed the deadline for the first submitted assignment because they thought 12:00PM meant midnight, which in some parts of the world made them a whole day late.

The arbitrary illogicality of the AM/PM distinction is not apparent to those of us who grew up with it. But my course TA and I are now very aware of the problems it can lead to! In future, we’ll stick to unambiguous times that stay away from noon and midnight. But even then, with local computer systems usually working on local time, to say nothing of the different Summer and Winter Times, which change on different dates around the world, timing events in MOOCs is going to remain a problematic issue, just as it is for international travelers and professionals who collaborate globally over Skype and other conferencing services. (When I used the Unix Time conversion app, I had to remember that Unix thinks New Jersey is currently just two hours ahead of California, not the three hours United Airlines uses when it flies me there. Confusing, isn’t it?)

The reason why times are an issue in my course is that it is a course. At first glance, it may look little different from Khan Academy, where there are no time issues at all. But Khan Academy is really just an educational resource. (At least, that’s the part most people are familiar with and use, namely the video library that started it all. People use it as a video version of a textbook — or more precisely a video equivalent to that good old standby Cliffs Notes, which got many of us through an exam in an obligatory subject we were not particularly interested in.)

In contrast, in my case, as I’ve discussed earlier in this blog series (in particular, Part 6), my goal was to take a standard university course (one I’ve given many times over the years, at different universities, including Stanford) and make it available to anyone in the world, for free. To the degree I could make it happen, they would get the same learning experience.

That meant that the main goal would be to build a (short-lived) learning community. The video-recorded lectures and tutorials were simply tools to make that happen and to orchestrate events. Real learning takes place when students work on assignments on their own, when they repeatedly fail to solve a problem, and when they interact (with the professor and with one another) — not when they watch a lecture or read a book.

To achieve that goal, the MOOC would, as I stated in Part 6, involve admissions, lectures, peer interaction, professor interaction, problem-solving, assignments, exams, deadlines, and certification. To use the mnemonic I coined early on in this series, the basic design principle is WYSIWOSG: What You See Is What Our Students Get.

As we go forward, I intend to iterate on the course design, based on the data we collect from the students (and 64,000 students very definitely puts us into the Big Data realm). But my basic principle will remain that of offering a course, not the provision of a video library. And the reason for that should be obvious to anyone who has been following this blog series, as well as some of the posts on my other blogs Devlin’s Angle and profkeithdevlin.org. The focus is not on acquiring facts or mastering basic skills, but on learning to think a certain way (in my case, like a professional mathematician). And that requires both a lot of effort and (for most of us) a lot of interaction with others trying to achieve the same goal.

Our ancestors in the 11th Century started to develop what to this day remains the best way we know to achieve this at scale: the university, where people become members of a learning community in which learning takes place in a hothouse atmosphere that involves periods of intense interaction as deadlines loom, sustained by the rapidly formed social bonds that emerge as a result of that same pressure.

While I will likely experiment with variants of this model that allow for participation by students who have demanding, full-time jobs, I doubt I will abandon that basic model. It has lasted for a thousand years for a good reason. It works.

To be continued …

Final Lecture: MOOC Planning – Part 9

Published October 20, 2012 Free online courses 8 Comments
Tags: , , , , , , , , ,

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

I gave my last lecture of the course yesterday (discounting the tutorial session that will go out next week), and we are now starting a two week exam period.

“Giving” a lecture means the video becomes available for streaming. For logistic reasons (high among them, my survival and continued sanity — assuming anyone who organizes and gives a MOOC, for no payment, is sane), I recorded all the lectures weeks ago, well before the course started.  The weekly tutorial sessions come the closest to being live. I record them one or two days before posting, so I can use them to respond to issues raised in the online course discussion forum.

The initial course enrollment of 63,649 has dropped to 11,848 individuals that the platform says are still active on the site. At around 20%, that’s pretty high by current MOOC standards, though I don’t know whether that is something to be pleased about, since  it’s not at all clear what the right definition of “success” is for a MOOC.

Some might argue that 20% completion indicates that the standards are too low. I don’t think that’s true for my course. Completion does, after all, simply mean that a student is still engaged. The degree to which they have mastered the material is unclear. So having 80% drop out could mean the standard is too high.

In my case, I did not set out to achieve any particular completion rate; rather I adopted a WYSIWOSG approach — “What You See Is What Our Students Get.” I offered a MOOC that is essentially the first half of a ten week course I’ve given at many universities over the years, including Stanford. That meant my students would experience a Stanford-level course. But they would not be subject to passing a Stanford-level exam.

In fact, I could not offer anything close to a Stanford-exam experience. There is a Final Exam, and it has some challenging questions, but it is not taken under controlled, supervised conditions. Moreover,  since it involves constructing proofs, it cannot be machine graded, and thus has to be graded by other students, using a crowd sourcing method (Calibrated Peer Review). That put a significant limitation on the kinds of exam questions I could ask. On top of that, the grading is done by as many different people as there are students, and I assume most of them are not expert mathematicians. As a result, it’s at most a “better-than-nothing” solution. Would any of us want to be treated by a doctor whose final exam had been peer graded (only) by fellow students, even if the exam and the grading had been carried out under strictly controlled conditions?

On the other hand, looking at and attempting to evaluate the work of fellow students is a powerful learning experience, so if you view MOOCs as vehicles for learning, rather than a route to a qualification, then peer evaluation has a lot to be said for it. Traditional universities offer both learning and qualifications. MOOCs currently provide the former. Whether they eventually offer the latter as well remains to be seen. There are certainly ways it can be done, and that may be one way that MOOCs will make money. (Udacity already does offer a credentialing option, for a fee.)

In designing my course, I tried to optimize for learning in small groups, perhaps five to fifteen at a time. The goal was to build learning communities, within which students could help one another. Since there is no possibility of regular, direct interaction with the instructor (me) and my one TA (Paul), students have to seek help from fellow students. There is no other way. But, on its own, group work is not enough. Learning how to think mathematically (the focus of my course) requires feedback from others, but it needs to include feedback from people already expert in mathematical thinking. This means that, in order to truly succeed, not only do students need to work in groups (at least part of the time), and subject their attempts to the scrutiny of others, some of those interactions have to be with experts.

One original idea I had turned out not to work, though whether through the idea itself being flawed or the naive way we implemented it is not clear to me. That was to ask students at the start of the course to register if they had sufficient knowledge and experience with the course material to act as “Community TAs”, and be so designated in the discussion forums. Though over 600 signed up to play that role, many soon found they did not have sufficient knowledge to perform the task. Fortunately,a relatively small number of sign-ups did have the necessary background, as well as the interpersonal skills to give advice in a supporting, non-threatening way, and they more or less  ensured that the forum discussions met the needs of many students (or so it seems).

Another idea was to assign students to study groups, and use an initial survey to try to identify those with some background knowledge and seed them into the groups. Unfortunately, Coursera does not (yet) have functionality to support the creation and running of groups, apart from the creation of forum threads. So instead, in my first lecture, I suggested to the students that they form their own study groups in whatever way they could.

The first place to do that was, of course, the discussion forums on the course website, which very soon listed several pages of groups. Some used the discussion forum itself to work together, while others migrated offsite to some other location, physical or virtual, with Skype seeming a common medium. Shortly after the course launched, several students discovered GetStudyRoom, a virtual meeting place dedicated to MOOCs, built by a small startup company.

In any event, students quickly found their own solutions. But with students forming groups in so many different ways on different media, there was no way to track how many remained active or how successful they have been.

The study groups listed on the course website show a wide variety of criteria used to bring the groups together. Nationality and location were popular, with groups such as Brazil Study Group, Grupo de Estudo Português, All Students From Asia, and Study Group for Students Located in Karachi, Pakistan. Then there were groups with a more specific focus, such as Musicians, Parents of Homeschooled Children, Older/Retired English Speakers Discussion for Assignment 1, and, two of my favorites, After 8pm (UK time) English speakers with a day job and the delightfully named Just Hanging on Study Group.

The forum has seen a lot of activity: 15,088 posts and 13,622 comments, spread across 2712 different threads, viewed 430,769 times. Though I have been monitoring the forums on an almost daily basis, to maintain an overall sense of how the course is going, it’s clearly not possible to view everything. For the most part I restricted my attention to the posts that garnered a number of up-votes. Students vote posts up and down, and once a post shows 5 or more up-votes, I take that as an indication that the issue may be worth looking at.

The thread with the highest number of up-votes (165) was titled Deadlines way too short. Clearly, the question of deadlines was a hot topic. How, if at all, to respond to such feedback is no easy matter. In a course with tens of thousands of students, even a post with hundreds of up-votes represents just a tiny fraction of the class. Moreover, threads typically include opinions on both sides of an issue.

For instance, in threads about the pace of the course, some students complained that they did not have enough time to complete assignments, and pleaded for more relaxed deadlines, whereas others said they thrived on the pace, which stimulated them to keep on top of the material. For many, an ivy-league MOOC offers the first opportunity to experience an elite university course, and I think some are surprised at the level and pace. (I fact, I did keep the pace down for the first three weeks, but I also do that when I give a transition course in a regular setting, since I know how difficult it is to make that transition from high school math to university level mathematics.)

A common suggestion/request was to simply post the course materials online and let students access them according to their own schedules, much like Khan Academy. This raises a lot of issues about the nature of learning and the role MOOCs can (might? should?) play. But this blog post has already gone on long enough, so I’ll take up that issue next time.

To be continued …

The Crucible: MOOC Planning – Part 8

Published October 6, 2012 Free online courses 23 Comments
Tags: , , , , , , , ,

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, I have survived the initial three weeks of my first MOOC. Though the bulk of the work (and I mean “bulk”) came before the course launched, it has still taken my TA and me a lot of time to keep things ticking over. There are the in-flight corrections of the inevitable errors that occur in a new course, together with the challenges presented by a completely new medium and a buggy, beta release platform, still under very rapid development.

The course website shows 61,846 registered students, but I suspect many of those have long stopped any kind of connection to the course, and another large group are simply watching the lecture videos. The really pleasing figure is that the number of active users last week (week 3) was 19,298. Based on what I hear about other MOOCs, retaining one student in three is a good number.

Both my hands-on TA, Paul, and the course Research Associate, Molly, are graduate students in Stanford’s School of Education, and besides helping me with aspects of the course design, they are approaching the project as an opportunity to carry out research in learning, particularly mathematics learning. Given the massive amount of data a MOOC generates, the education research world can expect to see a series of papers coming from them in the months ahead.

I’m not trained in education research, but some observations are self-evident when you look over the course discussion forums – something I’ve spent a lot of time doing, both to gauge how the course is going and to look for ways to improve it, either by an in-course modification of for a future iteration of the course.

I’ve always felt that the essence of MOOC learning is community building. There is no hope that the “instructor” can do more than orchestrate events. Without regular close contact with the students, the video-recorded lectures and the various course notes and handouts are like firing off a shotgun on a misty Scottish moor. The shot flies out and disperses into the mist, and you just hope some of it hits a target. (I haven’t actually fired a shotgun on a Scottish moor, or anywhere else for that matter, but I’ve seen it on TV and it seems the right metaphor.) With 60,000 (or 20,000) students, I can’t allow myself to respond to a forum post or an email from any single student. I have to rely on the voting procedure (“Like/Dislike”) of the forums to help me decide which questions to address.

This means the student body has to resolve things among themselves. It was fascinating watching the activity on the discussion forums take shape and develop a profile over the first couple of weeks.

One huge benefit for the instructor is the virtual elimination of the potentially disruptive influence – present in almost any class with more than twenty or so students – of the small number of students for whom nothing is good enough. Even in a totally free course, put on by volunteers, for which no college credential is awarded, there were a few early posts of that kind. But in each case the individual was rapidly put in his or her place by replies from other students, and before long stopped posting, and very likely dropped the course.

(An interesting feature of this was that each time it occurred, a number of students emailed me in private – rather than on the public course forum – to say they did not agree with the complainer, and to tell me they were enjoying the course. Clearly, even with the possibility of anonymous forum posts, which Coursera allows, at least for now, some people prefer to keep their communication totally private.)

Of far greater interest, at least to me, was how the student body rapidly split into two camps, based on how they reacted to the course content. As I’ve discussed in earlier posts to this blog, my course is a high-school to university transition course for mathematics. It’s designed to help students make the difficult (and for most of us psychologically challenging) transition from high school mathematics, with its emphasis on learning to follow procedures to solve highly contrived “math problems”, to developing an ability to think logically, numerically, analytically, quantitatively, and algebraically (i.e., in aggregate, mathematically) about novel problems, including often ill-defined or ambiguous real-world problems.

When I give this kind of course to a traditional class of twenty-five or so entering college students, fresh out of high school, the vast majority of them have a really hard time with it. In my MOOC, in contrast, the student body has individuals of all ages, from late teens into their sixties and seventies, with different backgrounds and experiences, and many of them said they found this approach the most stimulating mathematics class they had ever taken. They loved grappling with the inherent ambiguity and open-ended nature of some of the problems.

Our schools (at least in the US), by focusing on one particular aspect of mathematics – the formal, procedural – I think badly shortchange our students. They send them into the world with a fine scalpel, but life in that world requires a fairly diverse toolkit – including WD40 and a large roll of duct tape.

The real world rarely presents us with neat, encapsulated problems that can be solved in ten minutes. Real world problems are messy, ambiguous, ill-defined, and often with internal contradictions. Yes, precise, formal mathematics can be very useful in helping to solve such problems. But of far broader applicability is what I have been calling “mathematical thinking”, the title of my course.

I suspect the students who seemed to take to my course like ducks to water were people well beyond high school, who had discovered for themselves what is involved in solving real problems. Judging by the forum discussions, they are having a blast.

The others, the ones whose experience of mathematics has, I suspect, been almost entirely the familiar, procedural-skills learning of the traditional K-12 math curriculum, keep searching for precision that simply is not there, or (and I’ve been focusing a lot on this in the first three weeks) where the goal is to learn how to develop that precision in the first place.

The process of starting with a messy, real world problem, where we have little more than our intuitions to guide us, and then slowly distilling some precision to help us deal with that problem, is hugely valuable. Indeed, it is the engine that powered (and continues to power) the entire development of our science and our technology. Yet, in our K-12 system we hardly ever help students to learn how to do that.

Done well, the activities of the traditional math class can be great fun. I certainly found it so, and have spent a large part of my life enjoying the challenges of pure mathematics research. But a lot of that fun comes from working within the precise definitions and clear rules of engagement of the discipline.  To me mathematics was chess on steroids. I loved it. Still do, for that matter. But relatively few citizens are interested in making  a career in mathematics. An education system that derives its goals from the ivory-towered pursuit of pure mathematics (and I use that phrase in an absolutely non-denigrating way, knowing full well how important it is to society and to our culture that those ivory towers exist) does not well serve the majority of students.

It requires some experience and sophistication in mathematics to see how skill in abstract, pure reasoning plays an important role in dealing with the more messy issues of the real world. There is an onus on those of us in the math ed community  to help others to appreciate the benefits available to them by way of improved mathematical ability.

As I have followed the forum discussions in my MOOC, I have started to wonder if one thing that MOOCs can give to mathematics higher education in spades is a mechanism to provide a real bridge between K-12 education and life in the world that follows. By coming together in a large, albeit virtual community, the precision-seeking individuals who want clear rules and guidelines to follow find themselves side-by-side (actually, keyboard-to-keyboard) with others (perhaps with weak formal mathematics skills) more used to approaching open-ended, novel problems of the kind the real world throws up all the time. If so, that would make the MOOC a powerful crucible that would benefit both groups, and thus society at large.

To be continued …

I'm Dr. Keith Devlin, a mathematician at Stanford University. I gave my first free, open, online math course in fall 2012, and have been offering it twice a year since then. This blog chronicles my experiences as they happen.

Twitter Updates

New Book 2012

New book 2011

New e-book 2011

New book 2011

October 2012
M T W T F S S
1234567
891011121314
15161718192021
22232425262728
293031  
%d bloggers like this: