Archive Page 2

Answering the 64,000-Students Questions

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

With the “instructional” part of the course finished and the remaining students working on the Final Exam (it will be peer graded next week), at last I can sit back and take a short breather. The next step will be to debrief and reflect with my two course assistants (both PhD students in the Stanford Graduate School of Education) and decide where to ride the MOOC beast next.

For sure I’ll offer another version of this course next year, with changes based on the huge amounts of data you get with a global online class of 64,000 students. Despite the enormous effort in designing, preparing, and running such a massive enterprise, there are three very good reasons to pursue this.

First, and this I believe is one of the main reasons why Stanford is supporting the development of MOOCs (I am not part of the central, policy-making administration), designing, running, and analyzing the learning outcomes of MOOCs is a tremendous research opportunity that will almost certainly result in new understandings of how people learn, and as a result very likely will enable the university to improve the learning experience of our regular on-campus students. After just five weeks, my two graduate assistants have enough data to write several dissertations, in addition to the one they need to get their doctorates.

Second, there is a huge, overall, feel-good factor for those of us involved, knowing that we can help to provide life-changing opportunities for people around the world who would otherwise have no access to quality higher education. Is what they get as good as being at Stanford? I very much doubt it, though the scientist in me says we should keep an open mind into the eventual outcomes of what is at present a very novel phenomenon. But if you compare a Stanford MOOC with the alternative of nothing at all, then already you have an excellent reason to continue.

Third, and this is something that anyone in education will acknowledge makes up for our earning a much lower salary than our (often less formally qualified) friends in the business and financial worlds, there is the pleasure of hearing first-hand from some of our more satisfied customers. The following is one of many appreciative emails and forum posts I have received as my course came to and end:

Mr. Devlin and all members of the Introduction To Mathematical Thinking team, I just wanted to say Thank You for everything that you have done to share your knowledge and giving your time and great effort to help others learn. I imagine that this is not an easy project to lead and sustain on a continuous basis. However, you have done a wonderful job in relaying your message. Through your efforts, you have helped many people in the process; especially me. Until this class, I hated math. I hated the idea of learning math or thinking in mathematically analogous methods that are applicable to real world situations. I just didn’t get it. I’m still a little confused about why I am able to comprehend your lessons as effectively as I am (which is saying a lot considering how much I hated math) when I have not been able to do so in the past. Now, I find myself looking forward to your classes everyday! I look forward to using what I have learned from the last video lectures or assignments and using those lessons in situations I did not think possible. And now, I love math! Your instruction has helped me to think more logically and to draw more concise conclusions with issues that I am trying to handle. This is indeed a skill. This is also a skill that you can build upon throughout your lifetime if one chooses to do so. Though I may not be at the level of learning that I should be at, I have learned more in the past three weeks than I have learned throughout my life; and I will continue to learn. I am very serious about this statement. So, thank you All. Thank you, Mr. Devlin. Great Job and Cheers!

Nice!

To be sure, there were trolls on the course discussion forum, for whom nothing we did was right. But one of the benefits of having tens of thousand of students is that within at most an hour of a flame post appearing, tens of others jumped on the offending individual, and within a short while all that was left was a “This comment has been deleted” notice. As the course wore on, the trolls simply dropped away.

Though there was the one individual who, in week four, posted a comment that he hated my teaching style and was learning nothing. Given that this was a free course that no one was under any compulsion to take, and for which no official credential was awarded, one wonders why this person stuck around for so long!

That example provided no more than an amusing anecdote to tell when I start to give talks on “What’s it like to teach 64,000 students?” (Invitations are already coming in.) But there is a somewhat closely related issue that I find far more significant.

Like almost all current MOOCs, there was no real credentialing in my course, so the focus was entirely on learning for its own sake. (As a lifelong math professor, used to teaching classes where many of the students were there because they needed to fulfill a mathematics requirement, having a class of students who were there purely voluntarily added appeal to my giving a MOOC.) To be sure, there were in-lecture quizzes, machine-graded assignments, and a peer evaluated final exam, but the only people who had access to any student’s results were myself, my two course assistants, and the student. Moreover, there was no official certification to back up a good result (the course offered two levels, Completion and Completion with Distinction), and turn it into a form of credential.

Yet many students had an ongoing obsession with their grades, and indeed pleaded with me from time to time to re-grade their work. (Clearly not possible in a 64,000 student MOOC. Besides, I never saw their work. How could I?) As a competitive person myself, I can appreciate the desire to do well. But with literally nothing at stake, I was at first surprised by the degree to which it bothered some of them. When I figured out what was probably going on, I found something that bothered me.

Unlike most MOOCs, mine, being at first-year university level, can be taken by high school students. Indeed, since my primary target audience comprised students entering or about to enter university to study mathematics or a math-related subject, I expected to get high school seniors, and designed my course as much as possible to accommodate them.

I’m guessing that the majority of students who were obsessed with grades were still at high school – indeed, most likely a US high school. That grade obsession I observed is, I suspect, simply a learned behavior that reflects the way our K-12 system turns the learning of a fascinating subject – one of humankind’s most amazing, creative, intellectual achievements – into a seemingly endless sequence of bite-sized pieces that are fed to the student in a mandated hamster-wheel.

No wonder they could not relax and enjoy learning for its own sake. Any natural curiosity and desire to learn – something all humans are born with – had been driven out of them by the very institution that is supposed to encourage and develop that trait. In its place was mere grade hunting.

Do I know this for a fact? No. That’s why I used those hedging words “guess” and “suspect”. But something has to explain that grade obsession in my course, and it certainly brought to mind Paul Lockhart’s wonderful essay A Mathematician’s Lament, which I had the privilege to bring to a wider audience some years ago.

But now I digress. Time to wrap up and check the dashboard on the course website see how many students have submitted the Final Exam so far.

Though this post has dropped the title “MOOC Planning”, I am going to keep posting here, as the project goes forward. Stay tuned.

To be continued …

It’s About Time (in Part): MOOC Planning – Part 10

 A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, lectures have ended and the course has now switched gears. For those still left in the course (17% of the final enrollment total of 64,045), the next two weeks are focused on trying to make sense of everything they have learned, and working on the final exam — which in the case of my course involves peer evaluation.

Calibrated Peer Review is not new. A study of its use in the high school system by Sadler and Good, published in 2006, has become compulsory reading for those of us planning and giving MOOCs that cover material that cannot be machine graded. [If you want to see how I am using it, just enroll in the class and read the description of the "Peer Review system". There is no obligation to do anything more than browse around the site! No one will know you are not simply a dog that can use a computer.]

As I was working on my course, Coursera was still frantically building out their platform to support peer evaluation. There was a lot of just-in-time construction. It’s been a long time since I’ve had to go behind a user-friendly interface and dig into the underlying code to do something on a computer, and the programming languages have all changed since I last did that.

One thing I had to learn was one of the ways networked computers keep time. I now know that at the time of writing these words, 7:00AM Pacific Daylight Time on October 22, 2012,  exactly 1,350,914,400 seconds have elapsed since the first second of January 1st, 1970, Eastern Standard Time. That was the start of Unix Time.

I needed to learn to work in Unix Time in order to set the various opening times and completion deadlines for the exam process. I expect that by the time the next instructor puts together a MOOC, she or he will be greeted by a nice, friendly Coursera interface with pulldown menus and boxes to tick — which probably will come as a great relief to any humanities professors reading this, who don’t have any programming in their background.

[By coincidence, Unix was the last programming language I had any proficiency in, but I did not need to know Unix to use Unix Time. I just used an online converter. Unix was developed in 1969 at AT&T Bell Laboratories in New Jersey. Hence the 1970 EST baseline.]

In fact, time conversion issues in general turned out to be a  continuing, major headache in a course with students all over the world. One thing we will not do again is have 12:00PM Stanford Time, aka Coursera Time (i.e., PDT), as any of the course deadlines. It might seem a nice clean stopping point, and there are all those memories of Gary Cooper’s deadline in the classic Western movie High Noon, but many students missed the deadline for the first submitted assignment because they thought 12:00PM meant midnight, which in some parts of the world made them a whole day late.

The arbitrary illogicality of the AM/PM distinction is not apparent to those of us who grew up with it. But my course TA and I are now very aware of the problems it can lead to! In future, we’ll stick to unambiguous times that stay away from noon and midnight. But even then, with local computer systems usually working on local time, to say nothing of the different Summer and Winter Times, which change on different dates around the world, timing events in MOOCs is going to remain a problematic issue, just as it is for international travelers and professionals who collaborate globally over Skype and other conferencing services. (When I used the Unix Time conversion app, I had to remember that Unix thinks New Jersey is currently just two hours ahead of California, not the three hours United Airlines uses when it flies me there. Confusing, isn’t it?)

The reason why times are an issue in my course is that it is a course. At first glance, it may look little different from Khan Academy, where there are no time issues at all. But Khan Academy is really just an educational resource. (At least, that’s the part most people are familiar with and use, namely the video library that started it all. People use it as a video version of a textbook — or more precisely a video equivalent to that good old standby Cliffs Notes, which got many of us through an exam in an obligatory subject we were not particularly interested in.)

In contrast, in my case, as I’ve discussed earlier in this blog series (in particular, Part 6), my goal was to take a standard university course (one I’ve given many times over the years, at different universities, including Stanford) and make it available to anyone in the world, for free. To the degree I could make it happen, they would get the same learning experience.

That meant that the main goal would be to build a (short-lived) learning community. The video-recorded lectures and tutorials were simply tools to make that happen and to orchestrate events. Real learning takes place when students work on assignments on their own, when they repeatedly fail to solve a problem, and when they interact (with the professor and with one another) — not when they watch a lecture or read a book.

To achieve that goal, the MOOC would, as I stated in Part 6, involve admissions, lectures, peer interaction, professor interaction, problem-solving, assignments, exams, deadlines, and certification. To use the mnemonic I coined early on in this series, the basic design principle is WYSIWOSG: What You See Is What Our Students Get.

As we go forward, I intend to iterate on the course design, based on the data we collect from the students (and 64,000 students very definitely puts us into the Big Data realm). But my basic principle will remain that of offering a course, not the provision of a video library. And the reason for that should be obvious to anyone who has been following this blog series, as well as some of the posts on my other blogs Devlin’s Angle and profkeithdevlin.org. The focus is not on acquiring facts or mastering basic skills, but on learning to think a certain way (in my case, like a professional mathematician). And that requires both a lot of effort and (for most of us) a lot of interaction with others trying to achieve the same goal.

Our ancestors in the 11th Century started to develop what to this day remains the best way we know to achieve this at scale: the university, where people become members of a learning community in which learning takes place in a hothouse atmosphere that involves periods of intense interaction as deadlines loom, sustained by the rapidly formed social bonds that emerge as a result of that same pressure.

While I will likely experiment with variants of this model that allow for participation by students who have demanding, full-time jobs, I doubt I will abandon that basic model. It has lasted for a thousand years for a good reason. It works.

To be continued …

Final Lecture: MOOC Planning – Part 9

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

I gave my last lecture of the course yesterday (discounting the tutorial session that will go out next week), and we are now starting a two week exam period.

“Giving” a lecture means the video becomes available for streaming. For logistic reasons (high among them, my survival and continued sanity — assuming anyone who organizes and gives a MOOC, for no payment, is sane), I recorded all the lectures weeks ago, well before the course started.  The weekly tutorial sessions come the closest to being live. I record them one or two days before posting, so I can use them to respond to issues raised in the online course discussion forum.

The initial course enrollment of 63,649 has dropped to 11,848 individuals that the platform says are still active on the site. At around 20%, that’s pretty high by current MOOC standards, though I don’t know whether that is something to be pleased about, since  it’s not at all clear what the right definition of “success” is for a MOOC.

Some might argue that 20% completion indicates that the standards are too low. I don’t think that’s true for my course. Completion does, after all, simply mean that a student is still engaged. The degree to which they have mastered the material is unclear. So having 80% drop out could mean the standard is too high.

In my case, I did not set out to achieve any particular completion rate; rather I adopted a WYSIWOSG approach — “What You See Is What Our Students Get.” I offered a MOOC that is essentially the first half of a ten week course I’ve given at many universities over the years, including Stanford. That meant my students would experience a Stanford-level course. But they would not be subject to passing a Stanford-level exam.

In fact, I could not offer anything close to a Stanford-exam experience. There is a Final Exam, and it has some challenging questions, but it is not taken under controlled, supervised conditions. Moreover,  since it involves constructing proofs, it cannot be machine graded, and thus has to be graded by other students, using a crowd sourcing method (Calibrated Peer Review). That put a significant limitation on the kinds of exam questions I could ask. On top of that, the grading is done by as many different people as there are students, and I assume most of them are not expert mathematicians. As a result, it’s at most a “better-than-nothing” solution. Would any of us want to be treated by a doctor whose final exam had been peer graded (only) by fellow students, even if the exam and the grading had been carried out under strictly controlled conditions?

On the other hand, looking at and attempting to evaluate the work of fellow students is a powerful learning experience, so if you view MOOCs as vehicles for learning, rather than a route to a qualification, then peer evaluation has a lot to be said for it. Traditional universities offer both learning and qualifications. MOOCs currently provide the former. Whether they eventually offer the latter as well remains to be seen. There are certainly ways it can be done, and that may be one way that MOOCs will make money. (Udacity already does offer a credentialing option, for a fee.)

In designing my course, I tried to optimize for learning in small groups, perhaps five to fifteen at a time. The goal was to build learning communities, within which students could help one another. Since there is no possibility of regular, direct interaction with the instructor (me) and my one TA (Paul), students have to seek help from fellow students. There is no other way. But, on its own, group work is not enough. Learning how to think mathematically (the focus of my course) requires feedback from others, but it needs to include feedback from people already expert in mathematical thinking. This means that, in order to truly succeed, not only do students need to work in groups (at least part of the time), and subject their attempts to the scrutiny of others, some of those interactions have to be with experts.

One original idea I had turned out not to work, though whether through the idea itself being flawed or the naive way we implemented it is not clear to me. That was to ask students at the start of the course to register if they had sufficient knowledge and experience with the course material to act as “Community TAs”, and be so designated in the discussion forums. Though over 600 signed up to play that role, many soon found they did not have sufficient knowledge to perform the task. Fortunately,a relatively small number of sign-ups did have the necessary background, as well as the interpersonal skills to give advice in a supporting, non-threatening way, and they more or less  ensured that the forum discussions met the needs of many students (or so it seems).

Another idea was to assign students to study groups, and use an initial survey to try to identify those with some background knowledge and seed them into the groups. Unfortunately, Coursera does not (yet) have functionality to support the creation and running of groups, apart from the creation of forum threads. So instead, in my first lecture, I suggested to the students that they form their own study groups in whatever way they could.

The first place to do that was, of course, the discussion forums on the course website, which very soon listed several pages of groups. Some used the discussion forum itself to work together, while others migrated offsite to some other location, physical or virtual, with Skype seeming a common medium. Shortly after the course launched, several students discovered GetStudyRoom, a virtual meeting place dedicated to MOOCs, built by a small startup company.

In any event, students quickly found their own solutions. But with students forming groups in so many different ways on different media, there was no way to track how many remained active or how successful they have been.

The study groups listed on the course website show a wide variety of criteria used to bring the groups together. Nationality and location were popular, with groups such as Brazil Study Group, Grupo de Estudo Português, All Students From Asia, and Study Group for Students Located in Karachi, Pakistan. Then there were groups with a more specific focus, such as Musicians, Parents of Homeschooled Children, Older/Retired English Speakers Discussion for Assignment 1, and, two of my favorites, After 8pm (UK time) English speakers with a day job and the delightfully named Just Hanging on Study Group.

The forum has seen a lot of activity: 15,088 posts and 13,622 comments, spread across 2712 different threads, viewed 430,769 times. Though I have been monitoring the forums on an almost daily basis, to maintain an overall sense of how the course is going, it’s clearly not possible to view everything. For the most part I restricted my attention to the posts that garnered a number of up-votes. Students vote posts up and down, and once a post shows 5 or more up-votes, I take that as an indication that the issue may be worth looking at.

The thread with the highest number of up-votes (165) was titled Deadlines way too short. Clearly, the question of deadlines was a hot topic. How, if at all, to respond to such feedback is no easy matter. In a course with tens of thousands of students, even a post with hundreds of up-votes represents just a tiny fraction of the class. Moreover, threads typically include opinions on both sides of an issue.

For instance, in threads about the pace of the course, some students complained that they did not have enough time to complete assignments, and pleaded for more relaxed deadlines, whereas others said they thrived on the pace, which stimulated them to keep on top of the material. For many, an ivy-league MOOC offers the first opportunity to experience an elite university course, and I think some are surprised at the level and pace. (I fact, I did keep the pace down for the first three weeks, but I also do that when I give a transition course in a regular setting, since I know how difficult it is to make that transition from high school math to university level mathematics.)

A common suggestion/request was to simply post the course materials online and let students access them according to their own schedules, much like Khan Academy. This raises a lot of issues about the nature of learning and the role MOOCs can (might? should?) play. But this blog post has already gone on long enough, so I’ll take up that issue next time.

To be continued …

The Crucible: MOOC Planning – Part 8

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, I have survived the initial three weeks of my first MOOC. Though the bulk of the work (and I mean “bulk”) came before the course launched, it has still taken my TA and me a lot of time to keep things ticking over. There are the in-flight corrections of the inevitable errors that occur in a new course, together with the challenges presented by a completely new medium and a buggy, beta release platform, still under very rapid development.

The course website shows 61,846 registered students, but I suspect many of those have long stopped any kind of connection to the course, and another large group are simply watching the lecture videos. The really pleasing figure is that the number of active users last week (week 3) was 19,298. Based on what I hear about other MOOCs, retaining one student in three is a good number.

Both my hands-on TA, Paul, and the course Research Associate, Molly, are graduate students in Stanford’s School of Education, and besides helping me with aspects of the course design, they are approaching the project as an opportunity to carry out research in learning, particularly mathematics learning. Given the massive amount of data a MOOC generates, the education research world can expect to see a series of papers coming from them in the months ahead.

I’m not trained in education research, but some observations are self-evident when you look over the course discussion forums – something I’ve spent a lot of time doing, both to gauge how the course is going and to look for ways to improve it, either by an in-course modification of for a future iteration of the course.

I’ve always felt that the essence of MOOC learning is community building. There is no hope that the “instructor” can do more than orchestrate events. Without regular close contact with the students, the video-recorded lectures and the various course notes and handouts are like firing off a shotgun on a misty Scottish moor. The shot flies out and disperses into the mist, and you just hope some of it hits a target. (I haven’t actually fired a shotgun on a Scottish moor, or anywhere else for that matter, but I’ve seen it on TV and it seems the right metaphor.) With 60,000 (or 20,000) students, I can’t allow myself to respond to a forum post or an email from any single student. I have to rely on the voting procedure (“Like/Dislike”) of the forums to help me decide which questions to address.

This means the student body has to resolve things among themselves. It was fascinating watching the activity on the discussion forums take shape and develop a profile over the first couple of weeks.

One huge benefit for the instructor is the virtual elimination of the potentially disruptive influence – present in almost any class with more than twenty or so students – of the small number of students for whom nothing is good enough. Even in a totally free course, put on by volunteers, for which no college credential is awarded, there were a few early posts of that kind. But in each case the individual was rapidly put in his or her place by replies from other students, and before long stopped posting, and very likely dropped the course.

(An interesting feature of this was that each time it occurred, a number of students emailed me in private – rather than on the public course forum – to say they did not agree with the complainer, and to tell me they were enjoying the course. Clearly, even with the possibility of anonymous forum posts, which Coursera allows, at least for now, some people prefer to keep their communication totally private.)

Of far greater interest, at least to me, was how the student body rapidly split into two camps, based on how they reacted to the course content. As I’ve discussed in earlier posts to this blog, my course is a high-school to university transition course for mathematics. It’s designed to help students make the difficult (and for most of us psychologically challenging) transition from high school mathematics, with its emphasis on learning to follow procedures to solve highly contrived “math problems”, to developing an ability to think logically, numerically, analytically, quantitatively, and algebraically (i.e., in aggregate, mathematically) about novel problems, including often ill-defined or ambiguous real-world problems.

When I give this kind of course to a traditional class of twenty-five or so entering college students, fresh out of high school, the vast majority of them have a really hard time with it. In my MOOC, in contrast, the student body has individuals of all ages, from late teens into their sixties and seventies, with different backgrounds and experiences, and many of them said they found this approach the most stimulating mathematics class they had ever taken. They loved grappling with the inherent ambiguity and open-ended nature of some of the problems.

Our schools (at least in the US), by focusing on one particular aspect of mathematics – the formal, procedural – I think badly shortchange our students. They send them into the world with a fine scalpel, but life in that world requires a fairly diverse toolkit – including WD40 and a large roll of duct tape.

The real world rarely presents us with neat, encapsulated problems that can be solved in ten minutes. Real world problems are messy, ambiguous, ill-defined, and often with internal contradictions. Yes, precise, formal mathematics can be very useful in helping to solve such problems. But of far broader applicability is what I have been calling “mathematical thinking”, the title of my course.

I suspect the students who seemed to take to my course like ducks to water were people well beyond high school, who had discovered for themselves what is involved in solving real problems. Judging by the forum discussions, they are having a blast.

The others, the ones whose experience of mathematics has, I suspect, been almost entirely the familiar, procedural-skills learning of the traditional K-12 math curriculum, keep searching for precision that simply is not there, or (and I’ve been focusing a lot on this in the first three weeks) where the goal is to learn how to develop that precision in the first place.

The process of starting with a messy, real world problem, where we have little more than our intuitions to guide us, and then slowly distilling some precision to help us deal with that problem, is hugely valuable. Indeed, it is the engine that powered (and continues to power) the entire development of our science and our technology. Yet, in our K-12 system we hardly ever help students to learn how to do that.

Done well, the activities of the traditional math class can be great fun. I certainly found it so, and have spent a large part of my life enjoying the challenges of pure mathematics research. But a lot of that fun comes from working within the precise definitions and clear rules of engagement of the discipline.  To me mathematics was chess on steroids. I loved it. Still do, for that matter. But relatively few citizens are interested in making  a career in mathematics. An education system that derives its goals from the ivory-towered pursuit of pure mathematics (and I use that phrase in an absolutely non-denigrating way, knowing full well how important it is to society and to our culture that those ivory towers exist) does not well serve the majority of students.

It requires some experience and sophistication in mathematics to see how skill in abstract, pure reasoning plays an important role in dealing with the more messy issues of the real world. There is an onus on those of us in the math ed community  to help others to appreciate the benefits available to them by way of improved mathematical ability.

As I have followed the forum discussions in my MOOC, I have started to wonder if one thing that MOOCs can give to mathematics higher education in spades is a mechanism to provide a real bridge between K-12 education and life in the world that follows. By coming together in a large, albeit virtual community, the precision-seeking individuals who want clear rules and guidelines to follow find themselves side-by-side (actually, keyboard-to-keyboard) with others (perhaps with weak formal mathematics skills) more used to approaching open-ended, novel problems of the kind the real world throws up all the time. If so, that would make the MOOC a powerful crucible that would benefit both groups, and thus society at large.

To be continued …

Liftoff: MOOC planning – Part 7

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

It’s been three weeks since I last posted to this blog. The reason for the delay is I was swamped getting everything ready for the launch of my course four days ago, on Monday of this week. As of first thing this morning there are 57,592 students enrolled in the class.

The course was featured in an article on MOOCs in USA Today. It was a good article, but like every other news report I’ve seen on MOOCs, the focus was on the video lectures. Those certainly take a fair amount of time on the part of the instructor (me, in this case), and are perhaps the most visible feature of a MOOC, just as the classroom lecture is the most visible part of many on-campus courses.

For some subjects, lectures, either in-person or on a computer screen, may be a major part of a course. But for conceptual mathematics, which is what my course is about, they are one of the least important features.

Learning to think mathematically is like learning to swim, to ride a bicycle, to ski, to play golf, or to play a musical instrument. You can probably get some idea by having someone explain it to you, but you won’t learn how to do it that way. The key words in that last clause are “learn” and “do”. There is really only one way to learn how to do something, and that is by doing it. Or, to put it more bluntly, the only way to achieve mastery is by repeated failure. You keep trying until you get it. The one thing that can help is having someone who already has mastery look at your attempts and give you constructive feedback.

In fact, failing in attempting to do something new isn’t really failure at all in the sense the word is usually used. Rather, a failed attempt is a step towards eventual success. Edison put it well when asked how he felt about his many failures to make a light bulb. He replied, “I have not failed. I’ve just found 10,000 ways that don’t work.”

After just one week of my course, I’ve seen a lot of learning going on, but it wasn’t in the lectures. Even if I’d been able to see each student watching the lecture, I would not have seen much learning going on, if any.  Rather, the learning I saw was on the discussion forums, primarily the ones focused on the assignments I gave out after each lecture. As I explained to the students, the course assignments and the associated forum discussions are the heart of the course.

So what is my part in all of this? Well, first of all, I have to admit I am uncomfortable with the title “instructor,” since that does not really reflect my role, but it’s the name society generally uses. “Course designer, conductor (as for an orchestra), and exemplar” would be a much better reflection of what I have been doing. Once the course was designed, the lectures recorded, and all the ancillary materials prepared, my task was to set the agenda, provide motivation and context for the various topics, and give examples of mathematical thinking.

The rest is up to the students. It has to be. (At least, I don’t know of any other way to learn how to think mathematically.) To be sure, in a physical class, the instructor (and or the TAs) can interact with the students, and (if it occurs) that can be a huge factor. But that simply helps the students learn by repeated failure, it does not eliminate the need for that learning-by-trying-and-failing process. Let’s face it, if you are not failing at something, you have already learned it, and should move on to the next step or topic. (With understanding, once you get it, you don’t need to practice!)

In a MOOC, that regular contact with the instructor and or the TAs is missing, of course. That means the students have to rely on one another for feedback. This is where the Coursera platform delivers. Here are some recent stats from my course website:

Total Registered Users 57592
Active Users Last Week 32123

Video Lectures

Total Streaming Views 77415
Total Downloads 19491
# Unique users watching videos 21712

Discussion Forums

Total Threads 641
Total Posts 5414
Total Comments 3823
Total Views 119489

Though I’d like to see a lot more students posting to the forums, with almost 120,000 views (after just one lecture and one course assignment!), it’s clear that that is where a lot of the action is.

As I surmised in an early blog-post, I don’t think it was the widespread availability of video technology and sites like YouTube that set the scene for MOOCs. To my mind, Facebook opened the floodgates, by making digitally-mediated social networking a mainstream human activity. (I’d better add Skype, since there are already several Skype-based study groups for my course. And of course, students who live close together can do it the old-fashioned way, by getting together in person to work through the assignments.)

One feature of the course that did not surprise me was the sense of feeling lost some students reported (and I’m sure many more felt), in some cases maybe being accompanied by panic. For most students, not only does my course present a side of mathematics they have never seen before (the world of the professional mathematicians), on top of that, none of the strategies they were taught to succeed in high-school math work any more.

Because the focus of the course is on mathematical thinking, I can’t provide the students with a list of rules to follow, templates to recognize, or procedures to follow. The whole point is to help them develop the ability to solve novel problems for which no  rules are known.

Of course, at this stage, the problems I give them are ones that have been solved long ago, and which have been shown to provide good learning material. But to the student, they are new, and that’s what matters in terms of learning. Unless, of course, they look for the solution on the Web, which defeats the whole purpose. But in a voluntary course where the focus is on process, not “getting answers,” and which provides no college credential, I hope that does not occur. In fact, one of the things that attracted me to free MOOCs was that the students would enroll because they wanted to learn, not because they were forced to learn or simply in need of a diploma. (We mathematicians get a lot of students like that! But we get paid to teach those classes. So far, no one is paying MOOC faculty for their efforts.)

Most US students have a particularly hard time with this “there are no templates” approach, because of the way mathematics is typically taught in American schools.  Instead of helping students to learn mathematics by figuring it out for themselves, teachers frequently begin by providing instruction and following it up with examples. Michael Pershan has a nice summary of this on YouTube. (His initial focus is on Khan Academy, but Khan is simply providing a service that is molded on, and fits into, the US system. The crucial issue Pershan’s video addresses is the system.)

The pros and cons of the two approaches, instruction based or guided discovery, remains a topic of debate in this country, but in the case of my course, there can be no debate. The goal is to develop the ability to encounter a novel problem and eventually be able to figure it out. Providing instruction in such a course would be like giving a golf cart to someone who wants to walk to lose weight! It might get them to their destination with less effort, but it would defeat the real goal.

Having thought at length about how to structure this first version of the course, and played around with some approaches, I ended up, as I thought I probably would, going minimal.  Virtually no instruction, and what little there is presented as examples of mathematical thinking in action, not by way of a carefully planned lesson. I was pretty sure I’d do that, because that’s how I’ve always conducted classes where the goal is student learning (as opposed to passing a standardized test).

There are a number of studies pointing out the dangers of over-planned lessons, one of the most famous and influential being Alan Schoenfeld’s 1988 paper in Educational Psychologist (Vol 23(2), 1988), When Good Teaching Leads to Bad Results: The Disasters of “Well Taught” Mathematics Courses. Still, as I said, I did play around with alternatives, since I was worried how students would fare without having regular access to the instructor and the TAs. I may have to re-visit those other approaches, if things go worse this time than I fear.

But this time round, what the student gets is as close a simulation as I can produce of sitting next to me as I work through the material. The result is not perfect. It’s not meant to be. There are minor errors in there. It’s meant to provide an example of how a professional mathematician sets about things. Definitely not intended as something to be perceived as an entry in an instruction manual.

After those work sessions were video-recorded, they were edited, of course, but only to cut out pauses while I thought, and to speed up the handwriting in places. I found that on a screen, watching the handwriting in real time looked painfully slow, and rapidly became irritating, particularly in places where I had to write out an entire sentence. So I took a leaf out of Vi Hart‘s wonderful repertoire. The speed ramping ended up being the only place that modern digital technology actually impinged on the lecture. Everywhere else it merely provided a medium. The approach would be familiar to Euclid if he were somehow to come back and take (or give) the class.

To be continued …

You may be interested in two recent videos featuring the founders of the two Stanford MOOC platforms that started the current explosion of interest in these courses. In one, Sebastian Thrun talks about Udacity. In the other Daphne Koller discusses the creation of Coursera.

The “C” in “MOOC”: MOOC planning – Part 6

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

A few days ago, I went into our campus TV studio with the two course assistants for my upcoming MOOC, to record a short video introducing them to the students.  The students will see a lot of me, but my two TAs will be working behind the scenes, and the students will encounter them only through their contributions to the forum discussions. The videos were intended to compensate for that lack of human contact.

During the course of recording that video, the three of us got into a discussion about our backgrounds, our motives in giving the MOOC, and our views on mathematics, science, education, and our expectations for the MOOC format. The camera was rolling all the time, and we were able to select a few parts of that discussion and create a second video that I think will help our students understand some of our thinking in putting this course together.  I posted copies of both videos on YouTube.  (They are much lower resolution than the videos the registered students will see on the course website when it goes live on September 17 — the “first day of classes”.) I think the two videos provide an insight into our thinking as we designed this course.

The fact that the current round of MOOCs have a “first day of class” at all has been a matter of some debate. The C in MOOC stands for “course”, but is this the best way to go?  For example, see this blogpost from a graduate student at Berkeley, who argues for a more open framework of learning resources. He makes some good points that all of us involved in this initiative have thought about and discussed, but I’m not sure the kind of thing he advocates can work for disciplines and subjects that depend heavily on student-faculty and student-student interaction, as mine does.

In fact, I’m not sure the MOOC will work sufficiently well at all in such cases; this is very much an experiment that I anticipate will continue for several years before we get good answers either way. For the first iteration, it makes sense to start with a model we know does work. And important (we think!) elements of that model are, to repeat Sebastian Thrun’s list, as quoted in the Berkeley student’s blog: admissions, lectures, peer interaction, professor interaction, problem-solving, assignments, exams, deadlines, and certification. To use the mnemonic I coined earlier in this series, our basic design principle is WYSIWOSG: What You See Is What Our Students Get.

Since these courses are free, we can, of course, do a lot of A/B testing in future years, to see which of these truly are crucial, which can be changed and how, and which can be dropped. I suspect the answers we get will vary from discipline to discipline, and possibly from course to course.

All of us involved in this MOOC movement are trying to find out the best way that works for our particular discipline and is consistent with our own style as instructors. As I indicated in Part 4 of this diary, I think it makes sense to begin by trying to implement in a MOOC as much of our tried-and-trusted classroom-based teaching as we can (as Thrun did with Udacity), and then iterating in the light of what we learn.

This is why, instead of hiring a mathematics graduate student to TA my course, which is what I would have done for an on campus class, I brought onto my team two graduate students from Stanford’s School of Education with several years of experience in learning design and the use of technology in education. In addition to helping me with the design and running of the course, they will conduct research into the course’s efficacy and try to understand how learning occurs in a MOOC. (Other than a brief, non-compulsory questionnaire at the start and finish of the course, all their research will be based on data gathered on the Coursera course platform and human monitoring of the forum discussions. One huge benefit of MOOCs is that they facilitate Big Data research.)

It’s live beta, folks.

To be continued …

My first big mistake: MOOC planning – Part 5

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Wow! With three weeks to go to the course launch, I checked the course registrations for the first time. So far, almost 35,000 students have signed up. In theory, I knew this would happen; that’s been everyone else’s experience with MOOCs. But when you actually see that kind of figure on the stats page for your own course, it makes a big impression.

Then I made my first big mistake. I sent out a welcome email to the students who had already registered. That part was not the mistake. Of course I’d want to welcome the students! Nor was my error to mention this blog in my email. It does, after all, provide students with some background on my thinking behind the course and what I want to achieve. My mistake was not closing comments on this blog before I sent out the email.

I was online when the first few comments started coming in, and as usual I responded to them. Then the flood began. I managed to close comments before the WordPress servers shut me down.  :-)

So, sorry to all those who wrote in to this blog and did not get a reply. The Coursera platform, which is desgned to handle classes of many thousands of students, offers opportunities to comment and exchange ideas, with a mechanism to bring to the attention of me and my teaching assistants any discussion thread that is generating a lot of interest. That will be available once the course starts.

I wonder what my next mistake will be.

To be continued …


I'm Dr. Keith Devlin, a mathematician at Stanford University. In fall 2012, I gave my first free, open, online math course and this spring I am giving my second. This blog chronicles my experiences as they happen.

Twitter Updates

New Book 2012

New book 2011

New e-book 2011

New book 2011

May 2013
M T W T F S S
« Mar    
 12345
6789101112
13141516171819
20212223242526
2728293031  

Follow

Get every new post delivered to your Inbox.

Join 441 other followers

%d bloggers like this: