Posts Tagged 'Coursera'



It’s About Time (in Part): MOOC Planning – Part 10

 A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, lectures have ended and the course has now switched gears. For those still left in the course (17% of the final enrollment total of 64,045), the next two weeks are focused on trying to make sense of everything they have learned, and working on the final exam — which in the case of my course involves peer evaluation.

Calibrated Peer Review is not new. A study of its use in the high school system by Sadler and Good, published in 2006, has become compulsory reading for those of us planning and giving MOOCs that cover material that cannot be machine graded. [If you want to see how I am using it, just enroll in the class and read the description of the “Peer Review system”. There is no obligation to do anything more than browse around the site! No one will know you are not simply a dog that can use a computer.]

As I was working on my course, Coursera was still frantically building out their platform to support peer evaluation. There was a lot of just-in-time construction. It’s been a long time since I’ve had to go behind a user-friendly interface and dig into the underlying code to do something on a computer, and the programming languages have all changed since I last did that.

One thing I had to learn was one of the ways networked computers keep time. I now know that at the time of writing these words, 7:00AM Pacific Daylight Time on October 22, 2012,  exactly 1,350,914,400 seconds have elapsed since the first second of January 1st, 1970, Eastern Standard Time. That was the start of Unix Time.

I needed to learn to work in Unix Time in order to set the various opening times and completion deadlines for the exam process. I expect that by the time the next instructor puts together a MOOC, she or he will be greeted by a nice, friendly Coursera interface with pulldown menus and boxes to tick — which probably will come as a great relief to any humanities professors reading this, who don’t have any programming in their background.

[By coincidence, Unix was the last programming language I had any proficiency in, but I did not need to know Unix to use Unix Time. I just used an online converter. Unix was developed in 1969 at AT&T Bell Laboratories in New Jersey. Hence the 1970 EST baseline.]

In fact, time conversion issues in general turned out to be a  continuing, major headache in a course with students all over the world. One thing we will not do again is have 12:00PM Stanford Time, aka Coursera Time (i.e., PDT), as any of the course deadlines. It might seem a nice clean stopping point, and there are all those memories of Gary Cooper’s deadline in the classic Western movie High Noon, but many students missed the deadline for the first submitted assignment because they thought 12:00PM meant midnight, which in some parts of the world made them a whole day late.

The arbitrary illogicality of the AM/PM distinction is not apparent to those of us who grew up with it. But my course TA and I are now very aware of the problems it can lead to! In future, we’ll stick to unambiguous times that stay away from noon and midnight. But even then, with local computer systems usually working on local time, to say nothing of the different Summer and Winter Times, which change on different dates around the world, timing events in MOOCs is going to remain a problematic issue, just as it is for international travelers and professionals who collaborate globally over Skype and other conferencing services. (When I used the Unix Time conversion app, I had to remember that Unix thinks New Jersey is currently just two hours ahead of California, not the three hours United Airlines uses when it flies me there. Confusing, isn’t it?)

The reason why times are an issue in my course is that it is a course. At first glance, it may look little different from Khan Academy, where there are no time issues at all. But Khan Academy is really just an educational resource. (At least, that’s the part most people are familiar with and use, namely the video library that started it all. People use it as a video version of a textbook — or more precisely a video equivalent to that good old standby Cliffs Notes, which got many of us through an exam in an obligatory subject we were not particularly interested in.)

In contrast, in my case, as I’ve discussed earlier in this blog series (in particular, Part 6), my goal was to take a standard university course (one I’ve given many times over the years, at different universities, including Stanford) and make it available to anyone in the world, for free. To the degree I could make it happen, they would get the same learning experience.

That meant that the main goal would be to build a (short-lived) learning community. The video-recorded lectures and tutorials were simply tools to make that happen and to orchestrate events. Real learning takes place when students work on assignments on their own, when they repeatedly fail to solve a problem, and when they interact (with the professor and with one another) — not when they watch a lecture or read a book.

To achieve that goal, the MOOC would, as I stated in Part 6, involve admissions, lectures, peer interaction, professor interaction, problem-solving, assignments, exams, deadlines, and certification. To use the mnemonic I coined early on in this series, the basic design principle is WYSIWOSG: What You See Is What Our Students Get.

As we go forward, I intend to iterate on the course design, based on the data we collect from the students (and 64,000 students very definitely puts us into the Big Data realm). But my basic principle will remain that of offering a course, not the provision of a video library. And the reason for that should be obvious to anyone who has been following this blog series, as well as some of the posts on my other blogs Devlin’s Angle and profkeithdevlin.org. The focus is not on acquiring facts or mastering basic skills, but on learning to think a certain way (in my case, like a professional mathematician). And that requires both a lot of effort and (for most of us) a lot of interaction with others trying to achieve the same goal.

Our ancestors in the 11th Century started to develop what to this day remains the best way we know to achieve this at scale: the university, where people become members of a learning community in which learning takes place in a hothouse atmosphere that involves periods of intense interaction as deadlines loom, sustained by the rapidly formed social bonds that emerge as a result of that same pressure.

While I will likely experiment with variants of this model that allow for participation by students who have demanding, full-time jobs, I doubt I will abandon that basic model. It has lasted for a thousand years for a good reason. It works.

To be continued …

Final Lecture: MOOC Planning – Part 9

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

I gave my last lecture of the course yesterday (discounting the tutorial session that will go out next week), and we are now starting a two week exam period.

“Giving” a lecture means the video becomes available for streaming. For logistic reasons (high among them, my survival and continued sanity — assuming anyone who organizes and gives a MOOC, for no payment, is sane), I recorded all the lectures weeks ago, well before the course started.  The weekly tutorial sessions come the closest to being live. I record them one or two days before posting, so I can use them to respond to issues raised in the online course discussion forum.

The initial course enrollment of 63,649 has dropped to 11,848 individuals that the platform says are still active on the site. At around 20%, that’s pretty high by current MOOC standards, though I don’t know whether that is something to be pleased about, since  it’s not at all clear what the right definition of “success” is for a MOOC.

Some might argue that 20% completion indicates that the standards are too low. I don’t think that’s true for my course. Completion does, after all, simply mean that a student is still engaged. The degree to which they have mastered the material is unclear. So having 80% drop out could mean the standard is too high.

In my case, I did not set out to achieve any particular completion rate; rather I adopted a WYSIWOSG approach — “What You See Is What Our Students Get.” I offered a MOOC that is essentially the first half of a ten week course I’ve given at many universities over the years, including Stanford. That meant my students would experience a Stanford-level course. But they would not be subject to passing a Stanford-level exam.

In fact, I could not offer anything close to a Stanford-exam experience. There is a Final Exam, and it has some challenging questions, but it is not taken under controlled, supervised conditions. Moreover,  since it involves constructing proofs, it cannot be machine graded, and thus has to be graded by other students, using a crowd sourcing method (Calibrated Peer Review). That put a significant limitation on the kinds of exam questions I could ask. On top of that, the grading is done by as many different people as there are students, and I assume most of them are not expert mathematicians. As a result, it’s at most a “better-than-nothing” solution. Would any of us want to be treated by a doctor whose final exam had been peer graded (only) by fellow students, even if the exam and the grading had been carried out under strictly controlled conditions?

On the other hand, looking at and attempting to evaluate the work of fellow students is a powerful learning experience, so if you view MOOCs as vehicles for learning, rather than a route to a qualification, then peer evaluation has a lot to be said for it. Traditional universities offer both learning and qualifications. MOOCs currently provide the former. Whether they eventually offer the latter as well remains to be seen. There are certainly ways it can be done, and that may be one way that MOOCs will make money. (Udacity already does offer a credentialing option, for a fee.)

In designing my course, I tried to optimize for learning in small groups, perhaps five to fifteen at a time. The goal was to build learning communities, within which students could help one another. Since there is no possibility of regular, direct interaction with the instructor (me) and my one TA (Paul), students have to seek help from fellow students. There is no other way. But, on its own, group work is not enough. Learning how to think mathematically (the focus of my course) requires feedback from others, but it needs to include feedback from people already expert in mathematical thinking. This means that, in order to truly succeed, not only do students need to work in groups (at least part of the time), and subject their attempts to the scrutiny of others, some of those interactions have to be with experts.

One original idea I had turned out not to work, though whether through the idea itself being flawed or the naive way we implemented it is not clear to me. That was to ask students at the start of the course to register if they had sufficient knowledge and experience with the course material to act as “Community TAs”, and be so designated in the discussion forums. Though over 600 signed up to play that role, many soon found they did not have sufficient knowledge to perform the task. Fortunately,a relatively small number of sign-ups did have the necessary background, as well as the interpersonal skills to give advice in a supporting, non-threatening way, and they more or less  ensured that the forum discussions met the needs of many students (or so it seems).

Another idea was to assign students to study groups, and use an initial survey to try to identify those with some background knowledge and seed them into the groups. Unfortunately, Coursera does not (yet) have functionality to support the creation and running of groups, apart from the creation of forum threads. So instead, in my first lecture, I suggested to the students that they form their own study groups in whatever way they could.

The first place to do that was, of course, the discussion forums on the course website, which very soon listed several pages of groups. Some used the discussion forum itself to work together, while others migrated offsite to some other location, physical or virtual, with Skype seeming a common medium. Shortly after the course launched, several students discovered GetStudyRoom, a virtual meeting place dedicated to MOOCs, built by a small startup company.

In any event, students quickly found their own solutions. But with students forming groups in so many different ways on different media, there was no way to track how many remained active or how successful they have been.

The study groups listed on the course website show a wide variety of criteria used to bring the groups together. Nationality and location were popular, with groups such as Brazil Study Group, Grupo de Estudo Português, All Students From Asia, and Study Group for Students Located in Karachi, Pakistan. Then there were groups with a more specific focus, such as Musicians, Parents of Homeschooled Children, Older/Retired English Speakers Discussion for Assignment 1, and, two of my favorites, After 8pm (UK time) English speakers with a day job and the delightfully named Just Hanging on Study Group.

The forum has seen a lot of activity: 15,088 posts and 13,622 comments, spread across 2712 different threads, viewed 430,769 times. Though I have been monitoring the forums on an almost daily basis, to maintain an overall sense of how the course is going, it’s clearly not possible to view everything. For the most part I restricted my attention to the posts that garnered a number of up-votes. Students vote posts up and down, and once a post shows 5 or more up-votes, I take that as an indication that the issue may be worth looking at.

The thread with the highest number of up-votes (165) was titled Deadlines way too short. Clearly, the question of deadlines was a hot topic. How, if at all, to respond to such feedback is no easy matter. In a course with tens of thousands of students, even a post with hundreds of up-votes represents just a tiny fraction of the class. Moreover, threads typically include opinions on both sides of an issue.

For instance, in threads about the pace of the course, some students complained that they did not have enough time to complete assignments, and pleaded for more relaxed deadlines, whereas others said they thrived on the pace, which stimulated them to keep on top of the material. For many, an ivy-league MOOC offers the first opportunity to experience an elite university course, and I think some are surprised at the level and pace. (I fact, I did keep the pace down for the first three weeks, but I also do that when I give a transition course in a regular setting, since I know how difficult it is to make that transition from high school math to university level mathematics.)

A common suggestion/request was to simply post the course materials online and let students access them according to their own schedules, much like Khan Academy. This raises a lot of issues about the nature of learning and the role MOOCs can (might? should?) play. But this blog post has already gone on long enough, so I’ll take up that issue next time.

To be continued …

The Crucible: MOOC Planning – Part 8

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, I have survived the initial three weeks of my first MOOC. Though the bulk of the work (and I mean “bulk”) came before the course launched, it has still taken my TA and me a lot of time to keep things ticking over. There are the in-flight corrections of the inevitable errors that occur in a new course, together with the challenges presented by a completely new medium and a buggy, beta release platform, still under very rapid development.

The course website shows 61,846 registered students, but I suspect many of those have long stopped any kind of connection to the course, and another large group are simply watching the lecture videos. The really pleasing figure is that the number of active users last week (week 3) was 19,298. Based on what I hear about other MOOCs, retaining one student in three is a good number.

Both my hands-on TA, Paul, and the course Research Associate, Molly, are graduate students in Stanford’s School of Education, and besides helping me with aspects of the course design, they are approaching the project as an opportunity to carry out research in learning, particularly mathematics learning. Given the massive amount of data a MOOC generates, the education research world can expect to see a series of papers coming from them in the months ahead.

I’m not trained in education research, but some observations are self-evident when you look over the course discussion forums – something I’ve spent a lot of time doing, both to gauge how the course is going and to look for ways to improve it, either by an in-course modification of for a future iteration of the course.

I’ve always felt that the essence of MOOC learning is community building. There is no hope that the “instructor” can do more than orchestrate events. Without regular close contact with the students, the video-recorded lectures and the various course notes and handouts are like firing off a shotgun on a misty Scottish moor. The shot flies out and disperses into the mist, and you just hope some of it hits a target. (I haven’t actually fired a shotgun on a Scottish moor, or anywhere else for that matter, but I’ve seen it on TV and it seems the right metaphor.) With 60,000 (or 20,000) students, I can’t allow myself to respond to a forum post or an email from any single student. I have to rely on the voting procedure (“Like/Dislike”) of the forums to help me decide which questions to address.

This means the student body has to resolve things among themselves. It was fascinating watching the activity on the discussion forums take shape and develop a profile over the first couple of weeks.

One huge benefit for the instructor is the virtual elimination of the potentially disruptive influence – present in almost any class with more than twenty or so students – of the small number of students for whom nothing is good enough. Even in a totally free course, put on by volunteers, for which no college credential is awarded, there were a few early posts of that kind. But in each case the individual was rapidly put in his or her place by replies from other students, and before long stopped posting, and very likely dropped the course.

(An interesting feature of this was that each time it occurred, a number of students emailed me in private – rather than on the public course forum – to say they did not agree with the complainer, and to tell me they were enjoying the course. Clearly, even with the possibility of anonymous forum posts, which Coursera allows, at least for now, some people prefer to keep their communication totally private.)

Of far greater interest, at least to me, was how the student body rapidly split into two camps, based on how they reacted to the course content. As I’ve discussed in earlier posts to this blog, my course is a high-school to university transition course for mathematics. It’s designed to help students make the difficult (and for most of us psychologically challenging) transition from high school mathematics, with its emphasis on learning to follow procedures to solve highly contrived “math problems”, to developing an ability to think logically, numerically, analytically, quantitatively, and algebraically (i.e., in aggregate, mathematically) about novel problems, including often ill-defined or ambiguous real-world problems.

When I give this kind of course to a traditional class of twenty-five or so entering college students, fresh out of high school, the vast majority of them have a really hard time with it. In my MOOC, in contrast, the student body has individuals of all ages, from late teens into their sixties and seventies, with different backgrounds and experiences, and many of them said they found this approach the most stimulating mathematics class they had ever taken. They loved grappling with the inherent ambiguity and open-ended nature of some of the problems.

Our schools (at least in the US), by focusing on one particular aspect of mathematics – the formal, procedural – I think badly shortchange our students. They send them into the world with a fine scalpel, but life in that world requires a fairly diverse toolkit – including WD40 and a large roll of duct tape.

The real world rarely presents us with neat, encapsulated problems that can be solved in ten minutes. Real world problems are messy, ambiguous, ill-defined, and often with internal contradictions. Yes, precise, formal mathematics can be very useful in helping to solve such problems. But of far broader applicability is what I have been calling “mathematical thinking”, the title of my course.

I suspect the students who seemed to take to my course like ducks to water were people well beyond high school, who had discovered for themselves what is involved in solving real problems. Judging by the forum discussions, they are having a blast.

The others, the ones whose experience of mathematics has, I suspect, been almost entirely the familiar, procedural-skills learning of the traditional K-12 math curriculum, keep searching for precision that simply is not there, or (and I’ve been focusing a lot on this in the first three weeks) where the goal is to learn how to develop that precision in the first place.

The process of starting with a messy, real world problem, where we have little more than our intuitions to guide us, and then slowly distilling some precision to help us deal with that problem, is hugely valuable. Indeed, it is the engine that powered (and continues to power) the entire development of our science and our technology. Yet, in our K-12 system we hardly ever help students to learn how to do that.

Done well, the activities of the traditional math class can be great fun. I certainly found it so, and have spent a large part of my life enjoying the challenges of pure mathematics research. But a lot of that fun comes from working within the precise definitions and clear rules of engagement of the discipline.  To me mathematics was chess on steroids. I loved it. Still do, for that matter. But relatively few citizens are interested in making  a career in mathematics. An education system that derives its goals from the ivory-towered pursuit of pure mathematics (and I use that phrase in an absolutely non-denigrating way, knowing full well how important it is to society and to our culture that those ivory towers exist) does not well serve the majority of students.

It requires some experience and sophistication in mathematics to see how skill in abstract, pure reasoning plays an important role in dealing with the more messy issues of the real world. There is an onus on those of us in the math ed community  to help others to appreciate the benefits available to them by way of improved mathematical ability.

As I have followed the forum discussions in my MOOC, I have started to wonder if one thing that MOOCs can give to mathematics higher education in spades is a mechanism to provide a real bridge between K-12 education and life in the world that follows. By coming together in a large, albeit virtual community, the precision-seeking individuals who want clear rules and guidelines to follow find themselves side-by-side (actually, keyboard-to-keyboard) with others (perhaps with weak formal mathematics skills) more used to approaching open-ended, novel problems of the kind the real world throws up all the time. If so, that would make the MOOC a powerful crucible that would benefit both groups, and thus society at large.

To be continued …

Liftoff: MOOC planning – Part 7

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

It’s been three weeks since I last posted to this blog. The reason for the delay is I was swamped getting everything ready for the launch of my course four days ago, on Monday of this week. As of first thing this morning there are 57,592 students enrolled in the class.

The course was featured in an article on MOOCs in USA Today. It was a good article, but like every other news report I’ve seen on MOOCs, the focus was on the video lectures. Those certainly take a fair amount of time on the part of the instructor (me, in this case), and are perhaps the most visible feature of a MOOC, just as the classroom lecture is the most visible part of many on-campus courses.

For some subjects, lectures, either in-person or on a computer screen, may be a major part of a course. But for conceptual mathematics, which is what my course is about, they are one of the least important features.

Learning to think mathematically is like learning to swim, to ride a bicycle, to ski, to play golf, or to play a musical instrument. You can probably get some idea by having someone explain it to you, but you won’t learn how to do it that way. The key words in that last clause are “learn” and “do”. There is really only one way to learn how to do something, and that is by doing it. Or, to put it more bluntly, the only way to achieve mastery is by repeated failure. You keep trying until you get it. The one thing that can help is having someone who already has mastery look at your attempts and give you constructive feedback.

In fact, failing in attempting to do something new isn’t really failure at all in the sense the word is usually used. Rather, a failed attempt is a step towards eventual success. Edison put it well when asked how he felt about his many failures to make a light bulb. He replied, “I have not failed. I’ve just found 10,000 ways that don’t work.”

After just one week of my course, I’ve seen a lot of learning going on, but it wasn’t in the lectures. Even if I’d been able to see each student watching the lecture, I would not have seen much learning going on, if any.  Rather, the learning I saw was on the discussion forums, primarily the ones focused on the assignments I gave out after each lecture. As I explained to the students, the course assignments and the associated forum discussions are the heart of the course.

So what is my part in all of this? Well, first of all, I have to admit I am uncomfortable with the title “instructor,” since that does not really reflect my role, but it’s the name society generally uses. “Course designer, conductor (as for an orchestra), and exemplar” would be a much better reflection of what I have been doing. Once the course was designed, the lectures recorded, and all the ancillary materials prepared, my task was to set the agenda, provide motivation and context for the various topics, and give examples of mathematical thinking.

The rest is up to the students. It has to be. (At least, I don’t know of any other way to learn how to think mathematically.) To be sure, in a physical class, the instructor (and or the TAs) can interact with the students, and (if it occurs) that can be a huge factor. But that simply helps the students learn by repeated failure, it does not eliminate the need for that learning-by-trying-and-failing process. Let’s face it, if you are not failing at something, you have already learned it, and should move on to the next step or topic. (With understanding, once you get it, you don’t need to practice!)

In a MOOC, that regular contact with the instructor and or the TAs is missing, of course. That means the students have to rely on one another for feedback. This is where the Coursera platform delivers. Here are some recent stats from my course website:

Total Registered Users 57592
Active Users Last Week 32123

Video Lectures

Total Streaming Views 77415
Total Downloads 19491
# Unique users watching videos 21712

Discussion Forums

Total Threads 641
Total Posts 5414
Total Comments 3823
Total Views 119489

Though I’d like to see a lot more students posting to the forums, with almost 120,000 views (after just one lecture and one course assignment!), it’s clear that that is where a lot of the action is.

As I surmised in an early blog-post, I don’t think it was the widespread availability of video technology and sites like YouTube that set the scene for MOOCs. To my mind, Facebook opened the floodgates, by making digitally-mediated social networking a mainstream human activity. (I’d better add Skype, since there are already several Skype-based study groups for my course. And of course, students who live close together can do it the old-fashioned way, by getting together in person to work through the assignments.)

One feature of the course that did not surprise me was the sense of feeling lost some students reported (and I’m sure many more felt), in some cases maybe being accompanied by panic. For most students, not only does my course present a side of mathematics they have never seen before (the world of the professional mathematicians), on top of that, none of the strategies they were taught to succeed in high-school math work any more.

Because the focus of the course is on mathematical thinking, I can’t provide the students with a list of rules to follow, templates to recognize, or procedures to follow. The whole point is to help them develop the ability to solve novel problems for which no  rules are known.

Of course, at this stage, the problems I give them are ones that have been solved long ago, and which have been shown to provide good learning material. But to the student, they are new, and that’s what matters in terms of learning. Unless, of course, they look for the solution on the Web, which defeats the whole purpose. But in a voluntary course where the focus is on process, not “getting answers,” and which provides no college credential, I hope that does not occur. In fact, one of the things that attracted me to free MOOCs was that the students would enroll because they wanted to learn, not because they were forced to learn or simply in need of a diploma. (We mathematicians get a lot of students like that! But we get paid to teach those classes. So far, no one is paying MOOC faculty for their efforts.)

Most US students have a particularly hard time with this “there are no templates” approach, because of the way mathematics is typically taught in American schools.  Instead of helping students to learn mathematics by figuring it out for themselves, teachers frequently begin by providing instruction and following it up with examples. Michael Pershan has a nice summary of this on YouTube. (His initial focus is on Khan Academy, but Khan is simply providing a service that is molded on, and fits into, the US system. The crucial issue Pershan’s video addresses is the system.)

The pros and cons of the two approaches, instruction based or guided discovery, remains a topic of debate in this country, but in the case of my course, there can be no debate. The goal is to develop the ability to encounter a novel problem and eventually be able to figure it out. Providing instruction in such a course would be like giving a golf cart to someone who wants to walk to lose weight! It might get them to their destination with less effort, but it would defeat the real goal.

Having thought at length about how to structure this first version of the course, and played around with some approaches, I ended up, as I thought I probably would, going minimal.  Virtually no instruction, and what little there is presented as examples of mathematical thinking in action, not by way of a carefully planned lesson. I was pretty sure I’d do that, because that’s how I’ve always conducted classes where the goal is student learning (as opposed to passing a standardized test).

There are a number of studies pointing out the dangers of over-planned lessons, one of the most famous and influential being Alan Schoenfeld’s 1988 paper in Educational Psychologist (Vol 23(2), 1988), When Good Teaching Leads to Bad Results: The Disasters of “Well Taught” Mathematics Courses. Still, as I said, I did play around with alternatives, since I was worried how students would fare without having regular access to the instructor and the TAs. I may have to re-visit those other approaches, if things go worse this time than I fear.

But this time round, what the student gets is as close a simulation as I can produce of sitting next to me as I work through the material. The result is not perfect. It’s not meant to be. There are minor errors in there. It’s meant to provide an example of how a professional mathematician sets about things. Definitely not intended as something to be perceived as an entry in an instruction manual.

After those work sessions were video-recorded, they were edited, of course, but only to cut out pauses while I thought, and to speed up the handwriting in places. I found that on a screen, watching the handwriting in real time looked painfully slow, and rapidly became irritating, particularly in places where I had to write out an entire sentence. So I took a leaf out of Vi Hart‘s wonderful repertoire. The speed ramping ended up being the only place that modern digital technology actually impinged on the lecture. Everywhere else it merely provided a medium. The approach would be familiar to Euclid if he were somehow to come back and take (or give) the class.

To be continued …

You may be interested in two recent videos featuring the founders of the two Stanford MOOC platforms that started the current explosion of interest in these courses. In one, Sebastian Thrun talks about Udacity. In the other Daphne Koller discusses the creation of Coursera.

The “C” in “MOOC”: MOOC planning – Part 6

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

A few days ago, I went into our campus TV studio with the two course assistants for my upcoming MOOC, to record a short video introducing them to the students.  The students will see a lot of me, but my two TAs will be working behind the scenes, and the students will encounter them only through their contributions to the forum discussions. The videos were intended to compensate for that lack of human contact.

During the course of recording that video, the three of us got into a discussion about our backgrounds, our motives in giving the MOOC, and our views on mathematics, science, education, and our expectations for the MOOC format. The camera was rolling all the time, and we were able to select a few parts of that discussion and create a second video that I think will help our students understand some of our thinking in putting this course together.  I posted copies of both videos on YouTube.  (They are much lower resolution than the videos the registered students will see on the course website when it goes live on September 17 — the “first day of classes”.) I think the two videos provide an insight into our thinking as we designed this course.

The fact that the current round of MOOCs have a “first day of class” at all has been a matter of some debate. The C in MOOC stands for “course”, but is this the best way to go?  For example, see this blogpost from a graduate student at Berkeley, who argues for a more open framework of learning resources. He makes some good points that all of us involved in this initiative have thought about and discussed, but I’m not sure the kind of thing he advocates can work for disciplines and subjects that depend heavily on student-faculty and student-student interaction, as mine does.

In fact, I’m not sure the MOOC will work sufficiently well at all in such cases; this is very much an experiment that I anticipate will continue for several years before we get good answers either way. For the first iteration, it makes sense to start with a model we know does work. And important (we think!) elements of that model are, to repeat Sebastian Thrun’s list, as quoted in the Berkeley student’s blog: admissions, lectures, peer interaction, professor interaction, problem-solving, assignments, exams, deadlines, and certification. To use the mnemonic I coined earlier in this series, our basic design principle is WYSIWOSG: What You See Is What Our Students Get.

Since these courses are free, we can, of course, do a lot of A/B testing in future years, to see which of these truly are crucial, which can be changed and how, and which can be dropped. I suspect the answers we get will vary from discipline to discipline, and possibly from course to course.

All of us involved in this MOOC movement are trying to find out the best way that works for our particular discipline and is consistent with our own style as instructors. As I indicated in Part 4 of this diary, I think it makes sense to begin by trying to implement in a MOOC as much of our tried-and-trusted classroom-based teaching as we can (as Thrun did with Udacity), and then iterating in the light of what we learn.

This is why, instead of hiring a mathematics graduate student to TA my course, which is what I would have done for an on campus class, I brought onto my team two graduate students from Stanford’s School of Education with several years of experience in learning design and the use of technology in education. In addition to helping me with the design and running of the course, they will conduct research into the course’s efficacy and try to understand how learning occurs in a MOOC. (Other than a brief, non-compulsory questionnaire at the start and finish of the course, all their research will be based on data gathered on the Coursera course platform and human monitoring of the forum discussions. One huge benefit of MOOCs is that they facilitate Big Data research.)

It’s live beta, folks.

To be continued …

My first big mistake: MOOC planning – Part 5

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Wow! With three weeks to go to the course launch, I checked the course registrations for the first time. So far, almost 35,000 students have signed up. In theory, I knew this would happen; that’s been everyone else’s experience with MOOCs. But when you actually see that kind of figure on the stats page for your own course, it makes a big impression.

Then I made my first big mistake. I sent out a welcome email to the students who had already registered. That part was not the mistake. Of course I’d want to welcome the students! Nor was my error to mention this blog in my email. It does, after all, provide students with some background on my thinking behind the course and what I want to achieve. My mistake was not closing comments on this blog before I sent out the email.

I was online when the first few comments started coming in, and as usual I responded to them. Then the flood began. I managed to close comments before the WordPress servers shut me down.  :-)

So, sorry to all those who wrote in to this blog and did not get a reply. The Coursera platform, which is desgned to handle classes of many thousands of students, offers opportunities to comment and exchange ideas, with a mechanism to bring to the attention of me and my teaching assistants any discussion thread that is generating a lot of interest. That will be available once the course starts.

I wonder what my next mistake will be.

To be continued …

Khan Academy Meets Vi Hart: MOOC planning – Part 3

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

The ideal way to learn mathematical thinking (and a great many other things that involve understanding, not just doing) is in a small physical group with an expert. That provides frequent opportunities to interact one-on-one with the expert, during which the expert can observe you work in real time (on paper or at a board) and can give you direct feedback on written work you have done and handed in for evaluation. It also provides frequent opportunities to discuss what is being learned with other students at the same stage of their learning, sometimes with the expert present, other times with the expert absent.

Sometimes, the expert will provide instruction. Though there have been successful instances of mathematics professors who largely avoid instruction (R L Moore being the most notable example), most of us (i.e., university mathematics educators) find that instruction has a valuable place in mathematics education. But many of us view it as just one part of mathematics education.

Anyone who has experienced highly interactive mathematics teaching will know how different it is from mere instruction, and how much more effective. I wrote about this last March in my Devlin’s Angle column for the MAA. Unfortunately, it seems clear that a great many Americans have never experienced good mathematics teaching. If they had, you would not have thousands of Khan Academy users (including famous figures such as Bill Gates) declaring Salman Khan is the best math teacher ever. You can say a number of good things about Sal Khan (I am going to say some of them in just a moment), but being a great math teacher is not one of them. To say that he is, simply reflects on the miserable math ed diet that many millions of American have been fed, for whom Khan Academy offers something far better than they were ever exposed to.

I bring up Khan Academy for a couple of reasons, one being that it set the stage for the MOOC explosion. Indeed, former Stanford CS professor Sebastian Thrun stated publicly last January that it was Khan Academy that inspired him to give his first MOOC in fall 2011, and then to leave Stanford and launch his own MOOC service Udacity at the start of this year.

It’s not merely the wide reach that Khan Academy demonstrated. As I discussed in a recent article for the MAA, Sal Khan managed to tap into the power of the Web medium to achieve a critical element of good teaching that not all teachers can offer: a strong teacher-student bond. Moreover, he did so using just his voice and the electronic trail of a digital pen on the viewer’s computer screen. Yes, some of the math is wrong, and the pedagogy is so poor, experienced teachers tear their hair out, but the very success of Khan Academy shows how important is the teacher-student connection.

Khan Academy is not a MOOC, of course, but it does provide a model for online mathematics instruction. In starting to plan my MOOC, I began by trying variants of Sal’s approach for the instructional part. Like him, I have a voice that works on the radio (or a Web audio channel) — an accident of birth — which makes such an approach feasible.

I soon concluded that his approach would not work. It is fine for presenting short instructional mini-lectures on how to follow a particular mathematical procedure, but it is woefully impoverished for trying to help students understand a mathematics idea or a proof, and to form the right mental concepts. For that, the huge importance in mathematics teaching of physical gestures, in particular the hand(s), cannot be ignored.

There is an old challenge in which you ask someone to describe a helix while keeping their hands clasped firmly behind their back. (Try it!) But it’s not just helices. Explaining almost any mathematical concept without using at the very least hand and arm gestures, and in many cases full body motion, is difficult if not impossible. There is masses written about this topic, based on many years of research. For example, take a look at this summary, or this one, or this forthcoming book. Or Google on the terms “mathematics + learning + hand + gesture” or variants thereof to see a lot more.

Since MOOC students access the material on a wide range of devices, with different screen sizes, I felt that a full body recording of me working at (and in front of) a blackboard or whiteboard would not be ideal. Besides, I love the sense of intimacy Khan Academy offers. You get a strong sense of sitting next to a friendly relative who is personally instructing you. I wanted to create that environment.

But trying to follow an explanation of a mathematical concept or proof Khan-style, where the visual channel consists only of a digital pen trace, was impossible — at least, it was given my educational style. At the very least, I needed my writing hand to direct the student’s focus. The simplest way to achieve that was to have a video camera mounted above my desk and record me working through the material in the time-honored fashion of paper-and-pencil. That seemed to work.

Having decided on the basic modality, the next issue was one of style and tone. After playing with some variants of the basic format, I came down in favor of a very informal look, where I simply slap down a sheet of paper on the desk in front of me and the student, and work through the material. (Marking the exact position of the paper on the desk and letting it totally fill the screen looked too artificial — though at this stage the issue was largely one of taste, and this is a decision I may change based on the experience I get from this first course. I did have to tape down the paper, but the initial placement was fairly casual, and the taping was sufficiently loose that the paper could still move a little — it takes effort to create “informality” on video.)

To counter the inevitable sense of frustration when watching a pen write something out in real time, I decided to speed up a lot of the writing during the video editing phase. (Though not to the speed of the wonderful Vi Hart, whose purpose is informative entertainment.) So at that stage I found myself with a “Sal Khan meets Vi Hart” look. A great place to start, given the success both have achieved!

For standalone Web instruction, that would likely be enough, but a MOOC involves a lot more. It is, after all, a course — a structured experience over several weeks, with a professor. Regular connection to the instructor is important — at least, I think it is. (It was for me when I was a student.) To achieve that “human connection,” many of my Stanford colleagues who have given MOOCs have put a small head-and-shoulders video of themselves speaking in one corner of the screen, as the material being discussed occupies the rest of the display. I tried that, and found it did not work for me, with my material. The face was a distraction. I wanted to keep as much of the Khan Academy sense as possible — you don’t ignore success unless there is good reason! So I opted to keep video of me separate from the hand-writing part.

I’ve posted a short sample from Lecture 1 on YouTube. Given the low resolution of YouTube video encoding, this does not display well in terms of content, but the Coursera platform uses far higher resolution video.

I doubt much of this material will survive to a second iteration of the course next year. At the very least, I’d want to go back and pay more attention to lighting and audio levels and consistency.  But it does have the overall look and feel I was trying to achieve. This is live beta, folks.

But as I have already indicated in this blog series, I don’t see the video lectures as the heart of the course. They merely set the agenda for learning. The real learning takes place elsewhere. I’ll turn to that topic in a future post.

Meanwhile, my Stanford MOOC Introduction to Mathematical Thinking is scheduled to begin on September 17 on Coursera. If you want to do some preliminary reading, there is my low-cost course textbook by the same name. Though written to align to the course, it is not required in order to complete the course. (Indeed, I noted  above that I see MOOCs as replacing textbooks — though some MOOCs may have required textbooks, so it would be unwise to predict the imminent death of the printed textbook!)

To be continued …

NOTE: I mentioned Khan Academy to indicate its role in the MOOC explosion and acknowledge its role in guiding the design of the instructional videos in my MOOC. But the focus of this blog is on MOOCs in general and mathematics MOOCs in particular. Comments discussing the merits or demerits of Khan Academy are off topic and hence will not be published; there are many other venues for such discussions.


I'm Dr. Keith Devlin, a mathematician at Stanford University. I gave my first free, open, online math course in fall 2012, and have been offering it twice a year since then. This blog chronicles my experiences as they happen.

Twitter Updates

Error: Twitter did not respond. Please wait a few minutes and refresh this page.

New Book 2012

New book 2011

New e-book 2011

New book 2011

November 2014
M T W T F S S
« Sep    
 12
3456789
10111213141516
17181920212223
24252627282930

Follow

Get every new post delivered to your Inbox.

Join 573 other followers

%d bloggers like this: