Posts Tagged 'online education'



Final Lecture: MOOC Planning – Part 9

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

I gave my last lecture of the course yesterday (discounting the tutorial session that will go out next week), and we are now starting a two week exam period.

“Giving” a lecture means the video becomes available for streaming. For logistic reasons (high among them, my survival and continued sanity — assuming anyone who organizes and gives a MOOC, for no payment, is sane), I recorded all the lectures weeks ago, well before the course started.  The weekly tutorial sessions come the closest to being live. I record them one or two days before posting, so I can use them to respond to issues raised in the online course discussion forum.

The initial course enrollment of 63,649 has dropped to 11,848 individuals that the platform says are still active on the site. At around 20%, that’s pretty high by current MOOC standards, though I don’t know whether that is something to be pleased about, since  it’s not at all clear what the right definition of “success” is for a MOOC.

Some might argue that 20% completion indicates that the standards are too low. I don’t think that’s true for my course. Completion does, after all, simply mean that a student is still engaged. The degree to which they have mastered the material is unclear. So having 80% drop out could mean the standard is too high.

In my case, I did not set out to achieve any particular completion rate; rather I adopted a WYSIWOSG approach — “What You See Is What Our Students Get.” I offered a MOOC that is essentially the first half of a ten week course I’ve given at many universities over the years, including Stanford. That meant my students would experience a Stanford-level course. But they would not be subject to passing a Stanford-level exam.

In fact, I could not offer anything close to a Stanford-exam experience. There is a Final Exam, and it has some challenging questions, but it is not taken under controlled, supervised conditions. Moreover,  since it involves constructing proofs, it cannot be machine graded, and thus has to be graded by other students, using a crowd sourcing method (Calibrated Peer Review). That put a significant limitation on the kinds of exam questions I could ask. On top of that, the grading is done by as many different people as there are students, and I assume most of them are not expert mathematicians. As a result, it’s at most a “better-than-nothing” solution. Would any of us want to be treated by a doctor whose final exam had been peer graded (only) by fellow students, even if the exam and the grading had been carried out under strictly controlled conditions?

On the other hand, looking at and attempting to evaluate the work of fellow students is a powerful learning experience, so if you view MOOCs as vehicles for learning, rather than a route to a qualification, then peer evaluation has a lot to be said for it. Traditional universities offer both learning and qualifications. MOOCs currently provide the former. Whether they eventually offer the latter as well remains to be seen. There are certainly ways it can be done, and that may be one way that MOOCs will make money. (Udacity already does offer a credentialing option, for a fee.)

In designing my course, I tried to optimize for learning in small groups, perhaps five to fifteen at a time. The goal was to build learning communities, within which students could help one another. Since there is no possibility of regular, direct interaction with the instructor (me) and my one TA (Paul), students have to seek help from fellow students. There is no other way. But, on its own, group work is not enough. Learning how to think mathematically (the focus of my course) requires feedback from others, but it needs to include feedback from people already expert in mathematical thinking. This means that, in order to truly succeed, not only do students need to work in groups (at least part of the time), and subject their attempts to the scrutiny of others, some of those interactions have to be with experts.

One original idea I had turned out not to work, though whether through the idea itself being flawed or the naive way we implemented it is not clear to me. That was to ask students at the start of the course to register if they had sufficient knowledge and experience with the course material to act as “Community TAs”, and be so designated in the discussion forums. Though over 600 signed up to play that role, many soon found they did not have sufficient knowledge to perform the task. Fortunately,a relatively small number of sign-ups did have the necessary background, as well as the interpersonal skills to give advice in a supporting, non-threatening way, and they more or less  ensured that the forum discussions met the needs of many students (or so it seems).

Another idea was to assign students to study groups, and use an initial survey to try to identify those with some background knowledge and seed them into the groups. Unfortunately, Coursera does not (yet) have functionality to support the creation and running of groups, apart from the creation of forum threads. So instead, in my first lecture, I suggested to the students that they form their own study groups in whatever way they could.

The first place to do that was, of course, the discussion forums on the course website, which very soon listed several pages of groups. Some used the discussion forum itself to work together, while others migrated offsite to some other location, physical or virtual, with Skype seeming a common medium. Shortly after the course launched, several students discovered GetStudyRoom, a virtual meeting place dedicated to MOOCs, built by a small startup company.

In any event, students quickly found their own solutions. But with students forming groups in so many different ways on different media, there was no way to track how many remained active or how successful they have been.

The study groups listed on the course website show a wide variety of criteria used to bring the groups together. Nationality and location were popular, with groups such as Brazil Study Group, Grupo de Estudo Português, All Students From Asia, and Study Group for Students Located in Karachi, Pakistan. Then there were groups with a more specific focus, such as Musicians, Parents of Homeschooled Children, Older/Retired English Speakers Discussion for Assignment 1, and, two of my favorites, After 8pm (UK time) English speakers with a day job and the delightfully named Just Hanging on Study Group.

The forum has seen a lot of activity: 15,088 posts and 13,622 comments, spread across 2712 different threads, viewed 430,769 times. Though I have been monitoring the forums on an almost daily basis, to maintain an overall sense of how the course is going, it’s clearly not possible to view everything. For the most part I restricted my attention to the posts that garnered a number of up-votes. Students vote posts up and down, and once a post shows 5 or more up-votes, I take that as an indication that the issue may be worth looking at.

The thread with the highest number of up-votes (165) was titled Deadlines way too short. Clearly, the question of deadlines was a hot topic. How, if at all, to respond to such feedback is no easy matter. In a course with tens of thousands of students, even a post with hundreds of up-votes represents just a tiny fraction of the class. Moreover, threads typically include opinions on both sides of an issue.

For instance, in threads about the pace of the course, some students complained that they did not have enough time to complete assignments, and pleaded for more relaxed deadlines, whereas others said they thrived on the pace, which stimulated them to keep on top of the material. For many, an ivy-league MOOC offers the first opportunity to experience an elite university course, and I think some are surprised at the level and pace. (I fact, I did keep the pace down for the first three weeks, but I also do that when I give a transition course in a regular setting, since I know how difficult it is to make that transition from high school math to university level mathematics.)

A common suggestion/request was to simply post the course materials online and let students access them according to their own schedules, much like Khan Academy. This raises a lot of issues about the nature of learning and the role MOOCs can (might? should?) play. But this blog post has already gone on long enough, so I’ll take up that issue next time.

To be continued …

The Crucible: MOOC Planning – Part 8

A real-time chronicle of a seasoned professor embarking on his first massively open online course.

Well, I have survived the initial three weeks of my first MOOC. Though the bulk of the work (and I mean “bulk”) came before the course launched, it has still taken my TA and me a lot of time to keep things ticking over. There are the in-flight corrections of the inevitable errors that occur in a new course, together with the challenges presented by a completely new medium and a buggy, beta release platform, still under very rapid development.

The course website shows 61,846 registered students, but I suspect many of those have long stopped any kind of connection to the course, and another large group are simply watching the lecture videos. The really pleasing figure is that the number of active users last week (week 3) was 19,298. Based on what I hear about other MOOCs, retaining one student in three is a good number.

Both my hands-on TA, Paul, and the course Research Associate, Molly, are graduate students in Stanford’s School of Education, and besides helping me with aspects of the course design, they are approaching the project as an opportunity to carry out research in learning, particularly mathematics learning. Given the massive amount of data a MOOC generates, the education research world can expect to see a series of papers coming from them in the months ahead.

I’m not trained in education research, but some observations are self-evident when you look over the course discussion forums – something I’ve spent a lot of time doing, both to gauge how the course is going and to look for ways to improve it, either by an in-course modification of for a future iteration of the course.

I’ve always felt that the essence of MOOC learning is community building. There is no hope that the “instructor” can do more than orchestrate events. Without regular close contact with the students, the video-recorded lectures and the various course notes and handouts are like firing off a shotgun on a misty Scottish moor. The shot flies out and disperses into the mist, and you just hope some of it hits a target. (I haven’t actually fired a shotgun on a Scottish moor, or anywhere else for that matter, but I’ve seen it on TV and it seems the right metaphor.) With 60,000 (or 20,000) students, I can’t allow myself to respond to a forum post or an email from any single student. I have to rely on the voting procedure (“Like/Dislike”) of the forums to help me decide which questions to address.

This means the student body has to resolve things among themselves. It was fascinating watching the activity on the discussion forums take shape and develop a profile over the first couple of weeks.

One huge benefit for the instructor is the virtual elimination of the potentially disruptive influence – present in almost any class with more than twenty or so students – of the small number of students for whom nothing is good enough. Even in a totally free course, put on by volunteers, for which no college credential is awarded, there were a few early posts of that kind. But in each case the individual was rapidly put in his or her place by replies from other students, and before long stopped posting, and very likely dropped the course.

(An interesting feature of this was that each time it occurred, a number of students emailed me in private – rather than on the public course forum – to say they did not agree with the complainer, and to tell me they were enjoying the course. Clearly, even with the possibility of anonymous forum posts, which Coursera allows, at least for now, some people prefer to keep their communication totally private.)

Of far greater interest, at least to me, was how the student body rapidly split into two camps, based on how they reacted to the course content. As I’ve discussed in earlier posts to this blog, my course is a high-school to university transition course for mathematics. It’s designed to help students make the difficult (and for most of us psychologically challenging) transition from high school mathematics, with its emphasis on learning to follow procedures to solve highly contrived “math problems”, to developing an ability to think logically, numerically, analytically, quantitatively, and algebraically (i.e., in aggregate, mathematically) about novel problems, including often ill-defined or ambiguous real-world problems.

When I give this kind of course to a traditional class of twenty-five or so entering college students, fresh out of high school, the vast majority of them have a really hard time with it. In my MOOC, in contrast, the student body has individuals of all ages, from late teens into their sixties and seventies, with different backgrounds and experiences, and many of them said they found this approach the most stimulating mathematics class they had ever taken. They loved grappling with the inherent ambiguity and open-ended nature of some of the problems.

Our schools (at least in the US), by focusing on one particular aspect of mathematics – the formal, procedural – I think badly shortchange our students. They send them into the world with a fine scalpel, but life in that world requires a fairly diverse toolkit – including WD40 and a large roll of duct tape.

The real world rarely presents us with neat, encapsulated problems that can be solved in ten minutes. Real world problems are messy, ambiguous, ill-defined, and often with internal contradictions. Yes, precise, formal mathematics can be very useful in helping to solve such problems. But of far broader applicability is what I have been calling “mathematical thinking”, the title of my course.

I suspect the students who seemed to take to my course like ducks to water were people well beyond high school, who had discovered for themselves what is involved in solving real problems. Judging by the forum discussions, they are having a blast.

The others, the ones whose experience of mathematics has, I suspect, been almost entirely the familiar, procedural-skills learning of the traditional K-12 math curriculum, keep searching for precision that simply is not there, or (and I’ve been focusing a lot on this in the first three weeks) where the goal is to learn how to develop that precision in the first place.

The process of starting with a messy, real world problem, where we have little more than our intuitions to guide us, and then slowly distilling some precision to help us deal with that problem, is hugely valuable. Indeed, it is the engine that powered (and continues to power) the entire development of our science and our technology. Yet, in our K-12 system we hardly ever help students to learn how to do that.

Done well, the activities of the traditional math class can be great fun. I certainly found it so, and have spent a large part of my life enjoying the challenges of pure mathematics research. But a lot of that fun comes from working within the precise definitions and clear rules of engagement of the discipline.  To me mathematics was chess on steroids. I loved it. Still do, for that matter. But relatively few citizens are interested in making  a career in mathematics. An education system that derives its goals from the ivory-towered pursuit of pure mathematics (and I use that phrase in an absolutely non-denigrating way, knowing full well how important it is to society and to our culture that those ivory towers exist) does not well serve the majority of students.

It requires some experience and sophistication in mathematics to see how skill in abstract, pure reasoning plays an important role in dealing with the more messy issues of the real world. There is an onus on those of us in the math ed community  to help others to appreciate the benefits available to them by way of improved mathematical ability.

As I have followed the forum discussions in my MOOC, I have started to wonder if one thing that MOOCs can give to mathematics higher education in spades is a mechanism to provide a real bridge between K-12 education and life in the world that follows. By coming together in a large, albeit virtual community, the precision-seeking individuals who want clear rules and guidelines to follow find themselves side-by-side (actually, keyboard-to-keyboard) with others (perhaps with weak formal mathematics skills) more used to approaching open-ended, novel problems of the kind the real world throws up all the time. If so, that would make the MOOC a powerful crucible that would benefit both groups, and thus society at large.

To be continued …

Help wanted!

Why am I doing this? Attempting to give a five-week, school-to-university transition course to possibly thousands of students on the Web, I mean.

I always took my teaching seriously. (When I started out university teaching in the 1970s, that was actually not a requirement for faculty; the focus was all on research. My initial appointment in the UK was as a “Lecturer”. Along with the US title of “Instructor”, those names reflected the then-expectation of what the job entailed as far as teaching was concerned.) In many years of university teaching, I always felt that as the number of students increased beyond twenty or so, the quality of the learning fell significantly. Clearly, I am not referring to lecturing — that is, providing instruction, where the students are essentially passive receivers of information. That can clearly be scaled indefinitely, through videos, and arguably that is what textbooks have always done. What can’t be scaled, is the interaction between the professor and the students — which is where a lot of the real learning takes place.

I discussed the distinction between instruction and good, interactive teaching in my March Devlin’s Angle column for the MAA. From what I read and hear all the time, I suspect that many people in the US have never experienced anything beyond instruction, at least when it comes to their mathematics education. Providing mathematics instruction (and nothing more) is like trying to eliminate starvation by providing people with fish. That alleviates the immediate hunger, but it is not a long term solution, and moreover can create a dependency on others. A far better solution is to show people how to catch fish for themselves. That is what good teaching tries to do, by trying to help students learn to think for themselves.

Mathematics is a mental activity. It is something you do. Like all activities, doing it takes effort and it makes you tired. The best way to learn how to do something is to do it. Riding a bicycle, driving a car, playing golf or tennis, skiing, playing a musical instrument, playing chess, and so on, you didn’t learn them by sitting in a classroom, listening to someone provide instruction. Of course, instruction is valuable, but only when it accompanies learning-by-doing, and is provided to the learner on demand, based on that learner’s specific needs at that instant, when it makes sense and is most readily absorbed.

A good teacher, like a good music instructor or athletics coach, begins by identifying what the student knows and can do, and then builds on that. A personal tutor can provide a complete education that way, though besides being inefficient in terms of the utilization of human expertise, one-on-one instruction suffers the significant loss of collaborative work with a small group of peers. More optimal, in my view, an experienced classroom teacher can do wonders with a class of twenty or so, split into groups of four or five for periods of collaborative work.

But with more than twenty, the dynamics change; the teacher can no longer devote sufficient time to each individual and to each group.

In my later career, when I was able to set my own class limits, I always capped at twenty (though I occasionally relented and let the number creep up by one or two, when desperate, math-requirement-short seniors pleaded to be allowed in.) So, coming back to my opening question, why on earth did I decide to try offering an online course that could attract many thousands of students, none of whom I would meet in person?

The answer was a suspicion that, with a suitable re-assessment of the goals of the course, together with a little social engineering, a different dynamic could take over. Talking to some of the Stanford professors who had given, or were giving, MOOCs, provided some anecdotal confirmation of that suspicion. So I stepped forward and volunteered to offer a five-week “transition course” this coming fall.

The purpose of transition courses is to introduce students to mathematical thinking. In the high school mathematics class, the emphasis is on mastery of procedures for solving problems. As many students discover, and as many teachers instruct them, an effective way to succeed is to approach a new problem by looking for a template — a worked example in a textbook, or these days presented on a YouTube video — and then just changing the numbers. (That is actually a valuable skill in itself, but that’s another topic.) University mathematics, on the other hand — at least the mathematics taught at university to future math and science majors — has a different goal: Learning how to think like a mathematician. And that is something most of us initially find extremely hard, and very frustrating. I’ll elaborate in future postings, but for anyone unfamiliar with the transition problem, let me give an analogy.

If we compare mathematics with the automotive world, school math corresponds to learning to drive, whereas in the automotive equivalent to college math is learning how a car works, how to maintain and repair it, and, if a student pursues the subject far enough, how to design and build their own car.

I was one of the early pioneers of transition courses back in the late 1970s, and wrote one of the first companion books, Sets, Functions, and Logic. (It was written for the UK market, but it did make it into a US edition, though many American students, used to full-service textbooks, found it hard going.) So it was a natural for me to see if, and how, the teaching of such material could be ported to the Web as a MOOC.

The benefits of doing so would, of course, be significant. Not least, high school students could attempt it prior to going to college, and college frosch taking a (physical) transition course would have a secondary source for what many find an extremely difficult transition.

The particularly fascinating part to me, as a professor, is figuring out how to take a learning experience that works in a small-group setting on a campus, and create a functionally equivalent experience online. Note that I said “create a functionally equivalent experience;” I did not say “replicate the classroom experience.”

By far the greatest problem is how to provide the personal, expert feedback that is essential to good mathematics learning. Web delivery is fine for providing instruction, but that is just a part of learning, and a minor part at that, as I discussed in that March Devlin’s Angle I referred to earlier. Taking stock of the goal and the available resources, however, there were some hopeful signs.

First, the whole MOOC concept finally took off late last year (with Sebastian Thrun’s Stanford AI course) largely because Stanford and the now independent spinoff company Coursera built innovative new platforms. (Just last week, MIT and Harvard announced that they too were launching their own platform, edX.) Listening to some of my Stanford colleagues describe their experiences giving their first-generation MOOCs, I began to see the opportunities the new platforms (which are still under development) offer.
I’ll examine some of the affordances the new learning medium provides in future postings. (I’m still learning myself.) In the meantime, I need to assemble a small army of volunteers. This is where I’ll need help — possibly your help.
One of the things we’ve learned already about MOOCs, is that a key component is the creation of a strong online community. Learning is all about human interaction. The technology just provides the medium for that interaction. In offering my math transition MOOC at the start of the fall term, when many colleges and universities offer their own transition course, I am inviting any instructor who will be giving such a course, together with their students, to join me and my MOOC students online, making interaction with other students around the world a part of a much larger learning community.
In my May Devlin’s Angle post, I put out a first call for involvement of my fellow MAA members. Here, in summary, is what I wrote there.

I’m going to make my course just five weeks long, starting in early October. By incorporating participation in my Stanford course as part of your students’ learning experience, everyone could benefit. For one thing, your students are likely to be inspired by being part of an educational revolution that for millions of less privileged people around the globe can quite literally be life changing.

Because they will be supported by being part of a physical learning community, with the personal support of you, their instructor, your students will be highly empowered, privileged members of that online community. They can take advantage of your support so that they can help others. And as we all know, there is no more powerful way to learn than to try to teach others.

For that student half way round the world, perhaps working alone, trying to improve his or her life through education — by learning to think mathematically — the potential benefit is, of course, far greater. Helping that unknown young (or not so young) person make that step might just help inspire your own students to put in that bit of extra effort to master that tricky new transition material. Everyone wins.

If my Stanford MOOC draws a student body in the tens of thousands, which it might, based on the experience of my colleagues here, there is no way I and a couple of graduate TAs can provide individual feedback to every student. But if instructors and their students across the US join me, then maybe we can collectively achieve something remarkable.

I am making my MOOC deliberately short, five weeks, so participation will leave most of the semester open for participating instructors to concentrate on giving their own course, perhaps using their students’ initial experience in the MOOC community as a springboard for the rest of the course. I will make it a basic vanilla transition course, so other instructors can build on it.

Of course, you don’t have to be an instructor or a student in a transition class in order to participate. The course is totally open and free. You simply have to register (online) and start the course. So anyone who is familiar with the material — who already can think like a mathematician — can help out.

Those of us in education know how it can change lives. Growing up in a working class area of the UK in the early Post Second World War era, a free education provided by the government changed mine. Now, through technology, I can return the favor by helping others around the world change theirs. Please join me this fall as we learn how to teach the world.

Let’s teach the world

This coming October, I’ll be offering my first MOOC — massive(ly) open online course — one of a growing number of such offerings that have started to emerge from some leading US universities over the past few months. In this blog, I’ll chronicle my experience as it happens, and hopefully get useful feedback from others. This introductory post is a shortened version of my May 1 blogpost on Devlin’s Angle for the MAA.

Higher education as we know it just ended. Exactly what will take its place is not at all clear. All that can be said with certainty is that within a few short years the higher education landscape will look very different.

That is not to say that existing colleges and universities will suddenly go away, or indeed change what they do – though I think both will occur to varying degrees in due course. What is changing now is what classifies as higher education, who provides it, how they provide it, who will have access to it, how they will obtain it, and how it will be funded. Distance education, for many years the largely-ignored stepchild of the higher education system, is about to come of age.

This is not just my opinion. My own university, Stanford, recognizes what is going on, and is taking significant steps to lead and stay on top of the change, and a number of Silicon Valley’s famed venture capital firms, who make their fortunes by betting right on the future, have sunk significant funding into what they think may be key players in the new, higher ed world.

Last fall, Stanford computer science professor Sebastian Thrun used the Internet to open his on campus course in artificial intelligence to anyone in the world with Net access, and 160,000 students from 190 countries signed up. Some 22,000 of those students finished the course, receiving “certificates of completion” signed by Thrun (and co-teacher Peter Norvig of Google), but no Stanford credit. (For that, a student has to be on campus and officially registered; annual tuition is $40,050 and entry is fiercely competitive.)

Demonstrating the entrepreneurial spirit that Stanford faculty are famous for, Thrun promptly left Stanford to found a for-profit online university, Udacity. With Udacity receiving financial backing from a large Venture Capital firm, the MOOC – massive open online course – suddenly came of age. A short while later, two more Stanford computer science faculty, Andrew Ng and Daphne Koller, secured $16M of venture capital funding to launch a second Stanford spin-off company, Coursera, a Web platform to distribute a broad array of interactive courses in the humanities, social sciences, physical sciences, and engineering.

Initial courses offered on Coursera include, in addition to several from Stanford, offerings from faculty at the University of Michigan, the University of Pennsylvania, and Princeton. Stanford president John Hennessy appointed a blue-ribbon panel of Stanford faculty to develop a strategy for developing, and delivering, online courses. For free. To the world.

Not wanting to be left behind, just this week, MIT and Harvard announced the launch of edX, a joint effort to mount their own MOOC distribution platform, with each institution committing $30M to the project.

Yes, you read that correctly. The faculty, the universities, and the new platforms are making the courses available for free. All the funding is coming – for now – from for-profit investors and the private universities themselves. Why are they doing that? If you have to ask the question, you don’t really understand the Internet and how it changes everything. Think Napster and the music industry or Skype and the telephone industry. Like the settling of the American territories in the nineteenth century, the initial focus is on establishing a presence in the new land; monetization can come later – almost certainly in ways very different from today’s.

Computer-assisted, distance learning is not new, of course. Stanford was one of the universities that pioneered it the 1960s; many universities have for several decades offered adult professional education courses for a fee, largely to raise funds; and there are the for-profit online schools like the University of Phoenix. More recently, led by MIT, a number of universities started making recordings of their regular courses, together with course materials, available online for free. So what has changed now?

The answer is the platform and the target audience’s experience and expectations have changed. What has been missing so far is the active participation of the distant student in a learning community. Building on technology developed at Stanford to support flipped classroom experiences for its regular students, Udacity and Coursera have secured the major investments required to build scalable, robust platforms that can take the small learning seminar and create a similar experience across the Internet.

A generation that has grown up on the Web has taken to the new online learning medium like fish to water. For instance, during the term when Thrun made his AI course available online, most of the Stanford students enrolled in his class stopped attending his lectures and took their information delivery online, at times convenient to them. But the convenience of Stanford students is not what the MOOC initiative is about. What excites me and my colleagues is the possibility to reach millions who currently have no access to any university at all.

Welcome to the age of the MOOC.



I'm Dr. Keith Devlin, a mathematician at Stanford University. I gave my first free, open, online math course in fall 2012, and have been offering it twice a year since then. This blog chronicles my experiences as they happen.

Twitter Updates

New Book 2012

New book 2011

New e-book 2011

New book 2011

July 2022
M T W T F S S
 123
45678910
11121314151617
18192021222324
25262728293031

%d bloggers like this: