Posts Tagged 'Stanford MOOC'



MathThink MOOC v4 – Part 2

In Part 2, I reveal that I share with Steve Jobs, J K Rowling,  Sebastian Thrun, Thomas Edison, and a successful Finnish video-game studio head, a strong belief in the power of failure.

This post continues the one posted two days ago about the expectations students being to my MOOC.

One of the problematic expectations many students bring to my course is that I will show them how to solve certain kinds of problems, work through a couple of examples, and then ask them to solve one or two similar ones. When I don’t do that, some of them complain, in some cases loudly and repeatedly.

There are several reasons why I do not simply continue to serve up the pureed (instructional) diet they are familiar with, and instead offer them some raw meat to chew on.

Most importantly, the course is not about mastering yet more, specific procedures; rather the goal is to acquire a new way of thinking that can be used whenever a novel situation is encountered. Tautologically, that cannot be “taught.” It has to be learned. The role of the “instructor” is not to instruct, but to offer guidance and feedback – the latter being feasible in a MOOC by virtue of most beginners having broadly similar reactions and making essentially the same mistakes.

To progress in the course, the student has to grow accustomed to the way professional mathematicians (to say nothing of engineers, business leaders, athletes, and the like) make progress: learn by failing. That’s the raw meat I serve up: failure.

Not global failure that debilitates and marks an end to an endeavor; rather repeated local failures that lead to eventual success. (Though the distinction is really one of our attitude toward a failure – I’ll come back to this in a moment.)

Most of us find it difficult making the adjustment to regarding failing as an integral part of learning, in large part because our school system misguidedly penalizes (all) failures and rewards (every little) success.

Yet, it is only when we fail that we actually learn something. The more we fail, the better we learn; the more often we fail, the faster we learn. A person who tries to avoid failure will neither learn nor succeed. If you take a math test and score more than 75%, then you are taking a test that is too easy for you, and hence does not challenge you to learn. A score of 75% or more says you did not need to take the test! You were not pushing the frontiers of your current abilities.

I should add that I am not talking about tests and exams designed to determine what you have learned, rather those that are an integral part of the learning process – which in my case, giving a course that offers no credential, means all the “graded” work.

In my course, the numbers the system throws out after a machine-graded Problem Set, or the mark assigned by peer evaluation, are merely indicators of progress. A grade between 30% and 60% is very solid; above 60% means you are not yet at the threshold where significant (for you) learning will take place, while a score below 30% tells you either that you need to put more time and effort into mastering the material, or slow down, perhaps working through the remainder of the course at your own pace then trying again the next time it is offered. (Another great advantage of a free MOOC.)

What is important is not whether you fail, but what you do as a result. As I was working on this post, I came across an excellent illustration in an article in FastCompany about the Finnish video game studio Supercell. Though the young company has only two titles in the market – Clash of Clans and Hay Day – it grossed $100 million in 2012 and $179 million in the first quarter of 2013 alone.

Supercell’s developers work in autonomous groups of five to seven people. Each cell comes up with its own game ideas.  If the team likes it, the rest of the employees get to play. If they like it, the game gets tested in Canada’s iTunes App store. If it’s a hit there it will be deemed ready for global release.

This approach has killed off several games. But here is the kicker: each dead project is celebrated. Employees crack open champagne to toast their failure. “We really want to celebrate maybe not the failure itself but the learning that comes out of the failure,” says Ilkka Paananen, the company’s 34-year-old CEO.

It’s not just in the PISA scores where Finland shows the world it knows a thing or two about learning; you can find it manifested in the App Store download figures as well!

(And let’s not forget that another Finnish game studio, Rovio, produced over a dozen failed games before they hit the global App Store jackpot with Angry Birds.)

Where I live, in Silicon Valley, one of the oft-repeated mantras is, “Fail fast, fail often.” The folks who say that do pretty well in the App Store too. In fact, some of them own the App Store!

One of my main goals in giving my MOOC is helping people get comfortable with failing. You simply cannot be a good mathematical thinker if you are not prepared to fail – frequently and repeatedly. Failing is what professional mathematicians do maybe 99% of the time. Responding appropriately to failure is a key part of mathematical thinking.

And not just mathematical thinking. It’s definitely true of engineering as well. Remember Thomas Edison, who on being asked how he motivated himself to continue his efforts to build an electric light bulb when a thousand attempts had failed, replied (paraphrase), “They were not failures, I just found a thousand ways it won’t work.”

The metaphor I use regularly in my MOOC is learning to ride a bike. If you think about it, you don’t learn to ride a bike; you learn how not to fall off a bike. And you do that by repeatedly falling off until your body figures out how to avoid falling.

Incidentally, the fact that you really did not learn to ride a bike by learning how to is indicated by the fact that almost no one can correctly answer the question, What direction do you turn the handlebars in order for the bike to turn to the right? Your conscious mind, the one that would have been involved if you had learned how to ride a bike, says you twist the handlebars to the right in order to turn the bike to the right. But, if you are able to ride a bike, your body knows better. You turn the handlebars to the left in order to make the bike turn to the right. Your body figured that out when it learned how not to fall down.

Don’t believe me? Go out and try. Make a conscious attempt to turn right by twisting the handlebars to the right. Most likely, your body will prevent you carrying through. But if you manage to over-ride your body’s instinct, you will promptly fall off. So please, do this on grass, not the hard pavement.

Not surprisingly, six weeks in a MOOC is woefully little to adjust to the professionals’ view of failure. The ones who breezed through my course, unfazed by seeing the system return a grade of 30% on a Problem Set, were in most cases, I suspect (and in a fair number of cases that suspicion was confirmed), professional engineers, business people, or others with a fair bit of post-high-school education under their belts. Those for whom the course was one of their first ventures into collegiate education, often had a hard time of it. (Not a few gave up and dropped the course, sometimes leaving an angry, departing post on the class forum page.)

It’s not called a “transition course” for nothing.

I’ll continue this theme of dealing with student expectations in my next post.

Meanwhile, I’ll leave you with three more examples about the power of failing in the learning process.

The first is Steve Jobs’ 2005 commencement address at Stanford.

The second is J. K. Rowling’s 2008 commencement address at Harvard.

Finally, and very close to home, is Sebastian Thrun’s recent business pivot of his MOOC delivery company Udacity, which I discussed in a commentary in the Huffington Post. Though I would agree with the many commentators that his initial attempt had “failed,” where the tone of many was dismissive, I saw just another instance of someone on the pathway to (for him, yet another) success. It’s all about how you view failure and what you do next.

I’ll continue the theme of dealing with student expectations in my next post.

MathThink MOOC v4 – Part 1

In Part 1 of a series, I focus on the distinction between high school math and university-level mathematics, suggesting they are effectively different subjects that are best learned in different ways.

One of the biggest obstacles in giving an online course on mathematical thinking, which my MOOC is, is coping with the expectations students bring to the course – expectations based in large part on their previous experience of mathematics classes. To be sure, prior expectations are often an issue for regular, physical classes. But there the students have an opportunity to interact directly with the instructor on a regular basis. They also have the benefit of a co-present support group of others taking the same class.

But in a massive open online class, apart from locally configured support groups and text-based discussions on the MOOC platform discussion forum, each student is pretty much on her or his own.

The situation is particularly bad for a course like mine, designed to help students transition from high school mathematics to university-level mathematics. For one thing, the two are so different as to be in many ways completely distinct subjects.

School mathematics tends to be almost exclusively procedural, mastering established methods to solve artificially constructed problems designed to be amenable to such an approach. The student who best masters all the techniques in the syllabus and becomes skillful in pattern-matching problems to solution methods, does well. (I know that first hand; it’s how I got to university to study mathematics!)

In contrast, university mathematics is about learning how to deal with a novel situation of a kind you have not encountered before. (If no one else has encountered it, we call it mathematics research.) Though it certainly can involve pattern matching and the application of established, standard procedures, it usually does so only as components of a novel solution you develop to deal with that particular situation. Moreover, at university level, the problems are typically of a “prove that this is true (or false)” variety, rather than “solve this equation” or “compute the value of that formula.”

What is more, while a school math problem typically has a right answer, university mathematics generally involves much more than mere correctness. Indeed, there may not be a unique “right answer.”

Not only is the subject matter different, so too is the pedagogy. Almost all students’ experience of mathematics learning in school is teacher instruction. The teacher describes a method, does a few worked examples, and then asks the students to do a few similar ones. Rinse and repeat.

It’s a very efficient way to cover a lot of ground when the goal is pattern matching and procedure application. It works for school mathematics. Unfortunately, it does not prepare the graduates for the other kind of mathematics. (It also leaves them without ever having a satisfactory answer to their question “What is this good for?”, a question that leaves anyone versed in mathematics astounded. “What is it not good for?” is a more interesting question. It does not have a simple answer, by the way. It’s a very nuanced question.)

It’s like teaching someone the elements of bricklaying, carpentry, plumbing, and electrical wiring, and then asking them to go out and design and build a house. You need all of those skills to build a house, but on their own they are not enough. Not even close.

In deciding, almost two years ago now (before the New York Times had heard of MOOCs) to develop a MOOC to help people learn the other kind of mathematics, what I call mathematical thinking, I knew I was taking on a big challenge. I’d found it hard to teach that kind of course in a physical classroom with just 25, carefully selected students at elite colleges and universities.

On the other hand, most people go through their entire mathematics education without ever encountering what I and my colleagues would call “real mathematics,” and many of them eventually find they need to be able to handle novel situations that involve – or may involve – or could productively be made to involve – mathematical thinking. So I felt there was a need to have a resource publicly available to help them acquire this valuable ability.

The huge dropout rates in MOOCs did not really bother me. For a mathematical thinking course, it’s possible to gain value from dropping into the course for just a few days – and to keep coming back at future times if required. The focus was not on credentialing, it was developing a valuable mental ability – a powerful way of thinking that our ancestors have developed over three thousand years.

That way of thinking can be utilized profitably in many other courses that do yield a certified credential, so students could approach the course as a low-stress, no-risk way of preparing for subsequent learning.

The course is structured as course for those students who seek an encapsulated experience, and in many ways that yields the greatest benefits, in large part because of the interactions with other students working on the same stuff. But the majority of students who have taken it the three times I have offered it have just taken a part of the course.

Each time I gave the course, I changed it, based on what I had learned. When it launches again in February, it will be different again. This time, in some fairly significant ways. In the coming days, I’ll describe those changes and why I made them.

First out of the gate, I’ll describe what exactly were the problems caused by those expectations many students brought to the course, and  how did I try to deal with them. Also, what am I changing in the coming version of the course to try to help more people make what is a very difficult transition: from being taught (i.e., instructed) to being able to learn. The reward for making that one transition is huge. It opens up all of mathematics, and in the process makes it much, much easier.

The traditional, instructional way of teaching procedural mathematics frequently leaves students with the impression (dramatically documented by my Stanford colleague Jo Boaler) that mathematics consists of a large number of rules to be learned. But at the risk of sounding like those weird web advertisements (you know, the ones with a drawing or photo of a strange looking person) promising to teach that “one great trick” that will change your life, let me leave you by telling you the one great trick that all mathematicians learn:

You just have to master, once, a particular way of thinking, and you no longer need all those rules.

That’s what my course focuses on. Stay tuned.

The MOOC will soon die. Long live the MOOR

A real-time chronicle of a seasoned professor who just completed giving his second massively open online course.

The second running of my MOOC (massive open online course) Introduction to Mathematical Thinking ended recently. The basic stats were:

Total enrollment: 27,930

Number still active during final week of lectures: ca 4,000

Total submitting exam: 870

Number of students receiving a Statement of Accomplishment: 1,950

Number of students awarded a SoA with Distinction: 390

From my perspective, it went better than the first time, but this remains very much a research project, and will do for many more iterations. It is a research project with at least as many “Can we?” questions as “How do we?”

From the start, I took the viewpoint that, given the novelty of the MOOC platform, we need to examine the purpose, structure, and use of all the familiar educational elements: “lecture,” “quiz,” “assignment,” “discussion,” “grading,” “evaluation,” etc. All bets are off. Some changes to the way we use these elements might be minor, but on the other hand, some could be significant.

For instance, my course is not offered for any form of college credit. The goal is purely learning. This could be learning solely for its own sake, and many of my students approached it as such. On the other hand, as a course is basic analytic thinking and problem solving, with an emphasis on mathematical thinking in the second half of the course, it can clearly prepare a student to take (and hopefully do better in) future mathematics or STEM courses that do earn credit – and I have had students taking it with that goal in mind.

Separating learning from evaluation of what has been learned is enormously freeing, both to the instructor and to the student. In particular, evaluation of student work and the awarding of grades can be devoted purely to providing students with a useful (formative) indication of their progress, not a (summative) measure of their performance or ability.

To be sure, many of my students, conditioned by years of high stakes testing, have a hard time adjusting to the fact that a grade of 30% on a piece of work can be very respectable, indeed worth an A in many cases.

My typical response to students who lament their “low” grade is to say that their goal should be that a problem for which they struggle to get 30% in week 2 should be solvable for 80% or more by week 5 (say). And for problems they struggle with in week 8 (the final week of curriculum in my course), they should be able to do them more successfully if they take the course again the next time it is offered – something else that is possible in the brave new world of MOOCs. (Many of the students in my second offering of the course had attempted the first one a few months earlier.)

Incidentally, I think I have to make a comment regarding my statement above that the MOOC platform is novel. A number of commentators have observed that “online education is not new,” and they are right. But they miss the point that even this first generation of MOOC platforms represents a significant phase shift, not only in terms of the aggregate functionality but also the social and cultural context in which today’s MOOCs are being offered.

Regarding the context, not only have many of us grown accustomed to much of our interpersonal interaction being mediated by the internet, the vast majority of people under twenty now interact far more using social media than in person.

We could, of course, spend (I would say “waste”) our time debating whether or not this transition from physical space to cyberspace is a good thing. Personally, however, I think it is more productive to take steps to make sure it is – or at least ends up – a good thing. That means we need to take good education online, and we need to do so for the same reason that it’s important to embed good learning into video games.

The fact is, we have created for the new and future generations a world in which social media and video games are prevalent and attractive – just as earlier generations created worlds of books and magazines, and later mass broadcast media (radio, films, television) which were equally as widespread and attractive in their times. The media of any age are the ones through which we must pass on our culture and our cumulative learning. (See my other blog profkeithdevlin.org for my argument regarding learning in video games.)

Incidentally, I see the points I am making here (and will be making in future posts) as very much in alignment with, and definitely guided by, the views Sir Ken Robinson has expressed in a series of provocative lectures, 1, 2, 3.

Sir Ken’s thoughts influenced me a lot in my thinking about MOOCs. To be sure, there is much in the current version of my MOOC that looks very familiar. That is partly because of my academic’s professional caution, which tells me to proceed in small steps, starting from what I myself am familiar with; but in part also because the more significant changes I am presently introducing are the novel uses I am making (or trying to make) of familiar educational elements.

The design of my course was also heavily influenced by the expectation (more accurately a recognition, given how fast MOOCs are developing) that no single MOOC should see itself as the primary educational resource for a particular learning topic. Rather, those of us currently engaged in developing and offering MOOCs are, surely, creating resources that will be part of a vast smorgasbord from which people will pick and choose what they want or need at any particular time.

Given the way names get assigned and used, we may find we are stuck with the name MOOC (massive open online course), but a better term would be MOOR, for massive open online resource.

For basic, instructional learning, which makes up the bulk of K-12 mathematics teaching (wrongly in my view, but the US will only recognize that when virtually none of our home educated students are able to land the best jobs, which is about a generation away), that transition from course to resource has already taken place. YouTube is littered with short, instructional videos that teach people how to carry out certain procedures.

[By the way, I used the term “mathematical thinking” to describe my course, to distinguish it from the far more prevalent instructional math course that focuses on procedures. Students who did not recognize the distinction in the first three weeks, and approached the material accordingly, dropped out in droves in week four when they suddenly found themselves totally lost.]

By professional standards, many of the instructional video resources you can find on the Web (not just in mathematics but other subjects as well) are not very good, but that does not prevent them being very effective. As a professional mathematician and mathematics educator, I cringe when I watch a Khan Academy video, but millions find them of personal value. Analogously, in a domain where I am not an expert, bicycle mechanics, I watch Web videos to learn how to repair or tune my (high end) bicycles, and to assemble and disassemble my travel bike (a fairly complex process that literally has potential life and death consequences for me), and they serve my need, though I suspect a good bike mechanic would find much to critique in them. In both cases, mathematics and bicycle mechanics, some sites will iterate and improve, and in time they will dominate.

That last point, by the way, is another where many commentators miss the point. Something else that digital technologies and the Web make possible is rapid iteration guided by huge amounts of user feedback data – data obtained with great ease in almost real time.

In the days when products took a long time, and often considerable money, to plan and create, careful planning was essential. Today, we can proceed by a cycle of rapid prototypes. To be sure, it would be (in my view) unwise and unethical to proceed that way if a MOOC were being offered for payment or for some form of college credit, but for a cost-free, non-credit MOOC, learning on a platform that is itself under development, where the course designer is learning how to do it, can be in many ways a better learning experience than taking a polished product that has stood the test of time.

You don’t believe me? Consider this. Textbooks have been in regular use for over two thousand years, and millions of dollars have been poured into their development and production. Yet, take a look at practically any college textbook and ask yourself is you could, or would like to, learn from that source. In a system where the base level is the current college textbook and the bog-standard course built on it, the bar you have to reach with a MOOC to call it an improvement on the status quo is low indeed.

Again, Khan Academy provides the most dramatic illustration. Compared with what you will find in a good math classroom with a well trained teacher, it’s not good. But it’s a lot better than what is available to millions of students. More to the point, I know for a fact that Sal Khan is working on iterating from the starting point that caught Bill Gates’ attention, and has been for some time. Will he succeed? It hardly matters. (Well, I guess it does to Sal and his employees!) Someone will. (At least for a while, until someone else comes along and innovates a crucial step further.)

This, as I see it, is what, in general terms, is going on with MOOCs right now. We are experimenting. Needless to say – at least, it should be needless but there are worrying developments to the contrary – it would be unwise for any individual, any educational institution, or any educational district to make MOOCs (as courses) an important component of university education at this very early stage in their development. (And foolish to the point of criminality to take them into the K-12 system, but that’s a whole separate can of worms.)

Experimentation and rapid prototyping are fine in their place, but only when we all have more experience with them and have hard evidence of their efficacy (assuming they have such), should we start to think about giving them any critical significance in an educational system which (when executed properly) has served humankind well for several hundred years. Anyone who claims otherwise is probably trying to sell you something.

A final remark. I’m not saying that massive open online courses will go away. Indeed, I plan to continue offering mine – as a course – and I expect and hope many students will continue to take it as a complete course. I also expect that higher education institutions will increasingly incorporate MOOCs into their overall offerings, possibly for credit. (Stanford Online High School already offers a for-certificate course built around my MOOC.) So my use of the word “die” in the title involved a bit of poetic license

But I believe my title is correct in its overall message. We already know from the research we’ve done at Stanford that only a minority of people enroll for a MOOC with the intention of taking it through to completion. (Though that “minority” can comprise several thousand students!) Most MOOC students already approach it as a resource, not a course! With an open online educational entity, it is the entire community of users that ultimately determines what it primarily is and how it fits in the overall educational landscape. According to the evidence, they already have, thereby giving us a new (and more accurate) MOOC mantra: resources, not courses. (Even when they are courses and when some people take them as such.)

In the coming posts to this blog, I’ll report on the changes I made in the second version of my MOOC, reflect on how things turned out, and speculate about the changes I am thinking of making in version 3, which is scheduled to start in September. First topic up will be peer evaluation – something that I regard as key to the success of a MOOC on mathematical thinking.

Those of us in education are fortunate to be living in a time where there is so much potential for change. The last time anything happened on this scale in the world of education was the invention of the printing press in the Fifteenth Century. As you can probably tell, I am having a blast.

To be continued …

How are MOOCs organized?

A real-time chronicle of a seasoned professor who is about to give his second massively open online course.

With exactly one week to go before the second edition of my MOOC Introduction to Mathematical Thinking goes live, my TA and I have been working feverishly to get everything ready — a task far more complex and time consuming than preparing for a traditional (physical) course. (If you have been following this blog since I launched it last summer, when I started to plan my first edition of the course, you likely have some idea of the complexities involved.)

MOOCs continue to be in the news. Just last week, NBC-tv used my course as an illustration in a news story (4 min 21 secs) they ran about the American Council on Education’s recommendation that some Coursera MOOCs be considered eligible to receive college credit.

But what exactly is a MOOC and how are they organized? The easiest way to find out is to simply sign up for one or more and take a look. They are all free (at least, all the ones everyone is talking about are free), and there is no requirement to do any more than hang around online and see what is going on. If you do that, you’ll find that they all exhibit some differences from one another, as well as many similarities. Moreover, almost everyone giving a MOOC approaches it as an experiment, so they often change from one edition to the next.

Taking my own MOOC as an illustration, when the course website opens to registered students next weekend (Saturday March 2), they will initially find themselves in a website populated with several pages of information about the course structure, together with a bit of background information relevant to the course content, but none of the lectures, assignments, quizzes, problem sets, or tutorials will be available. Those are released at specified times throughout the ten weeks the course will run, starting with Lecture 1 on March 4.

For a sample of a lecture, see this short clip (7min 16 sec) from Lecture 1 on YouTube. (But note that Coursera videos are much higher resolution than YouTube, so the YouTube video is hard to follow — it’s purely an illustration of the overall format of the lectures.)

One of the main informational pages the students will see describes the various components of the course. Here, verbatim, are the contents of that page.

Basic elements of the course

Consult the Daily timetable (see link on left) on a regular basis to see what is due at any one time.

1. Lectures – videos presented by the instructor.
2. In-lecture quizzes – simple multiple-choice questions that stop the lecture, designed to assist you in pacing and monitoring your progress.
3. Assignment sheets (one for each lecture) – downloadable PDF files to work through in your own time at your own pace, ideally in collaboration with other students. Not graded.
4. Problem sets (one a week for weeks 1 through 8) – in-depth problems like those on the assignment, but with a deadline for submitting your answers (in a multiple choice format). Machine graded.
5. Tutorial sessions – the instructor provides (video) comments and answers to some of the previous week’s assignment problems.
6. Reading assignments – downloadable PDFs files providing important background information.
7. Final exam – a downloadable PDF file that you will have one week to complete before participating in a peer review process. Required to be eligible for a grade of completion with distinction.

Lectures

Lecture videos are released at 10:00AM US-PDT on Wednesdays. (Weeks 1 and 2 are slightly different, with lectures released on Monday and Wednesday.) Each lecture comprises one or two videos, with each video of length 25 to 35 minutes if played straight through. Completing the embedded progress quizzes will extend the total duration of a video-play by a few minutes, and you will likely want to stop the playback several times for reflection, and sometimes you will want to repeat a section, perhaps more than once. So you can expect to spend between one and two hours going through each lecture, occasionally perhaps more.

The lecture videos are not carefully crafted, heavily edited productions. If you want a polished presentation of the course material, you can read the course textbook. My goal with the lectures is to provide as best I can the experience of sitting alongside me as we work through material together. And, guess what, I often make mistakes, and sometimes mis-speak. I want to dispel any misconception that mathematicians are people who generate perfect logical arguments all the time. We’re not. We just keep going until we get it right.

In-lecture quizzes (Ungraded)

Each lecture is broken up by short multiple-choice “progress quizzes”. The vast majority of these in-lecture quizzes are essentially punctuation, providing a means for you to check that you are sufficiently engaged with the material.

Slightly modified versions of the quizzes will also be released as standalones at the same time as the lecture goes live, so if you do not have a good broadband connection and have to download the lecture videos to watch offline, you can still take the quizzes. In which case, you should do so as close in time to viewing the lecture as possible, to ensure gaining maximim benefit from the quizzes in monitoring your progress. The standalone quizzes are grouped according to lecture.

Completion of all the quizzes is a requirement (along with watching all the lectures) for official completion of the course, but we do not record your quiz scores, so quiz performance does not directly affect your final grade. If you complete the quizzes while watching the lecture (the strongly preferred method, as it helps you monitor your progress in mastering the material), you do not need to complete the standalone versions.

BTW, you may notice that it is possible to speed up video replay up to a factor of double speed. This can be a useful device when watching a video a second or third time. Going beyond 1.50 speed, however, can sometimes lead to problems with the display of the quizzes (besides making me sound like a chipmunk (though some may find that an enhancement).

Course assignments (Self graded)

An assignment will be released at the end of each lecture, as a downloadable PDF file. The assignment is intended to guide understanding of what has been learned. Worked solutions to problems from the assignments will be demonstrated (video) or distributed (PDFs) in a tutorial session released the Monday following the lecture (so in Weeks 2 through 9). The tutorial sessions will be released at 10:00AM US-PDT.

Working on these assignment problems forms the heart of the learning process in this course. You are strongly urged to form or join a study group, discuss the assignment problems with others in the group, and share your work with them. You should also arrange to assess one another’s answers. A structured form of peer review will be used for the final exam, when you will be graded by, and grade the work of, other students, randomly (and blindly) assigned, so it will help to familiarize yourself beforehand with the process of examining the work of others and providing (constructive) feedback.

Problem Sets (Machine graded)

Each Wednesday (in weeks 1 thtough 8), following the lecture, a for-credit Problem Set will be posted, with submission due by 9:00AM US-PDT the following Monday. The scores on these problem sets will count toward the course grade. Though the Problem Set has a multiple-choice quiz format, these questions are not the kind you can answer on the spot (unlike most of the in-lecture quizzes). You will need to spend some time working on them before entering your answers.

Though you are strongly encouraged to work with others on understanding the lecture material and attempting the regular assignments, the intention is that you work alone on the Problem Sets, which are designed to give you and us feedback on how you are progressing.

Tutorial sessions

The tutorial sessions are more than mere presentations of solutions to the previous week’s assignments and problem session. They are really lectures based on problems that the student has already attempted. You can expect to expand your knowledge of the course material beyond the lectures. Not all questions on the assignments sheets and problem set will be considered in the tutorial session.

Final exam (Peer graded)

Though the lectures end after week 8 (apart from a tutorial on the final assignment), the final two weeks are intended to be highly active ones for any students seeking a grade of distinction, with considerable activity online in the various forums and discussion groups. This is when you are supposed to help one another make sense of everything.

At the start of week 9, an open-book exam will be released, to be completed by the end of the week. Completed exams will have to be uploaded as either images (or scanned PDFs) though students sufficiently familiar with TeX have an option of keyboard entry on the site. The exam will be graded during week 10 by a calibrated peer review system. The exam will be based on material covered in the entire course.

As with the weekly Problem Sets, the intention is that you work alone in completing the final exam.

NOTE: The process of peer reviewing the work of others (throughout the course, not just in the final exam) is intended to be a significant part of the learning experience and participating in the formal peer review procedure for the final exam is a requirement for getting a grade of distinction. In principle, it is during week 10 that stronger students will make cognitive breakthroughs. (Many of today’s professors really started to understand mathematics when, as graduate student TAs, they first helped others learn it!)

Course completion and final grade

There are two final course grades: “completion” and “completion with distinction”. Completion requires viewing all the lectures and completing all the (in-lecture) quizzes and the weekly problem sets. Distinction depends on the scores in the problem sets and the result of the final exam.

Pacing

The pacing of the lecture releases is designed to help you maintain a steady pace. At high school, you probably learned that success in mathematics comes from working quickly (and alone) and getting to the right answer as efficiently as possible. This course is about learning to think a certain way – the focus is on the process not the product. You will need time to understand and assimilate new ideas. Particularly if you were a whiz at high-school math, you will need to slow down, and to learn to think and reflect (and ideally discuss with others) before jumping in and doing. A steady pace involving some period of time each day is far better than an all-nighter just before a Problem Set is due.

Keeping track

Consult the Daily timetable on the website on a regular basis to see what is due.

SO NOW YOU KNOW!

Here we go again

A real-time chronicle of a seasoned professor who is about to give his second massively open online course.

The second offering of my MOOC Introduction to Mathematical Thinking begins on March 4 on Coursera. (The site actually opens on March 2, so students can familiarize themselves with its structure and start to make contact with other students before the first lecture.) So far, 13,000 students have registered. Last time I got 65,000, but back then there was the novelty factor. I’m expecting about 35,000 this time round.

For a quick overview of my current thoughts on MOOCs, see this 13 minute TV interview I did at Tallinn University of Technology in Estonia last November. (As the home of Skype, global-tech-hub Tallinn is particularly interested in MOOCs, of course.)

It’s been almost four months since my first foray into the chaotic new world of MOOCs came to an end, and ten weeks since I posted my last entry on this blog. I have decided that giving a MOOC falls into the same category as running a marathon (I’ve done maybe two dozen), completing the Death Ride (three), and – I am told – having a baby (I played a decidedly minor role in two). At the time you wonder why you are putting yourself through such stress, and that feeling continues for a while after the event is over. But then the strain of it all fades and you are left with feelings of pleasure, accomplishment, and satisfaction. And with that comes the desire to do it all again – better in the case of running, cycling, and MOOCing.

Coursera, we have a problem

It’s important to remember that genuinely massive MOOCs are a mere eighteen months old, and each one is very much a startup operation — as are the various platform providers such as Udacity, edX, Coursera, Venture Labs. and Class2Go (all except edX coming out of Global Startup Central, i.e., Stanford). One of the features of any startup operation is that there will be plenty of missteps along the way. Given the complexity of designing  and delivering a university course in real time to tens of thousands of students around the world, it’s amazing that to date there have been just two missteps. The first, when the instructor had to pull the plug on a MOOC on designing online courses (yes, a particularly poignant topic as it turned out) and then more recently when the instructor pulled out, leaving the course to be run by the support staff.

Notice that I did not refer to either as a “failure.” Anyone who views such outcomes as failures has clearly never tried to do anything new and challenging, where you have to make up some of the rules as you go on. We are less than two years into this whole MOOC thing, so it’s worth reminding ourselves what it took (VIDEO) the USA to put a man on the Moon and bring him back alive, and to go on and build the Space Shuttle. The pedagogic fundamental that we gain confidence from our successes but learn from our mistakes, is as true for MOOC platform builders and MOOC instructors as it is for MOOC students.

Fortunately, I survived my first test flight relatively unscathed. I may not be so lucky second time round. I’ve made some changes that are intended to make the course better, but won’t know if they do until the course is underway.

Perhaps the most obvious change is to stretch the course from seven weeks (five weeks of lectures followed by two weeks of final exam work) to ten (8 + 2). Many students in my first course told me that the “standard university pace” with which I covered the curriculum was simply too much for online students who were fitting the course around busy professional and family schedules. I doubt that change will have any negative consequences.

More uncertain in their outcome are the changes I have made to the peer review process, that forms a major component of the course for students who are taking it for a Certificate of Completion (particularly Completion with Distinction).

Give credit where credit is due? Maybe

Talking of which, the issue of credentialing continues to generate a lot of discussion. My course does not offer College Credit (and it is not clear any Stanford MOOC ever will), but just recently, the American Council on Education’s College Credit Recommendation Service (ACE CREDIT)  has evaluated and recommended college credit be given for five MOOCs currently offered (by other universities) on Coursera. (Starting this March, it will be possible to take an enhanced version of my MOOC given by Stanford Online High School, for which a credential is awarded, but that course, aimed at high flying high school juniors and seniors, has a restricted enrollment and carries a fee, so it is not a MOOC, rather a course with tutors and assessment, built around my MOOC.)

But I digress. As I observed on a number of occasions in this blog and my MAA blog Devlin’s Angle, I see group work and peer evaluation as the key to making quality mathematics education available in a MOOC. So students who took the first version of my course and are planning on enrolling again (and I know many are) will see some changes there. Not huge ones. Like NASA’s first fumbling steps into space, I think it is prudent to make small changes that have a good chance of being for the better. But I learned a lot from my first trip into MOOC-space, and I expect to learn more, and make further changes, on my second flight.

Finally, if you want to learn more about my reflections on my first MOOC and MOOCs in general, and have a two hour car drive during which you would find listening to a podcast about MOOCs marginally better than searching through an endless cycle of crackly Country and Western radio stations, download the two podcast files from Wild About Math, where host Sol Lederman grills me about MOOCs.

Coming up for air (and spouting off)

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

Almost a month has passed since I last posted to this blog. Keeping my MOOC running took up so much time that, once it was over, I was faced with a huge backlog of other tasks to complete. Taking a good look at the mass of data from the course is just one of several post-MOOC activities that will have to wait until the New Year. So readers looking for statistics, analyses, and conclusions about my MOOC will, I am afraid, have to wait a little bit longer. Like most others giving these early MOOCs, we are doing so on the top of our existing duties; the time involved has yet to be figured into university workloads.

One issue that came up recently was when I put on my “NPR Math Guy” hat and talked with Weekend Edition host Scott Simon about my MOOC experience.

In the interview, I remarked that MOOCs owed more to Facebook than to YouTube. This observation has been questioned by some people, who believe Kahn Academy’s use of YouTube was the major inspiration. In making this comment, they are echoing the statement made by former Stanford Computer Science professor Sebastian Thrun when he announced the formation of Udacity.

In fact, I made my comment to Scott with my own MOOC (and many like it) in mind. Though I have noted in earlier posts to this blog how I studied Sal Khan’s approach in designing my own, having now completed my first MOOC, I am now even more convinced than previously that the eventual (we hope) success of MOOCs will be a consequence of Facebook (or social media in general) rather than of Internet video streaming.

The reason why I felt sure this would be the case is that (in most disciplines) the key to real learning has always been bi-directional human-human interaction (even better in some cases, multi-directional, multi-person interaction), not unidirectional instruction.

What got the entire discussion about MOOCs off in the wrong direction – and with it the public perception of what they are – is the circumstance of their birth, or more accurately, of their hugely accelerated growth when a couple of American Ivy League universities (one of them mine) got in on the act.

But it’s important to note that the first major-league MOOCs all came out of Stanford’s Computer Science Department, as did the two spinoff MOOC platforms, Udacity and Coursera. When MIT teamed up with Harvard to launch their edX platform a few months later, it too came from their Computer Science Department.

And there’s the rub. Computer Science is an atypical case when it comes to online learning. Although many aspects of computer science involve qualitative judgments and conceptual reasoning, the core parts of the subject are highly procedural, and lend themselves to instruction-based learning and to machine evaluation and grading. (“Is that piece of code correct?” Let the computer run it and see if it performs as intended.)

Instructional courses that teach students how to carry out various procedures, which can be assessed to a large degree by automatic grading (often multiple choice questions) are the low hanging fruit for online education. But what about the Humanities, the Arts, and much of Science, where instruction is only a small part of the learning process, and a decidedly unimportant part at that, and where machine assessment of student work is at best a goal in the far distant future, if indeed it is achievable at all?

In the case of my MOOC, “Introduction to Mathematical Thinking,” the focus was the creative/analytic mathematical thinking process and the notion of proof. But you can’t learn how to think a certain way or how prove something by being told or shown how to do it any more than you can learn how to ride a bike by being told or shown. You have to try for yourself, and keep trying, and falling, until it finally clicks. Moreover, apart from some very special, and atypical, simple cases, neither thinking nor proofs can be machine graded. Proofs are more like essays than calculations. Indeed, one of the things I told my students in my MOOC was that a good proof is a story, that explains why something is the case.

For the vast majority of students, discussion with (and getting feedback from) professors, TAs, and other students struggling to acquire problem solving ability and master abstract concepts and proofs, is an essential part of learning. For those purposes, the online version does not find its inspiration in Khan Academy as it did for Thrun, but in Facebook, which showed how social interaction could live on the Internet.

When the online version of Thrun’s Stanford AI class attracted 160,000 students, he did not start a potential revolution in global higher education, but two revolutions, only the first of which he was directly involved in. The first one is relatively easy to recognize and understand, especially for Americans, who for the most part have never experienced anything other than instruction-based education.

For courses where the goal is for the student to achieve mastery of a set of procedures (which is true of many courses in computer science and in mathematics), MOOCs almost certainly will change the face of higher education. Existing institutions that provide little more than basic, how-to instruction have a great deal to fear from MOOCs. They will have to adapt (and there is a clear way to do so) or go out of business.

If I want to learn about AI, I would prefer to do so from an expert such as Sebastian Thrun. (In fact, when I have time, I plan on taking his Udacity course on the subject!) So too will most students. Why pay money to attend a local college and be taught by a (hopefully competent) instructor of less stature when you can learn from Thrun for free?

True, Computer Science courses are not just about mastery of procedures. There is a lot to be learned from the emphases and nuances provided by a true expert, and that’s why, finances aside, I would choose Thrun’s course. But at the end of the day, it’s the procedural mastery that is the main goal. And that’s why that first collection of Computer Science MOOCs has created the popular public image of the MOOC student as someone watching canned instructional videos (generally of short duration and broken up by quizzes), typing in answers to questions to be evaluated by the system.

But this kind of course occupies the space in the overall educational landscape that McDonalds does in the restaurant business. (As someone who makes regular use of fast food restaurants, this is most emphatically not intended as a denigratory observation. But seeing utility and value in fast food does not mean I confuse a Big Mac with quality nutrition.)

Things are very, very different in the Humanities, Arts, and most of Science (and some parts of Computer Science), including all of mathematics beyond basic skills mastery – something that many people erroneously think is an essential prerequisite for learning how to do math, all evidence from people who really do learn how to do math to the contrary.

[Ask the expert. We don’t master the basic skills; we don’t need them because, early on in our mathematic learning, we acquired one – yes, just one – fundamental ability: mathematical thinking. That’s why the one or two kids in the class who seem to find math easy seem so different. In general, they don’t find math easy, but they are doing something very different from everyone else. Not because they are born with a “math gene”. Rather, instead of wasting their time mastering basic skills, they spent that time learning how to think a certain way. It’s just a matter of how you devote your learning time. It doesn’t help matters that some people managed to become qualified math teachers and professors seemingly without figuring out that far more efficient path, and hence add their own voice to those who keep calling for “more emphasis on basic skills” as being an essential prerequisite to mathematical power.]

But I digress. To get back to my point, while the popular image of a MOOC centers on lecture-videos and multiple-choice quizzes, what Humanities, Arts, and Science MOOCs (including mine) are about is community building and social interaction. For the instructor (and the very word “instructor” is hopelessly off target in this context), the goal in such a course is to create a learning community.  To create an online experience in which thousands of self-motivated individuals from around the world can come together for a predetermined period of intense, human–human interaction, focused on a clearly stated common goal.

We know that this can be done at scale, without the requirement that the participants are physically co-located or even that they know one another. NASA used this approach to put a man on the moon. MMOs (massively multiplayer online games – from which acronym MOOCs got their name) showed that the system works when the shared goal is success in a fantasy game world.

Whether the same approach works for higher education remains an open question. And, for those of us in higher education, what a question! A question that, in my case at least, has proved irresistible.

This, then, is the second MOOC revolution. The social MOOC. It’s outcome is far less evident than the first.

The evidence I have gathered from my first attempt at one of these second kinds of MOOC is encouraging, or at least, I find it so. But there is a long way to go to make my course work in a fashion that even begins to approach what can be achieved in a traditional classroom.

I’ll pursue these thoughts in future posts to this blog — and in future versions of my Mathematical Thinking MOOC, of which I hope to offer two variants in 2013.

Meanwhile, let me direct you to a recent article that speaks to some of the issues I raised above. It is from my legendary colleague in Stanford’s Graduate School of Education, Larry Cuban, where he expresses his skepticism that MOOCs will prove to be an acceptable replacement for much of higher education.

To be continued …

Peer grading: inventing the light bulb

A real-time chronicle of a seasoned professor who has just completed giving his first massively open online course.

With the deadline for submitting the final exam in my MOOC having now passed, the students are engaging in the Peer Evaluation process. I know of just two cases where this has been tried in a genuine MOOC (where the M means what it says), one in Computer Science, the other in Humanities, and both encountered enormous difficulties, and as a result a lot of student frustration. My case was no different.

Anticipating problems, I had given the class a much simplified version of the process – with no grade points at stake – at the end of Week 4, so they could familiarize themselves with the process and the platform mechanics before they had to do it for real. That might have helped, but the real difficulties only emerged when 1,520 exam scripts started to make their way through the system.

By then the instructional part of the course was over. The class had seen and worked through all the material in the curriculum, and had completed five machine-graded problem sets. Consequently, there were enough data in the system to award certificates fairly if we had to abandon the peer evaluation process as a grading device, as happened for that humanities MOOC I mentioned, where the professor decided on the fly to make that part of the exam optional. So I was able to sleep at night. But only just.

With over 1,000 of the students now engaged in the peer review process, and three days left to the deadline for completing grading, I am inclined to see the whole thing through to the (bitter) end. We need the data that this first trial will produce so we can figure out how to make it work better next time.

Long before the course launched, I felt sure that there were two things we would need to accomplish, and accomplish well, in order to make a (conceptual, proof-oriented) advanced math MOOC work: the establishment (and data gathering from) small study groups in which students could help one another, and the provision of a crowd-sourced evaluation and grading system.

When I put my course together, the Coursera platform supported neither. They were working on a calibrated peer review module, but implementing the group interaction side was still in the future. (The user-base growth of Coursera has been so phenomenal, it’s a wonder they can keep the system running at all!)

Thus, when my course launched, there was no grouping system, nor indeed any social media functionality other than the common discussion forums. So the students had to form their own groups using whatever media they could: Facebook, Skype, Google Groups, Google Docs, or even the local pub, bar, or coffee shop for co-located groups. Those probably worked out fine, but since they were outside our platform, we had no way to monitor the activity – an essential functionality if we are to turn this initial, experimental phase of MOOCs  into something robust and useful in the long term.

Coursera had built a beta-release, peer evaluation system for a course on Human Computer Interaction, given by a Stanford colleague of mine. But his needs were different from mine, so the platform module needed more work – more work than there was really time for! In my last post, I described some of the things I had to cope with to get my exam up and running. (To be honest, I like the atmosphere of working in startup mode, but even in Silicon Valley there are still only 24 hours in a day.)

It’s important to remember that the first wave of MOOCs in the current, explosive, growth period all came out of computer science departments, first at Stanford, then at MIT. But CS is an atypical case when it comes to online learning. Although many aspects of computer science involve qualitative judgments and conceptual reasoning, the core parts of the subject are highly procedural, and lend themselves to instruction-based learning and to machine evaluation and grading. (“Is that piece of code correct?” Just see if it runs as intended.)

The core notion in university level mathematics, however, is the proof. But you can’t learn how to prove something by being told or shown how to do it any more than you can learn how to ride a bike by being told or shown. You have to try for yourself, and keep trying, and falling, until it finally clicks. Moreover, apart from some very special, and atypical, simple cases, proofs cannot be machine graded. In that regard, they are more like essays than calculations. Indeed, one of the things I told my students was that a good proof is a story, that explains why something is the case.

Feedback from others struggling to master abstract concepts and proofs can help enormously. Study groups can provide that, along with the psychological stimulus of knowing that others are having just as much difficulty as you are. Since companies like Facebook have shown us how to build platforms that support the creation of groups, that part can be provided online. And when Coursera is able to devote resources to doing it, I know it will work just fine. (If they want to, they can simply hire some engineers from Facebook, which is little more than a mile away. I gather that, like Google before it, the fun period there has long since passed and fully vested employees are looking to move.)

The other issue, that of evaluation and grading, is more tricky. The traditional solution is for the professor to evaluate and grade the class, perhaps assisted by one or more TAs (Teaching Assistants). But for classes that number in the tens of thousands, that is clearly out of the question. Though it’s tempting to dream about building a Wikipedia-like community of dedicated, math-PhD-bearing volunteers, who will participate in a mathematical MOOC whenever it is offered – indeed I do dream about it – it would take time to build up such a community, and what’s more, it’s hard to see there being enough qualified volunteers to handle the many different math MOOCs that will soon be offered by different instructors. (In contrast, there is just one Wikipedia, of course.)

That leaves just one solution: peer grading, where all the students in the class, or at least a significant portion thereof, are given the task of grading the work of their peers. In other words, we have to make this work. And to do that, we have to take the first step. I just did.

Knowing just how many unknowns we were dealing with, my expectations were not high, and I tried to prepare the students for what could well turn out to be chaos. (It did.) The website description of the exam grading system was littered with my cautions and references to being “live beta”. On October 15, when the test run without the grading part was about to launch, I posted yet one more cautionary note on the main course announcements page:

… using the Calibrated Peer Review System for a course like this is, I believe, new. (It’s certainly new to me and my assistants!) So this is all very much experimental. Please approach it in that spirit!

Even so, many of the students were taken aback by just how clunky and buggy the thing was, and the forums sprung alive with exasperated flames. I took solace in the recent release of Apple Maps on the iPhone, which showed that even with the resources and expert personnel available to one of the world’s wealthiest companies, product launches can go badly wrong – and we were just one guy and two part-time, volunteer student assistants, working on a platform being built under us by a small startup company sustained on free Coke and stock options. (I’m guessing the part about the Coke and the options, but that is the prevalent Silicon Valley model.)

At which point, one of those oh-so-timely events occurred that are often described as “Acts of God.” Just when I worried that I was about to witness, and be responsible for starting, the first global, massive open online riot (MOOR) in a math class, Hurricane Sandy struck the Eastern Seaboard, reminding everyone that a clunky system for grading math exams is not the worst thing in the world. Calm, reasoned, steadying, constructive posts started to appear on the forum.  I was getting my feedback after all. The world was a good place once again.

Failure (meaning things don’t go smoothly, or maybe don’t work at all) doesn’t bother me. If it did, I’d never have become a mathematician, a profession in which the failure rate in first attempts to solve a problem is somewhere north of 95%. The important thing is to get enough data to increase the chances of getting it right – or far more likely, just getting it better – the second time round. Give me enough feedback, and I count that “failure” as a success.

As Edison is said to have replied to a young reporter about his many failed attempts to construct a light bulb, “Why would I ever give up? I now know definitively over 9,000 ways that an electric light bulb will not work. Success is almost in my grasp.” (Edison supposedly failed a further 1,000 times before he got it right. Please don’t tell my students that. We are just at failure 1.)

If there were one piece of advice I’d give to anyone about to give their first MOOC, it’s this: remember Edison.

To be continued …


I'm Dr. Keith Devlin, a mathematician at Stanford University. I gave my first free, open, online math course in fall 2012, and have been offering it twice a year since then. This blog chronicles my experiences as they happen.

Twitter Updates

New Book 2012

New book 2011

New e-book 2011

New book 2011

November 2014
M T W T F S S
« Sep    
 12
3456789
10111213141516
17181920212223
24252627282930

Follow

Get every new post delivered to your Inbox.

Join 573 other followers

%d bloggers like this: