Archive Page 2

MathThink MOOC v4 – Part 6

In Part 6, I talk about the new Test Flight process.

In the past, when students enrolled for my MOOC, they essentially had three options. One was not to take it as a course at all, but just regard it as a resource to peruse over time or to pick and choose from. A second was to take the entire course, but do so on their own time-scale. Or they could take it as a course, and go through it at the designated pace.

As do many MOOC designers, I tried to make sure my course could be used in all three ways. Though the vast majority of MOOC students fall into the first category, the other two are the ones that require by far the greatest effort by the course designer. They are the learners who have significant ambitions and will put in a lot of effort over several weeks.

The students in the last category will surely gain the most. In particular, they move through the course in lockstep with a cohort of several thousand other students who can all learn from and support one another, as they face each course deadline at the same time. Those students form the core community that is the heart of the course.

When the new class enrolls at the start of February, the ones intending to take an entire course as scheduled will have a new choice. They can take what I am calling the Basic Course, which lasts eight weeks, or the Extended Course, which lasts ten. As I described in my last post, those extra two weeks are devoted to a process I am calling Test Flight.

In the previous two versions of the course, the final weeks nine and ten had been devoted to a Final Exam, one week for completion of the (open book) exam itself, the following week to peer evaluation. In peer evaluation, which started as soon as the class had completed and submitted their exam solutions, each student went through a number of activities:

1. Using a rubric I supplied, each student evaluated three completed examination scripts assembled by me, and then compared their results to mine. (Those three samples were selected by me to highlight particular features of evaluation that typically arise for those problems.)

2. Having thus had some initial practice at evaluation, each student then evaluated three examination scripts submitted by fellow students. (The Coursera platform randomly and anonymously distributed the completed papers.)

3. Each student then evaluated their own completed examination.

This was the system Coursera recommended, and for which they developed their peer evaluation module. (Actually, they suggested that each student evaluated five peer submissions, but at least for my course, that would have put a huge time requirement on the students, so I settled for three.)

Their original goal, and mine, was to provide a means for assigning course grades in a discipline where machine evaluation is not possible. The theory was that, if each student is evaluated by sufficiently many fellow students, each of whom had undergone an initial training period, then the final grade – computed from all the peer grades plus the self-grade – would be fairly reliable, and indeed there is research that supports this assumption. (Certainly, students who evaluate their own work immediately after evaluating that of other students tend to be very objective.)

As far as I could tell, the system worked as intended. If the goal of a MOOC is to take a regular university course and make it widely available on the Internet, then my first three sessions of the course were acceptably successful. But MOOCifying my regular Mathematical Thinking (transition) class was always just my starting point.

Since I was aware from the outset that the MOOC version of my regular classroom course was just a two-dimensional shadow of the real thing, where I interact with my class on a regular basis and give them specific feedback on their work, my intention always was to iteratively develop the MOOC into something that takes maximum advantage of the medium to provide something new of value – whatever that turns out to be.

I expected that, as MOOCs evolve, they would over time come to be structured differently and be used in ways that could be very different from our original design goals. That, after all, is what almost always happens with any new product or technology.

One thing I observed was that, while students often began feeling very nervous about the requirement that they evaluate the work of fellow students, and (justifiably) had significant doubts about being able to do a good job, the majority found the process of  evaluating mathematical arguments both enjoyable and a hugely beneficial learning process.

Actually, I need to say a bit more about that “majority” claim. My only regular means of judging the reactions of the class to the various elements of the course was to read the postings on the course discussion forums. I spent at least an hour every day going through those forums, occasionally posting a response of my own, but mostly just reading.

Since the number of regular forum posters is in the hundreds, but the effective (full-term) class was in excess of 5,000 in each of the sessions, forum posters are, by virtue of being forum posters, not representative. Nevertheless, I had to proceed on the assumption that any issue or opinion that was shared (or voted up) by more than one or two forum posters was likely to reflect the views of a significant percentage of the entire (full-term) class.

Since I made gradual changes to the course based on that feedback, this means that over time, my course has been developing in a way that suits the more active forum posters. Arguably that is reasonable, since their level of activity suggests they are the ones most committed, and hence the ones whose needs and preferences the course should try to meet. Still, there are many uncertainties here.

To return to my point about the learning and comprehension benefits evaluators gained from analyzing work of their peers, that did not come as a surprise. I had found that myself when, as a graduate student TA, I first had to evaluate students’ work. I had observed it in my students when I had used it in some of my regular classes. And I had read and heard a number of reports from other instructors who noted the same thing.

It was when I factored the learning benefits of evaluating mathematical arguments in with my ongoing frustration with the degree to which “grade hunting” kept getting in the way of learning, that I finally decided to turn the whole exam part on its head.

While some universities and some instructors may set out to provide credentialing MOOCs, my goal was always to focus on the learning, drawing more on my knowledge of video games and video-game learning (see my blog profkeithdevlin.org) than on my familiarity with university education (see my Stanford homepage).

Most of what I know about giving a university-level course involves significant student-faculty interaction and interpersonal engagement, whereas a well-designed video game maintains the player’s attention and involvement using very different mechanisms. With a MOOC of necessity being absent any significant instructor-student interaction, I felt from the outset that the worlds of television and gaming would provide the key weapons I needed to create and maintain student attention in a MOOC.

[A lot of my understanding of how TV captures the viewer’s attention I learned from my close Stanford colleague, Prof Byron Reeves, who did a lot of the groundbreaking research in that area. He subsequently took his findings on television into the video game business, co-authoring the book Total Engagement: Using Games and Virtual Worlds to Change the Way People Work and Businesses Compete.]

So from the outset of my foray into the world of online education, I was looking to move away from traditional higher-education pedagogic models and structure, and towards what we know about (television and) video games, hopefully ending up with something of value in between.

The idea of awarding a Statement of Accomplishment based on accumulated grade points had to go sooner or later, and along with it the Final Exam. Hence, with Session Four, both will be gone. From now on, it is all about the experience – about trying (and failing!).

The intention for the upcoming session is that a student who completes the Basic Course will have learned enough to be able to make useful, and confident use of mathematical thinking in their work and in their daily lives. Completion of the Test Flight process in the Extended Course will (start to) prepare them for further study in mathematics or a mathematically-dependent discipline – or at least provide enough of a taste of university-level mathematics to help them decide if they want to pursue it further.

At heart, Test Flight is the original Final Exam process, but with a very different purpose, and accordingly structured differently.

As a course culmination activity, building on but separate from the earlier part of the course – and definitely not designed to evaluate what has been learned in the course – Test Flight has its own goal: to provide those taking part with a brief hands-on experience of “life as a mathematician.”

The students are asked to construct mathematical arguments to prove results, and then to evaluate other proofs of the same results. The format is just like the weekly Problem Sets that have met throughout the course, and performance level has no more or less significance.

The evaluation rubric, originally employed to try to guarantee accurate peer grading of the exam, has been modified to guide the evaluator in understanding what factors go into making a good mathematical argument.  (I made that change in the previous session.)

After the students have used the rubric to evaluate the three Problem Set solutions supplied by me, they view a video in which I evaluate the same submissions. Not because mine provides the “correct” evaluations. There is usually no single solution to a question and no such thing as the “right” one. Rather, I am providing examples, so they can compare their evaluations with mine.

After that, they then proceed to evaluate three randomly-assigned, anonymously-presented submissions from other students, and finally they evaluate their own submission.

Procedurally, it is essentially the same as the previous Final Exam. But the emphasis has been totally switched from a focus on the person being evaluated (who wants to be evaluated fairly, of course) to the individual doing the evaluation (where striving for a reliable evaluation is a tool to aid learning on the part of the evaluator).

Though I ran a complete trial of the process last time, the course structure was largely unchanged. In particular, there was still a Final Exam for which performance affected the grade, and hence the awarding of a certificate. As a consequence, although I observed enough to give me confidence the Test Flight process could be made to work, there was a square-peg-in-a-round-hole aspect in what I did then that caused some issues.

I am hoping (and expecting) things will go smoother next time. For sure, further adjustments will be required. But overall, I am happy with the way things are developing. I feel the course is moving in the general direction I wanted to go when I set out. I believe I (and the successive generations of students) are slowly getting there. I just don’t know where “there” is exactly, what “there” looks like, and how far in the future we’ll arrive.

As the man said, “To boldly go …”

MathThink MOOC v4 – Part 5

This post continues the previous four in this series.

In Part 5, I wrestle with grading, and once again expectations raise their troublesome head.

 Like all MOOCs coming out of Stanford, my Introduction to Mathematical Thinking course carries no college credit, nor does it lead to any kind of Stanford certificate. For the first three sessions, students whose aggregate grade was above a certain threshold did receive a Statement of Accomplishment, and if their grade was particularly high, their statement testified that the Accomplishment was “with Distinction”. Those terms were stipulated by Stanford, though it was I, as instructor, who issued them, not my university.

Though some courses on Coursera can, for a small fee, be taken in a fashion that provides a degree of certification that the individual named on the course completion certificate is indeed the person who took the course, that’s not the case for Stanford MOOCs. It is also very much inconsistent with my intent in offering the course, which is to offer a course where the entire focus is on learning, not credentialing.

To me, focusing entirely on learning, and not offering a credential, is particularly suited to a course that aims to provide general purpose thinking skills that can be used in many ways in different walks of life. There are so many ways in which mastery of mathematical thinking can be advantageous in other courses, it can help people get credentials in many subjects, to say nothing of the non-academic benefits it can yield in professional and personal life. In today’s world, mathematical thinking ought to be regarded as on a par with basic literacy, not something to emblaze on a certificate.

In fact, I structured the course to maximize those general benefits, by keeping the mathematical content at a very elementary level for the first six weeks, focusing instead on logical and analytic thinking and the process of bringing precision to issues that are initially vague or ambiguous.

My decision to focus on learning, not the awarding of credit, was also heavily motivated by the fact that, arguably much more than in any other discipline, misguided educational policy has turned mathematics from a creative human endeavor into a relentless and mind-deadening treadmill of test taking. Not in all countries, to be sure, but certainly in the two with which I am intimately familiar as a practitioner, the US and the UK, and many others whose educational systems I am also acquainted with.

The majority of my MOOC students would, I knew, have never encountered a course that focuses on (creative, original) mathematical thinking (as opposed to mastering and applying standard procedures – something that can often be done with almost no thought whatsoever, and indeed can be done far more efficiently these days by apps you can run on your smartphone). So why spoil their first taste of something different, something creative and rewarding, by testing them?

Of course, in a discipline that is about problem solving, each student’s work needs to be evaluated and the results transmitted to them, so they know how they are progressing. But I did not want there to be any more significance attached to those grades than that.

As a person who is fiercely competitive, and does not like to lose, I knew that many would seek to score the highest grades they could on each piece of evaluated work. My hope was that they could approach the course much the same way I and my cycling colleagues approach a race. During the event, no quarter is taken as we all fight to win. But the moment we have all crossed the finish line, the final result ceases to be important. (Okay, it can last until we hit the bar that evening and begin to exchange embellished personal stories of the event. But definitely no longer than that.)

Unfortunately, what works easily for amateur bike racing, does not seem to work for taking a math course. For all our ultra-light, carbon-fiber racing machines and skin-tight lycra, I and my two-wheeled buddies know we are not professional cyclists and our event is not the Tour de France. We do not approach our races with any expectations carried over from previous experience.

With a math course, unfortunately, people do come along with expectations. Though some students successfully managed to focus on the content and the learning thereof, and were not put off by continually getting grades down in the 30 – 40% range (results that I kept stressing were as good as could be expected for anyone who had not encountered this kind of mathematical activity before), many could not make that shift. For them, many years of high stakes testing had turned mathematics into fierce competition to “get an A”, and anything less was “failure.”

Calling my course “Mathematical Thinking” was not enough to counter those expectations, and there was a lot of forum obsessing with grades.

It is, to be sure, a difficult transition to make. (Courses like mine are often called (high school to university) “transition courses.”) To this day, I remember the trauma of going from being an ace at high school, procedural math to being totally lost in my first-year university courses. The main thing that kept me going was the recognition that all my classmates were having the same difficulties. Getting your work back with a 30% at the top is a lot easier to take when everyone sitting in the same room as you is having the same experience – a support mechanism often missing for students in an online course.

The expectations that color how people view course grade-points also affect how they perceive the course certificate. I assumed from the start that many people would attach personal value to the Statement of Accomplishment, even if it has no street value. (On occasion, it appears it does. See the story half way down this article.)

I definitely wanted to make the SoA as meaningful as possible for the person who earns it. My course is difficult, and anyone who completes it should feel proud of what they have done. Accordingly I set the threshold so that approximately 80% of students who completed the course received a SoA, and 20% of those SoAs were with Distinction.

Current platform limitations meant this was not ideal. (The Coursera platform is still under very active development.) Though the instructor was free to specify the algorithm whereby the final course grade was computed from the student grades the system had assigned for each individual piece of work, the only measure the Coursera platform provided on which to make the certificate decisions was that final grade. This meant that some students who did not complete the course, but who scored highly on what they did do, also got certificates. Some of them were not happy to do so.

My response to those was essentially, “Don’t bother to print off the statement then,” though contexed to make it clear I understood why it bothered them.

Still, after enduring for three iterations of the course what for me, given my goals, was a distraction of grades and certificates, at the end of the most recent session I decided to make the award of a SoA in future dependent purely on completing the course.

For students who sign up for my spring session, starting on February 1, SoAs will be awarded for time spent and effort, not level of performance. The grade points awarded for each individual piece of work will cease to have even minimal significance outside the course, not even by way of the SoA.

Students in the spring session will have a choice of two versions of the course. The Basic Course will last for eight weeks, and completion leads to a Statement of Accomplishment. The Extended Course will continue for a further two weeks, devoted to a process I am calling Test Flight, with completion resulting in the award of a Statement of Accomplishment with Distinction.

The Basic Course is designed to develop mathematically-based analytical thinking skills having wide applicability. The additional learning provided by the Extended Course is focused on applying those skills to mathematics itself, in particular building on the earlier analysis of mathematical proof to establish some basic properties of whole and real numbers.

The Statements of Accomplishment will be awarded on essentially a “Pass/Fail” basis, and the certificate will not state a grade.

Both courses will use grades points purely as a metric of progress, not a record of achievement, nor as a criterion for awarding a SoA.

Not quite. I will use grade points in each individual piece of submitted work to determine what constitutes “completion” of the course. To guarantee a SoA, a student will have to submit at least five of the course’s eight, machine-graded Problem Sets,  earning at least 5% for those five Sets. That will determine a lower bound for SoAs.

Again, it means that any student who earns a higher overall grade will get a SoA, even if they complete fewer than five Problem Sets, but as long as everyone knows that system, I have no problem with that. Some commentators say that the lack of a guarantee that the person with a SoA really earned it is a weakness of MOOCs, but that’s true only if you view the goal of education as being evaluation and certification — a problem I have with American education in general.

When Coursera develops a more fine-grained framework for determining the awarding of certificates, I will probably modify the process I use, but frankly I do not regard this as a big issue.

Anyone who “cheats” in my course simply cheats themselves, by not getting the benefit of actually learning. I think education will be a lot better if we separate certification from education. I would have no problem with a third party organization coming along and offering an accreditation service for my course. For sure, an equivalent check will already happen if one of my students uses a SoA from my course to secure a job interview at a large company. The first thing the company is likely to do is ask the applicant to take a short test. As long as that test involves mathematical thinking – and any reasonably well designed test surely will – it will become immediately clear if that individual really did benefit from my course.

I am in the education business, not credentialing. If someone simply wants a SoA for my course, the quickest way to get one is to find an image of a certificate on the Web using Google image search, and use graphics processing software to insert their own name. In doing so, they will be demonstrating useful skill with digital media, of course, but I don’t believe that process will provide them with good mathematical thinking skills.

If you really want mathematical thinking, you need to actually work through my course, using the grades to measure your progress, and stay for the full ten weeks, completing the Test Flight process at the end.

So what’s Test Flight? Tune in next time. Meanwhile, here is a clue.

MathThink MOOC v4 – Part 4

In Part 4, I describe how I try to avoid the educational danger of a well taught class.

This post continues the previous three in this series.

We’re still on the problems caused by the expectations students bring to a course – problems I hope can be alleviated (to some extent) by that new introductory video I talked about in my last post.

Unfortunately, working against any positive effect my pep talk may have is the fact that my MOOC looks like a traditional, instruction-based course.

So too does the regular classroom instantiation of the course, whenever I give it. It’s not.

Neither version of the course is instruction-based. When the twenty-five or so students in a physical class are glowering at me from a few feet away, because I have not shown them in advance how to solve the problems on the assignment, I can patiently explain that their expectations have led them astray, and I can coach them into seeing my lectures for what they are: motivational examples. It’s not about learning how to solve a particular kind of problem, I can tell them. It’s developing the ability to set about solving a completely novel problem. And doing that inevitably involves a lot of failure. For solving novel problems frequently boils down to acquiring the ability to respond positively and constructively to failure.

That kind of direct feedback is not possible in a MOOC. There’s the difference.

Yes, I do use those lectures to provide some initial coverage of some standard material (as well as to do some sample worked solutions). But it’s not pretty. It’s WYSIWOSG – “What You See Is What Our Students Get.” When I record one of those lectures, I imagine a student has come into my office asking for help, and we sit down together at the desk and work through whatever is giving them grief. It is a simulation of two people working together, rather than one giving instruction.

It’s not scripted, I record “as live,” and I leave any mistakes there on the page. The technology is kept deliberately at the low end. (I do sometimes ramp up the speed of my handwriting at the editing stage to match my voice, but interestingly many students tell me they had not noticed that until they read or heard me point it out.)

Because it’s not about the content, you see. It’s all about the thinking. Those lectures are really about the how, not the what.

For anyone who wants to see a polished presentation of the what, the basic factual content, beautifully laid out, there is a short, ultra cheap, self-published (optional) textbook (more accurately, a course companion book) I wrote to accompany the course. Or the student can reconstruct their own polished account based on my lectures, basing them either on brief notes they take during the lecture or by printing off the screen after I have finished the page.

It’s important to see my lectures for what they are: examples of mathematical thinking. They are certainly not provided so the student can learn to replicate me. Each student has to develop mathematical thinking for themself, and it doesn’t have to be identical to mine in all respects.

Of course, the nature of mathematics places significant limitations on what constitutes mathematical thinking. There is an intrinsic limit to how far they can deviate from me or any other mathematician. But in many cases, the range can be much broader than most people realize – certainly so wide that the idea of a “model solution set” makes no sense.

If I were to provide a model solution to a given problem, as students frequently ask for, I am depriving them of one more well constructed novel problem on which to develop their ability to solve new problems. I will have shown them my way of solving it, and for ever after they will be that bit more likely to approach any similar problem my way, instead of developing their own ability.

It’s the same when I record a video of myself solving an assignment problem after I have given the students time to attempt it themselves. (I call those “tutorial videos.”) The actual solution I produce is not important – there are usually other ways to solve the problem, or at least other ways to express the solution.

Notice that I said I solve problems after I have asked the students to try them. I don’t assign problems that are slight variants of ones I have demonstrated. I give them an opportunity to try – and fail – to solve the problems I assign, before I demonstrate one of the possible solutions – again as an example, not “the solution.”

Of course, the whole thing is planned out. It could not be otherwise. But the planning is at a high level, well above the details. A casual viewer would not see my performance as a “well taught lesson.”

At least, I hope they would not, since “well taught lessons” rarely lead to good results, as has been demonstrated by a number of research studies, starting with Alan Schoenfeld’s much-cited 1988 paper When Good Teaching Leads to Bad Results: The Disasters of “Well-Taught” Mathematics Courses.

For students who seek the comfort of a well-presented coverage of the core content, there is my book. Though cheap, it is not free, and that too is deliberate. It is not a part of the course. It’s something that is there for anyone who wants it. There is a (token) cost for getting it. It is a carefully constructed, stand-alone artifact, of value in its own right. Take it or leave it. Consulting books when required is a valuable part of mathematical thinking. But my book is not part of the course; it is not really a course textbook as such, rather it is optional, supplemental reading. A course companion, or a reference, if you will.

The course is about thinking. About doing. About action. In my lectures I am demonstrating what it is like to do mathematics – to generate it within one’s own mind. Reading it is something else.

The videos are an attempt to take the student inside my mind, in real time, to get a sense of what it means to think mathematically. Again, let me stress, not to replicate me. Just to provide an example.

It should be obvious that having pre-prepared pages or slides in the videos, either hand-written or typed, with or without staged reveals, will not work. That approach might help a student learn how to read and comprehend mathematics, but not how to produce it.

That people can in fact acquire the ability to think mathematically from such limited exposure to another person doing so is, in many ways, remarkable. But it’s hardly unusual; countless people have learned this way over thousands of years.

The ingredients of mathematical thinking are already present in the human brain. (See my 2000 book The Math Gene, an account of the evolutionary development of human mathematical capacity.) It just takes a series of suitable triggers to bring those ingredients together. My role as instructor is to make those triggers available. But the student has to pull them and to have a reasonably good aim when doing so.

This is the only way I know how to “teach” mathematics (i.e., mathematical thinking). It’s how I learned.

It’s possible mathematical thinking may not be achievable for everyone, even with a personal tutor. But making the attempt is achievable.

An absolutely key ingredient to any level of success, however, is accepting that failure is part of the process, the focus of Part 2 in this series.

That’s why, having observed how three MathThink MOOC classes have responded to the three versions of the course I have offered, I have decided to make awarding of the course Statement of Accomplishment in future dependent entirely on course completion, not level of performance. I’ll pick up that theme next time.

Step by step I am moving away from features common in traditional, classroom courses. I am also constantly asking myself what exactly is a course, and why do we so often package learning into courses. There too, student expectations play a significant role. Clearly, this blog  is not close to ending. I, for one, am curious to know where it will go next. (Seriously. Like my MathThink course and MOOCs in general, this blog is a work in progress. A living document. Think of it as reflective lab notes.)

MathThink MOOC v4 – Part 3

In Part 3, I describe some aspects and origins of the basic course pedagogy, and how they relate to student expectations.

This post continues the previous two in this series.

Expectations. So far I’ve talked about two expectations many students bring to my MOOC that cause problems:

(1) a perception that learning is a cycle of

instruction –> worked examples –> student exercises

(a process that’s better described as training, not learning), and

(2) a belief that failure is something to be avoided (rather than the essential part of learning that it is).

A third problematic expectation many students bring is based on the assumption that mathematics is a body of knowledge to be absorbed, rather than a way of thinking that has to be learned/acquired/developed. That belief is what can lead to the erroneous, and educationally debilitating, perception of mathematics that what makes it hard to learn is the sheer number of different rules and tricks that have to be learned, as described in the article about Jo Boaler’s work I cited in Part 1 of this series.

The view of mathematics as a large collection of procedures can get you quite a way, which explains the huge success of Khan Academy, which shows you all those rules – thousands of them! But it won’t get you to the stage of thinking like a mathematician. Mastering an array of procedures is fine if you are (1) willing to invest the time to keep learning new tricks and (2) prepared to end up working for someone who can do the latter (i.e., think mathematically). Because, increasingly, in the western world, it is that latter that is the valuable commodity. (I wrote about this back in 2008.) My use of the term “mathematical thinking”, rather than just “mathematics”, to title my course was designed to highlight the distinction, but many students nevertheless come to my MOOC expecting a mathematics course (in the sense they have come to understand that term), and are disappointed to discover that it is nothing of the kind. (Some have even asked why I don’t make it more like Khan Academy, a bizarre request which leaves me wondering why they don’t just enter the KA URL in their browser rather than navigate to my MOOC.)

Based on the kinds of issues I’ve been discussing regarding mathematical thinking, in designing my MOOC (and the classroom course that came earlier), I drew on a number of established pedagogies. Most notably among them is Inquiry-Based Learning. For a general background on this powerful and effective learning method, check out this 21-minute video.

Do please watch this video. The focus of much of the video is producing professional mathematicians, and that reflects a common use of the IBL method in mathematics majors classes. In my course, however, with its focus on general mathematical thinking skills for use in many life situations, I don’t ask the students to act as those in a regular IBL class – that would be impossibly hard to pull off in a MOOC in any case. But I believe the general learning principles apply (perhaps even more so), and some of the comments in the video from people who pursued careers in industry address that aspect.

Another pedagogic strategy I adopt is one that has been used in mathematic education since the time of the ancients, which I usually refer to as the Mr Miyagi Method, after the Japanese martial arts expert in the hit 1984 movie The Karate Kid. Having promised to teach karate to the young American Daniel Larusso, Mr Myagi makes his young student paint a fence, wax a floor, and polish several cars. Only with great reluctance does Daniel acquiesce, but in due course he discovers the value of all that effort, as you see from this brief clip.

As I say, this form of teaching has been used in mathematics for centuries. The reason is that in many cases it is impossible to appreciate how mathematics can be applied in a particular situation until enough of the relevant mathematics has been learned. So you design small, self-contained exercises to develop the individual component abilities. Mathematics textbooks have been doing this since they were written on clay tablets five thousand years ago. It’s what most people experience as “mathematics education.”

An attractive alternative is project-based learning. (Again, please do watch this short video.) Unfortunately, whereas PBL is fine for a regular course, in a MOOC that is designed to be of value both to students working on their own, with few if any additional resources, and to students who just participate in a part of the course, it is not an option. That leaves the Miyagi Method as the only game in town.

Even is a regular classroom, and for sure in a MOOC, I would however strongly recommend not adopting Mr Miyagi’s method of delivery. It would surely have been better (as an educational strategy, though not as a movie scene) if he had first explained to Daniel what those chores had to do with learning karate. If a student has to ask, “Why am I learning this?”, the teaching has failed. Why not tell the student from the start?

But remember, times change, and skills and abilities that were valuable in one era sometimes become far less significant, as we are reminded by another Hollywood blockbuster character, Indiana Jones. So you’d better be sure that when you tell a student why a particular topic is important, the reason you give is plausible. (Note: In today’s world, no one balances checkbooks any more – heavens, most people no longer have a checkbook – and no householder uses geometry to figure out how much carpet to order for a room.)

Turning the failing-as-part-of-learning meme on my own journey of learning how to design and give a MOOC, I think that so far I have definitely failed to make sufficiently clear to my MOOC students (1) the basic goals of the course, (2) the approach I am taking to try to achieve those goals, and (3) how those goals lead to adopting the methods I have just outlined above.

To be sure, I laid everything out in detail in the guide-notes I posted on the course website, and in some of my earlier posts to this blog, that I link to from the course site. The problem was, many students never read everything on the site; indeed, some appear not to have read any of the site information.

Now, you might say, they had an obligation to do so. It’s their education, after all, not mine. But MOOCs are about taking learning to a much wider audience than is reached by traditional higher education, and if a MOOC instructor does not manage to connect to that audience, then that is a failure of mission.

As a result, one change I am making with the new version of the course in February is that one of the first things the students will encounter is a video of me explaining the course pedagogy.

[From the very first offering of the course, I posted video discussions between me and my then course TAs, in which we discussed the course design, but those discussions really only made sense after a student had spent some time in the course. So from the second run onwards I cut them into short segments that were released on the site throughout the course. I suspect those discussions were perceived more as “Charlie Rose type” television conversations, rather than providing key information about how to take the course. (In the second of the two discussions, I even asked a professional television and radio host I know to moderate the discussion.) In any case, they did not have the effect I hope will be achieved by a face-to-face explanation by me, as the instructor, of the course goals and structure, given before the course starts. You can view those two earlier videos at: Team Discussion (8mins), What’s New in Number Two (10min 45sec).]

Will my new introductory video solve the problem? I don’t know. For sure, MOOC students do watch (almost) all the videos. Indeed, if there is a problem, it is that some seem to perceive the videos as the most important component of the course, a perception the news media seem to share. Why is that a problem? Because video instruction (i.e., direct instruction) in fact-based, science disciplines does not work. Indeed, instructional videos do actual harm by re-enforcing any prior-held false beliefs, as Derek Muller explains in this video. (Yup, putting math and science education out as a MOOC is hard!)

My guess is that my new introductory video will have an effect, but it will be limited, and many will still be left feeling confused as the course moves ahead. Unfortunately, since the only tools we have at our disposal in a MOOC are video, text, and social media, I don’t see what more I can do, so my gut feeling at this stage is that with the new video I will have gone as far as the medium allows. Nothing works for everyone. All we can do is design for a feasible maximum.

I’ll say (yet) more on this theme of recognizing, anticipating, and dealing with student expectations in my next post. Based on giving three successive versions of my MOOC now, I think the student expectations issue is much more significant in a MOOC than in a regular class. The reason is that in a MOOC, because you have no direct contact with the students, you have very limited ability to counter or correct or allow for those expectations. Your only real strategy is to identify them, and pre-emptively try to lessen their impact on the student.

great-expectations-posterTRAILER (LOOKS GOOD)

MathThink MOOC v4 – Part 2

In Part 2, I reveal that I share with Steve Jobs, J K Rowling,  Sebastian Thrun, Thomas Edison, and a successful Finnish video-game studio head, a strong belief in the power of failure.

This post continues the one posted two days ago about the expectations students being to my MOOC.

One of the problematic expectations many students bring to my course is that I will show them how to solve certain kinds of problems, work through a couple of examples, and then ask them to solve one or two similar ones. When I don’t do that, some of them complain, in some cases loudly and repeatedly.

There are several reasons why I do not simply continue to serve up the pureed (instructional) diet they are familiar with, and instead offer them some raw meat to chew on.

Most importantly, the course is not about mastering yet more, specific procedures; rather the goal is to acquire a new way of thinking that can be used whenever a novel situation is encountered. Tautologically, that cannot be “taught.” It has to be learned. The role of the “instructor” is not to instruct, but to offer guidance and feedback – the latter being feasible in a MOOC by virtue of most beginners having broadly similar reactions and making essentially the same mistakes.

To progress in the course, the student has to grow accustomed to the way professional mathematicians (to say nothing of engineers, business leaders, athletes, and the like) make progress: learn by failing. That’s the raw meat I serve up: failure.

Not global failure that debilitates and marks an end to an endeavor; rather repeated local failures that lead to eventual success. (Though the distinction is really one of our attitude toward a failure – I’ll come back to this in a moment.)

Most of us find it difficult making the adjustment to regarding failing as an integral part of learning, in large part because our school system misguidedly penalizes (all) failures and rewards (every little) success.

Yet, it is only when we fail that we actually learn something. The more we fail, the better we learn; the more often we fail, the faster we learn. A person who tries to avoid failure will neither learn nor succeed. If you take a math test and score more than 75%, then you are taking a test that is too easy for you, and hence does not challenge you to learn. A score of 75% or more says you did not need to take the test! You were not pushing the frontiers of your current abilities.

I should add that I am not talking about tests and exams designed to determine what you have learned, rather those that are an integral part of the learning process – which in my case, giving a course that offers no credential, means all the “graded” work.

In my course, the numbers the system throws out after a machine-graded Problem Set, or the mark assigned by peer evaluation, are merely indicators of progress. A grade between 30% and 60% is very solid; above 60% means you are not yet at the threshold where significant (for you) learning will take place, while a score below 30% tells you either that you need to put more time and effort into mastering the material, or slow down, perhaps working through the remainder of the course at your own pace then trying again the next time it is offered. (Another great advantage of a free MOOC.)

What is important is not whether you fail, but what you do as a result. As I was working on this post, I came across an excellent illustration in an article in FastCompany about the Finnish video game studio Supercell. Though the young company has only two titles in the market – Clash of Clans and Hay Day – it grossed $100 million in 2012 and $179 million in the first quarter of 2013 alone.

Supercell’s developers work in autonomous groups of five to seven people. Each cell comes up with its own game ideas.  If the team likes it, the rest of the employees get to play. If they like it, the game gets tested in Canada’s iTunes App store. If it’s a hit there it will be deemed ready for global release.

This approach has killed off several games. But here is the kicker: each dead project is celebrated. Employees crack open champagne to toast their failure. “We really want to celebrate maybe not the failure itself but the learning that comes out of the failure,” says Ilkka Paananen, the company’s 34-year-old CEO.

It’s not just in the PISA scores where Finland shows the world it knows a thing or two about learning; you can find it manifested in the App Store download figures as well!

(And let’s not forget that another Finnish game studio, Rovio, produced over a dozen failed games before they hit the global App Store jackpot with Angry Birds.)

Where I live, in Silicon Valley, one of the oft-repeated mantras is, “Fail fast, fail often.” The folks who say that do pretty well in the App Store too. In fact, some of them own the App Store!

One of my main goals in giving my MOOC is helping people get comfortable with failing. You simply cannot be a good mathematical thinker if you are not prepared to fail – frequently and repeatedly. Failing is what professional mathematicians do maybe 99% of the time. Responding appropriately to failure is a key part of mathematical thinking.

And not just mathematical thinking. It’s definitely true of engineering as well. Remember Thomas Edison, who on being asked how he motivated himself to continue his efforts to build an electric light bulb when a thousand attempts had failed, replied (paraphrase), “They were not failures, I just found a thousand ways it won’t work.”

The metaphor I use regularly in my MOOC is learning to ride a bike. If you think about it, you don’t learn to ride a bike; you learn how not to fall off a bike. And you do that by repeatedly falling off until your body figures out how to avoid falling.

Incidentally, the fact that you really did not learn to ride a bike by learning how to is indicated by the fact that almost no one can correctly answer the question, What direction do you turn the handlebars in order for the bike to turn to the right? Your conscious mind, the one that would have been involved if you had learned how to ride a bike, says you twist the handlebars to the right in order to turn the bike to the right. But, if you are able to ride a bike, your body knows better. You turn the handlebars to the left in order to make the bike turn to the right. Your body figured that out when it learned how not to fall down.

Don’t believe me? Go out and try. Make a conscious attempt to turn right by twisting the handlebars to the right. Most likely, your body will prevent you carrying through. But if you manage to over-ride your body’s instinct, you will promptly fall off. So please, do this on grass, not the hard pavement.

Not surprisingly, six weeks in a MOOC is woefully little to adjust to the professionals’ view of failure. The ones who breezed through my course, unfazed by seeing the system return a grade of 30% on a Problem Set, were in most cases, I suspect (and in a fair number of cases that suspicion was confirmed), professional engineers, business people, or others with a fair bit of post-high-school education under their belts. Those for whom the course was one of their first ventures into collegiate education, often had a hard time of it. (Not a few gave up and dropped the course, sometimes leaving an angry, departing post on the class forum page.)

It’s not called a “transition course” for nothing.

I’ll continue this theme of dealing with student expectations in my next post.

Meanwhile, I’ll leave you with three more examples about the power of failing in the learning process.

The first is Steve Jobs’ 2005 commencement address at Stanford.

The second is J. K. Rowling’s 2008 commencement address at Harvard.

Finally, and very close to home, is Sebastian Thrun’s recent business pivot of his MOOC delivery company Udacity, which I discussed in a commentary in the Huffington Post. Though I would agree with the many commentators that his initial attempt had “failed,” where the tone of many was dismissive, I saw just another instance of someone on the pathway to (for him, yet another) success. It’s all about how you view failure and what you do next.

I’ll continue the theme of dealing with student expectations in my next post.

MathThink MOOC v4 – Part 1

In Part 1 of a series, I focus on the distinction between high school math and university-level mathematics, suggesting they are effectively different subjects that are best learned in different ways.

One of the biggest obstacles in giving an online course on mathematical thinking, which my MOOC is, is coping with the expectations students bring to the course – expectations based in large part on their previous experience of mathematics classes. To be sure, prior expectations are often an issue for regular, physical classes. But there the students have an opportunity to interact directly with the instructor on a regular basis. They also have the benefit of a co-present support group of others taking the same class.

But in a massive open online class, apart from locally configured support groups and text-based discussions on the MOOC platform discussion forum, each student is pretty much on her or his own.

The situation is particularly bad for a course like mine, designed to help students transition from high school mathematics to university-level mathematics. For one thing, the two are so different as to be in many ways completely distinct subjects.

School mathematics tends to be almost exclusively procedural, mastering established methods to solve artificially constructed problems designed to be amenable to such an approach. The student who best masters all the techniques in the syllabus and becomes skillful in pattern-matching problems to solution methods, does well. (I know that first hand; it’s how I got to university to study mathematics!)

In contrast, university mathematics is about learning how to deal with a novel situation of a kind you have not encountered before. (If no one else has encountered it, we call it mathematics research.) Though it certainly can involve pattern matching and the application of established, standard procedures, it usually does so only as components of a novel solution you develop to deal with that particular situation. Moreover, at university level, the problems are typically of a “prove that this is true (or false)” variety, rather than “solve this equation” or “compute the value of that formula.”

What is more, while a school math problem typically has a right answer, university mathematics generally involves much more than mere correctness. Indeed, there may not be a unique “right answer.”

Not only is the subject matter different, so too is the pedagogy. Almost all students’ experience of mathematics learning in school is teacher instruction. The teacher describes a method, does a few worked examples, and then asks the students to do a few similar ones. Rinse and repeat.

It’s a very efficient way to cover a lot of ground when the goal is pattern matching and procedure application. It works for school mathematics. Unfortunately, it does not prepare the graduates for the other kind of mathematics. (It also leaves them without ever having a satisfactory answer to their question “What is this good for?”, a question that leaves anyone versed in mathematics astounded. “What is it not good for?” is a more interesting question. It does not have a simple answer, by the way. It’s a very nuanced question.)

It’s like teaching someone the elements of bricklaying, carpentry, plumbing, and electrical wiring, and then asking them to go out and design and build a house. You need all of those skills to build a house, but on their own they are not enough. Not even close.

In deciding, almost two years ago now (before the New York Times had heard of MOOCs) to develop a MOOC to help people learn the other kind of mathematics, what I call mathematical thinking, I knew I was taking on a big challenge. I’d found it hard to teach that kind of course in a physical classroom with just 25, carefully selected students at elite colleges and universities.

On the other hand, most people go through their entire mathematics education without ever encountering what I and my colleagues would call “real mathematics,” and many of them eventually find they need to be able to handle novel situations that involve – or may involve – or could productively be made to involve – mathematical thinking. So I felt there was a need to have a resource publicly available to help them acquire this valuable ability.

The huge dropout rates in MOOCs did not really bother me. For a mathematical thinking course, it’s possible to gain value from dropping into the course for just a few days – and to keep coming back at future times if required. The focus was not on credentialing, it was developing a valuable mental ability – a powerful way of thinking that our ancestors have developed over three thousand years.

That way of thinking can be utilized profitably in many other courses that do yield a certified credential, so students could approach the course as a low-stress, no-risk way of preparing for subsequent learning.

The course is structured as course for those students who seek an encapsulated experience, and in many ways that yields the greatest benefits, in large part because of the interactions with other students working on the same stuff. But the majority of students who have taken it the three times I have offered it have just taken a part of the course.

Each time I gave the course, I changed it, based on what I had learned. When it launches again in February, it will be different again. This time, in some fairly significant ways. In the coming days, I’ll describe those changes and why I made them.

First out of the gate, I’ll describe what exactly were the problems caused by those expectations many students brought to the course, and  how did I try to deal with them. Also, what am I changing in the coming version of the course to try to help more people make what is a very difficult transition: from being taught (i.e., instructed) to being able to learn. The reward for making that one transition is huge. It opens up all of mathematics, and in the process makes it much, much easier.

The traditional, instructional way of teaching procedural mathematics frequently leaves students with the impression (dramatically documented by my Stanford colleague Jo Boaler) that mathematics consists of a large number of rules to be learned. But at the risk of sounding like those weird web advertisements (you know, the ones with a drawing or photo of a strange looking person) promising to teach that “one great trick” that will change your life, let me leave you by telling you the one great trick that all mathematicians learn:

You just have to master, once, a particular way of thinking, and you no longer need all those rules.

That’s what my course focuses on. Stay tuned.

The MOOC Express – Less Hype, More Hope

A real-time chronicle of a seasoned professor just about to launch the fourth edition of his massively open online course.

???????????????????????????????

Last week, I headed off to Arlington, Texas, to participate in a large, international conference on MOOC education, part of the Gates Foundation funded MOOC Research Initiative (MRI). While the founders of the big, massively-funded American MOOC (“MFAM”) platforms Coursera, edX, Udacity, and Novo Ed capture most of the media’s attention, this conference was led by the small band of far less well known Canadian online-education pioneers who actually developed the MOOC concept some years earlier, in particular George Siemens and Stephen Downes who organized and ran the first MOOC in 2008, and David Cornier who forever has to live with having coined the name “MOOC”.

(There were so many Canadians in Arlington, they brought their own weather with them, as you can see from the photograph. The conference ended with participants having to change flights and book additional nights in the conference hotel, as a severe ice storm hit the area. With a return flight that happened to lie within a brief lull in the storm, I managed to get out on time, though only after a slip-sliding taxi ride at a snail’s pace to get to DFW Airport. Others had it a lot worse.)

While teams of engineers in Silicon Valley and Cambridge (MA) are building out MOOC platforms that provide huge opportunities for massive scale-up, the two hundred or so researchers and educators who came together in Texas represent the vanguard of the educational revolution that is underway. If you wanted to know what MOOC learning might look at in a few years time, you would have better spent your time at the University of Texas in Arlington last week rather than in Silicon Valley. In his October announcement of the conference, organizer Siemens described it as “the greatest MOOC conference in the history of MOOCs,” a bit of satirical hype that is almost certainly true.

You need both, of course, the technology to reach millions of people and the appropriate, quality pedagogy. For example, the movie industry required a series of major advances in motion picture technology, but fancy film cameras alone are not what gave us Hollywood. As the technology advanced, so too did the art and craft of motion picture writing, acting and directing. Similarly with MOOCs. The focus in Arlington was on the educational equivalent of the latter, human expertise factors.

To pursue the movie analogy a bit further (and with that great spoof movie poster as a visual aid, how could I resist?), the MOOC action that gets reported in the Chronicle of Higher Education and the breathless (but hopelessly off-base and over-hyped) prose of Thomas Friedman in the New York Times is the equivalent of taking an early movie camera into a theater and recording what happens on the stage.

If you want to see the future of MOOCs, you need to hang around with the instructors in the lesser known, small universities and community colleges who, for many years, have been experimenting with online learning. Most of the leaders of that loosely knit band could be found in a large, cavernous room in Arlington for two days last week – along with key figures from such new-MOOC, Ivy League players as Stanford, MIT, and Georgia Tech.

What was the take-home message? In my case, and I suspect everyone else’s, confirmation that we really don’t know where the MOOC train is taking us. The problem is not an absence of good ideas or useful leads; rather the opposite. Don’t expect a “Conference Proceedings” volume any day soon. The best summary you will find is probably the conference Twitter stream (hashtag #mri13).

The fact is, there isn’t even a clear definition of what a MOOC is. The common classifications of c-MOOC (for the original, Canadian, connectionist animal) and x-MOOC (for the later, scalability-focused, Stanford version) don’t help much, since many (most?) of the MOOCs being developed now have elements of both. Commenting on the lack of a generally accepted definition in his welcoming remarks, conference organizer George Siemens opined that it may be that no agreed definition will emerge, and that the term “MOOC” will be similar to “Web 2.0″, remaining an undefined term that “reflect[s] a mess of concepts that represent foundational changes that we don’t yet understand, can barely articulate, but that will substantially impact the system.”

Actually, I should modify my remark above that Stanford is a MOOC player. That’s not really accurate. Some Stanford scholars (including me) are developing and offering MOOCs, but the main focus at my university is research into learning in a digital age, including online learning. There is a lot of such research, but MOOCs are just one part of it. And those MOOC platform providers you keep reading about, Udacity, Coursera, and Novo Ed? They are all private companies that span out of Stanford. They are now (and  were from their creation) Silicon Valley entities, outside the university. Stanford’s original, in-house MOOC platform, eventually called Class2Go, was recently folded into edX, to be run by MIT and Harvard as an open source platform.

As the MOOC research train goes forward, don’t expect the big, Ivy League universities to exclusively dominate the leading edge of the research. The reason so many participants in the Arlington conference came from far less wealthy, and often smaller institutions is that they have been developing and using online education for years to cater to their geographically-diverse, economically-diverse, and education-preparedness-diverse student bodies. The connections made or strengthened at the Arlington conference are likely to result in many collaborations between institutions both big and small, from the wealthy to the impoverished. The Digital World is like that. When it comes to expertise, today’s academic world is just as flat as the economic one – as Siemens clearly realized in drawing up his conference invitation list.

Expect to see MOOC pioneer Stephen Downes leading a lot of the action, as the head of a new, $19 million, 5-year R&D initiative for the Canadian National Research Council. Keep your eyes too on Siemens himself, who at the conference announced his move from Athabacsa University in Canada to be a professor in, and executive director of, the  University of Texas at Arlington’s new Learning Innovations & Networked Knowledge (LINK) Research Lab. (So new, it doesn’t have a website yet!) Track conference keynoter Jim Groom, who whiled away the time holed up at snowed-in DWF Airport to draft this summary of the gathering.  And don’t overlook the University of Prince Edward Island, where Dave Cormier and Bonnie Stewart brave the weather of the far North East. There are a whole host more. Check the list of recipients of MRI grants for other names to follow, and remember that this is just the tip of, dare I say it under the circumstances, the iceberg.

And never, ever forget to read the excellent (and highly knowledgable) ed tech commentator Audrey Watters, who was invited to attend the Arlington conference but had a scheduling conflict. New readers interested in MOOCs, start here.

Recent headlines may have given the impression that MOOCs were a short-lived bubble, and the experiment has largely failed. Nothing could be further from the truth. For one thing, even the much reported business pivot of Udacity was a familiar event in Silicon Valley, as I argued in a commentary in the Huffington Post.

As Siemens put it in his conference welcome, “the ‘failure’ of MOOCs is a failure of hype and an antiquated notion of learning,  not a failure of open online learning.”


I'm Dr. Keith Devlin, a mathematician at Stanford University. I gave my first free, open, online math course in fall 2012, and have been offering it twice a year since then. This blog chronicles my experiences as they happen.

Twitter Updates

New Book 2012

New book 2011

New e-book 2011

New book 2011

December 2014
M T W T F S S
« Nov    
1234567
891011121314
15161718192021
22232425262728
293031  

Follow

Get every new post delivered to your Inbox.

Join 578 other followers

%d bloggers like this: