This post continues the previous four in this series.
In Part 5, I wrestle with grading, and once again expectations raise their troublesome head.
Like all MOOCs coming out of Stanford, my Introduction to Mathematical Thinking course carries no college credit, nor does it lead to any kind of Stanford certificate. For the first three sessions, students whose aggregate grade was above a certain threshold did receive a Statement of Accomplishment, and if their grade was particularly high, their statement testified that the Accomplishment was “with Distinction”. Those terms were stipulated by Stanford, though it was I, as instructor, who issued them, not my university.
Though some courses on Coursera can, for a small fee, be taken in a fashion that provides a degree of certification that the individual named on the course completion certificate is indeed the person who took the course, that’s not the case for Stanford MOOCs. It is also very much inconsistent with my intent in offering the course, which is to offer a course where the entire focus is on learning, not credentialing.
To me, focusing entirely on learning, and not offering a credential, is particularly suited to a course that aims to provide general purpose thinking skills that can be used in many ways in different walks of life. There are so many ways in which mastery of mathematical thinking can be advantageous in other courses, it can help people get credentials in many subjects, to say nothing of the non-academic benefits it can yield in professional and personal life. In today’s world, mathematical thinking ought to be regarded as on a par with basic literacy, not something to emblaze on a certificate.
In fact, I structured the course to maximize those general benefits, by keeping the mathematical content at a very elementary level for the first six weeks, focusing instead on logical and analytic thinking and the process of bringing precision to issues that are initially vague or ambiguous.
My decision to focus on learning, not the awarding of credit, was also heavily motivated by the fact that, arguably much more than in any other discipline, misguided educational policy has turned mathematics from a creative human endeavor into a relentless and mind-deadening treadmill of test taking. Not in all countries, to be sure, but certainly in the two with which I am intimately familiar as a practitioner, the US and the UK, and many others whose educational systems I am also acquainted with.
The majority of my MOOC students would, I knew, have never encountered a course that focuses on (creative, original) mathematical thinking (as opposed to mastering and applying standard procedures – something that can often be done with almost no thought whatsoever, and indeed can be done far more efficiently these days by apps you can run on your smartphone). So why spoil their first taste of something different, something creative and rewarding, by testing them?
Of course, in a discipline that is about problem solving, each student’s work needs to be evaluated and the results transmitted to them, so they know how they are progressing. But I did not want there to be any more significance attached to those grades than that.
As a person who is fiercely competitive, and does not like to lose, I knew that many would seek to score the highest grades they could on each piece of evaluated work. My hope was that they could approach the course much the same way I and my cycling colleagues approach a race. During the event, no quarter is taken as we all fight to win. But the moment we have all crossed the finish line, the final result ceases to be important. (Okay, it can last until we hit the bar that evening and begin to exchange embellished personal stories of the event. But definitely no longer than that.)
Unfortunately, what works easily for amateur bike racing, does not seem to work for taking a math course. For all our ultra-light, carbon-fiber racing machines and skin-tight lycra, I and my two-wheeled buddies know we are not professional cyclists and our event is not the Tour de France. We do not approach our races with any expectations carried over from previous experience.
With a math course, unfortunately, people do come along with expectations. Though some students successfully managed to focus on the content and the learning thereof, and were not put off by continually getting grades down in the 30 – 40% range (results that I kept stressing were as good as could be expected for anyone who had not encountered this kind of mathematical activity before), many could not make that shift. For them, many years of high stakes testing had turned mathematics into fierce competition to “get an A”, and anything less was “failure.”
Calling my course “Mathematical Thinking” was not enough to counter those expectations, and there was a lot of forum obsessing with grades.
It is, to be sure, a difficult transition to make. (Courses like mine are often called (high school to university) “transition courses.”) To this day, I remember the trauma of going from being an ace at high school, procedural math to being totally lost in my first-year university courses. The main thing that kept me going was the recognition that all my classmates were having the same difficulties. Getting your work back with a 30% at the top is a lot easier to take when everyone sitting in the same room as you is having the same experience – a support mechanism often missing for students in an online course.
The expectations that color how people view course grade-points also affect how they perceive the course certificate. I assumed from the start that many people would attach personal value to the Statement of Accomplishment, even if it has no street value. (On occasion, it appears it does. See the story half way down this article.)
I definitely wanted to make the SoA as meaningful as possible for the person who earns it. My course is difficult, and anyone who completes it should feel proud of what they have done. Accordingly I set the threshold so that approximately 80% of students who completed the course received a SoA, and 20% of those SoAs were with Distinction.
Current platform limitations meant this was not ideal. (The Coursera platform is still under very active development.) Though the instructor was free to specify the algorithm whereby the final course grade was computed from the student grades the system had assigned for each individual piece of work, the only measure the Coursera platform provided on which to make the certificate decisions was that final grade. This meant that some students who did not complete the course, but who scored highly on what they did do, also got certificates. Some of them were not happy to do so.
My response to those was essentially, “Don’t bother to print off the statement then,” though contexed to make it clear I understood why it bothered them.
Still, after enduring for three iterations of the course what for me, given my goals, was a distraction of grades and certificates, at the end of the most recent session I decided to make the award of a SoA in future dependent purely on completing the course.
For students who sign up for my spring session, starting on February 1, SoAs will be awarded for time spent and effort, not level of performance. The grade points awarded for each individual piece of work will cease to have even minimal significance outside the course, not even by way of the SoA.
Students in the spring session will have a choice of two versions of the course. The Basic Course will last for eight weeks, and completion leads to a Statement of Accomplishment. The Extended Course will continue for a further two weeks, devoted to a process I am calling Test Flight, with completion resulting in the award of a Statement of Accomplishment with Distinction.
The Basic Course is designed to develop mathematically-based analytical thinking skills having wide applicability. The additional learning provided by the Extended Course is focused on applying those skills to mathematics itself, in particular building on the earlier analysis of mathematical proof to establish some basic properties of whole and real numbers.
The Statements of Accomplishment will be awarded on essentially a “Pass/Fail” basis, and the certificate will not state a grade.
Both courses will use grades points purely as a metric of progress, not a record of achievement, nor as a criterion for awarding a SoA.
Not quite. I will use grade points in each individual piece of submitted work to determine what constitutes “completion” of the course. To guarantee a SoA, a student will have to submit at least five of the course’s eight, machine-graded Problem Sets, earning at least 5% for those five Sets. That will determine a lower bound for SoAs.
Again, it means that any student who earns a higher overall grade will get a SoA, even if they complete fewer than five Problem Sets, but as long as everyone knows that system, I have no problem with that. Some commentators say that the lack of a guarantee that the person with a SoA really earned it is a weakness of MOOCs, but that’s true only if you view the goal of education as being evaluation and certification — a problem I have with American education in general.
When Coursera develops a more fine-grained framework for determining the awarding of certificates, I will probably modify the process I use, but frankly I do not regard this as a big issue.
Anyone who “cheats” in my course simply cheats themselves, by not getting the benefit of actually learning. I think education will be a lot better if we separate certification from education. I would have no problem with a third party organization coming along and offering an accreditation service for my course. For sure, an equivalent check will already happen if one of my students uses a SoA from my course to secure a job interview at a large company. The first thing the company is likely to do is ask the applicant to take a short test. As long as that test involves mathematical thinking – and any reasonably well designed test surely will – it will become immediately clear if that individual really did benefit from my course.
I am in the education business, not credentialing. If someone simply wants a SoA for my course, the quickest way to get one is to find an image of a certificate on the Web using Google image search, and use graphics processing software to insert their own name. In doing so, they will be demonstrating useful skill with digital media, of course, but I don’t believe that process will provide them with good mathematical thinking skills.
If you really want mathematical thinking, you need to actually work through my course, using the grades to measure your progress, and stay for the full ten weeks, completing the Test Flight process at the end.
So what’s Test Flight? Tune in next time. Meanwhile, here is a clue.
MOOCtalk 