Automation of teaching

For my first post, I am not going to talk directly about MOOCs as such.  Rather, I want to highlight a recent study concerning a language-learning web site called Duolingo. It is claimed that "A team of independent researchers recently found that students on Duolingo take 34 hours to learn as much as a one-semester university course". (link to study)

What I find pertinent about this, relative to MOOCs, is the following.  MOOCs are touted as beneficial because they can automate instructional tasks previously undertaken by human beings.  The MOOC, then, is the latest in a long line of technological innovations that reduce or even eliminate human effort.

That said, it is arguable that there are aspects of teaching that are not amenable to automation.  My own hunch is that automated teaching is essentially an application of artificial intelligence.  Since AI is not (at least not yet) coextensive with human intellectual capability, it seems that certain tasks remain that are not amenable to automation, where a human teacher still has a valuable role to play.

We might welcome, then, a technology that allows the aspects of teaching that can be automated to be automated so that we might have more time and energy for the aspects where a human still ought to be in the loop.  Considering again Duolingo, while it might be able to replace human instruction for the first semester (or year) of reading and writing practice, what it does is open up an opportunity for such courses to increase the emphasis on conversation, discussion, creative writing, and even tentative exploration of literature.  I suspect the situation might be similar with MOOCs across a variety of courses.

Are MOOCs a Real Challenge to the Residential Colleges and Universities?

I Just finished the five-week long MOOCs course titled “E-learning and Digital Cultures” taught by five instructors at the University of Edinburgh. This was my first MOOC experience and it was eye-opening and thought-provoking.

A total of 42874 participants registered for this course. However, it’s reported that only 17% of the participants were active in the last seven days before the course was over. About 60% of the respondents came either from “teaching and education” or reported themselves to be “students”. Over 60% of the entire respondent group has postgraduate level qualifications, and a further 35% have a university or college degrees. 

Course contents included video clips and reading materials. There was discussion forum for the topic covered each week. Participants were also encouraged to conduct discussions through Facebook and Twitter. Additionally, the instructors also organized a couple of “hangouts” through Google+.

The final assessment for this course included a digital artefact and assessment of three artefacts of other participants.  

Reflections on this MOOC Experience.

Some years ago, when Internet-based distance education was getting popular, a lot of discussions were conducted on the impact of distance education on physical campuses. The emergence of MOOCs has aroused similar discussions and concerns.

Apparently MOOCs have both promises and pitfalls for disseminating knowledge. Promises include its potential to reach unlimited number of participants and unlimited boundaries; the second promise many of the MOOCs are “taught” by the renowned professors from the elite institutions; the third one is the any-time, any-place flexibility compared to face2face courses.

 

Yet, there are many problems MOOCs have to address. One, the quality of the MOOC education. Because of the large number of participants, it’s almost impossible for an individual to get attention of the course professors. In the course I took, many participants complained of the limited presence of the course instructors. As a matter of fact, the instructors did not participate in the discussions at all, and even they did, their posts would be buried by the hundreds of other posts. The second pitfall of MOOCs, as I see it, is the validity of the course evaluation. This problem also has something to do with the big number of participants. As far as the course I took, the evaluation assignments did not seem to be closely relevant with what was covered in the course. It seemed that even someone who had not taken the course, he/she could still make a digital artifact and assess other’s artifact. The third problem is the high rate dropout. This problem may have a lot to do with the fact that many participants took the course merely for gaining an experience rather than credit or certificate. Nevertheless, students’ motivation is a critical factor for the successful completion of the course.

Implications of MOOCs for ACS Institutions

 Finally, I want to raise the following questions to my colleagues for discussion. What’s the impact of MOOCs to residential colleges and universities? Are MOOCs a real threat to residential colleges and universities such as our ACS institutions? What can we, as a consortium, do to meet the challenge posed by MOOCs and adapt to new technology that drives MOOCs?

What are the advantages of MOOCs?

"We had MOOCs back in my day.  They were called 'books.'"

A recent visiting speaker at Trinity University said the above to me.  And to first order, it's true.  Until very recently, if you were motivated to learn a subject on your own, you turned to books.  Books are self-pacing, portable, and adjust to your time schedule.  And in the infancy of MOOCs, you can find a greater variety of approaches and skill levels in books for just about any subject.  So what advantages do MOOCs offer at the moment?  I'm listing these from least to most persausive, in my opinion.

Some People Learn Better By Listening -- I've heard this in the past, but the two times I've mentioned it to psychology faculty, they've bitten my head off.

Technology Makes Everything More Fun -- In "The Simpsons," Bart won't do yard work, but will wait in line to play with the Yard Work Simulator.  I can see the same idea working here, but surely the shine would wear off after several weeks of work.

Interaction With Instructor -- Message boards should allow you to get questions answered directly by the instructor or teaching assistants.  Unfortunately the signal-to-noise in the classes I've observed has been horrible, with a couple of students turning the boards into their personal blogs.  (Digression:  Is this a science-specific issue?  Do the humanities have an equivalent to the "Einstein is WRONG!!" stereotype?  Do they get regular, mammoth letters outlining how Shakespeare's plays could be decoded to reveal their authorship by a Belgian farmer predicting the end of the world?)

Feedback -- My favorite feature of MOOCs so far has been end-of-video quizzes, which help cement the ideas you've just learned and explain the right answers when you choose incorrectly.  Books typically do a lousy job of this.  If they have exercises, often they put a subset of answers in the back with no explanation.  But there's nothing about the platform of books that prevents them from having this feature.

Multimedia -- Now we're getting somewhere.  MOOCs can supplement their instruction with sound, video, and animation, such as in The Mechanical Universe, probably the best example of this I've ever seen.  But this content takes a lot of effort to create, and the course in which I am now enrolled uses almost none.  And as I mentioned in my last post, I haven't come upon a concept yet where I felt I needed such visual cues to understand it.

Reward Structure -- People are more motivated when there is a reward at the end, especially one more tangible than self-improvement.  A friend of mine is participating in a weight-loss program wherein you pay money, and get double the money back if reach a target weight in a certain time frame.  Even though he knew the benefits of losing weight, he told me he didn't think he could do it without the added incentive of having money on the line.  As it pertains to MOOCs, I am astounded at how much emotional investment people place in their completion certificates.  My course is two weeks behind due to the instructor's illness.  He still intends to end the course officially on March 11th, but will post all the delayed content after that date so people don't miss anything.  The boards are rife with complaints that they would rather work faster and complete it all by the 11th so that their certificates mean more.  Um...they don't mean anything now.

Activation Energy -- With books, you have to search for the right one, and either plunk down some money on Amazon or drive to your local library.  Not exactly an impulse buy.  But with MOOCs, if I were to page through the Coursera offerings and find something that I liked, I could sign up right now.  No cost, no investment, and nobody looking over my shoulder if I decide it's not right for me and drop it later.  And since I have the course open in another window...
 
Multitasking -- I could listen to the content while I finished this blog post.  And that is what prompted the quote that began this post; the speaker's daughter was enrolled in MOOCs, and would listen to them while writing emails or checking Facebook, but always while her attention was divided.  Can't really do that with a book.

So what this stream-of-consciousness has led me to is the following:  while there are several potential advantages to MOOCs, as currently constructed they can encourage bad study habits.  It's too easy to think we're learning when we're not, because our mind is elsewhere.  If we become too sorely challenged, we can drop the course and move onto the next one, because what has been lost?

Professor Stonedahl mentioned a couple of his bad habits in the previous post, so let me add my own.  I am really enjoying Galaxies and Cosmology, and have been filling a lot of holes in my astronomy knowledge.  But I worry about whether I will remember much of it once the course ends.  Right now I find it easiest to complete the course by setting aside a 2-hour block once a week, devouring a week's worth of videos in rapid succession, and then taking the weekly quiz (one attempt only in this course) while the information is fresh in my mind.  It's working; I haven't scored below 90% yet, though there are some lucky guesses in there.   But is this the best way to store this knowledge in my long-term memory?  Already I feel like the early chapters are slipping away.

The point -- when we teach face-to-face, we discuss study habits with our students all the time.  Read the material before you come to class.  Don't start the homework at the last minute, but spread it out so that you are reviewing concepts more regularly.  Learn through practice.  In MOOCs, where we don't have as direct an interaction with our students, we need to think how to structure them to make bad habits harder to form in the first place.

The blessing and the curse of self-pacing

I am experiencing first-hand the blessing and the curse of self-pacing.   But before I elaborate on that theme, permit me to introduce myself to the blog audience. 

My name is Forrest Stonedahl, and I am an assistant professor of Computer Science and Mathematics at Centre College, in Danville, Kentucky.  I became interested in the MOOC phenomenon last summer, and am currently the (proud?) possessor of two certificates attesting to my quasi-academic achievement in two Coursera courses last fall.  Specifically, I completed the same computer science course on "Functional Programming in Scala" that Mark Lewis of Trinity has previously blogged about, as well as a marketing course on "Gamification".  I found the computer science course to be intellectually stimulating and enjoyable - even though most of the concepts were already familiar to me, it's always fun to learn a new computer language, and I had been meaning to learn Scala for some time.  That course gave me the excuse (and extrinsic motivation) to actually dig in and do it. I'll probably post some more about my experience with this course later, to provide another perspective to complement Mark's view of the course.

With the marketing course,  I "attended" most of the first week's video lectures after which point I decided that the course was rather dull, and I wasn't learning much.  Now, I could have dropped the course at this point, but instead I decided to try an experiment to see if I could pass the course while doing as little work as possible, and without watching any more of the course videos.  I became, "virtually", one of those students who never comes to class except on the exam days.  (We don't tolerate such behavior here at Centre College, but I know it can happen at some larger universities...).  A quick text search of the lecture notes each week would provide the definitions needed to pass the multiple-guess quizzes (and the quizzes could be repeated to get higher scores -- thus, I came close to acing most of them).  There were also a couple of peer-graded writing assignments, but I am perfectly capable of expounding on a theme while injecting some appropriate marketing jargon here and there.  The experiment's result:  I ended the course with an overall score of 92.5%, despite having learned very little.  Is this a failure of MOOC-based learning?  Perhaps.  But I rather suspect my experience would have been even more abysmal if I had been forced to sit through the physical version of the class at UPenn.  You are free to choose among a few possible morals of this story:

1. An intelligent but uninterested person can still pass a MOOC with relatively little effort, and little educational gain.  While this may de-value the reputation of MOOCs, the same observation applies to some brick and mortar courses as well. 
2. Despite Coursera (et al's) attempts to court only the best faculty at the best universities,  occasional courses can be uninspiring.
3. I personally am not a good candidate for business school -- probably some people found the course scintillating.  But in my defense, I came into the course thinking that the *idea* of "gamification" is very interesting.  And I still do.  I just don't think that this course on the topic was very interesting.  In fact, I could imagine myself teaching a short course on the subject with a small group of energetic undergraduates where we actually spend time working on projects to *implement* gamification ideas in a real context.   That would be interesting.  But I digress.

But the fact that I continually digress rather than moving the primary narrative forward provides a nice segue back to the title topic: self-pacing.  Despite (or perhaps because of?) decades of formal education, I still have a incurable habit of procrastination.  I am currently enrolled in Udacity's course CS271 on Artificial Intelligence.  Yes, this is the famous class taught by Sebastian Thrun and Peter Norvig that launched a thousand metaphorical ships (or autonomously driving cars?) into the shifting landscape of higher education.  It's being re-offered again, this time in a self-paced format. I can feel Udacity cheerily reassuring me: Take as long as you want, Forrest.  We'll be here for you.  There are no deadlines.  No stress.  Just drop in and learn at your own pace.

And I would like to.  Honestly, I would.  Although I have a background in A.I., I'm excited about learning this material which focuses specifically on the A.I. techniques useful for programming self-driving vehicles.  Furthermore, I'm currently teaching a course on A.I.and I am interested to see if there are any topics that I can tie in to my own class.  However, I've only made it through one week of this course that I started almost a month ago.  The road to procrastination is paved with good intentions.  And with a busy spring teaching term incessantly pushing new deadlines to the top of my to-do list, I just haven't found the time to make progress.  However, I feel pretty certain that if this course was like the Coursera courses I've taken, with weekly homework and quiz deadlines, I would have found (or made) the time.   As it is, the temporal gap widens, and the knowledge I gained in week 1 may already be fading, and becoming stale as it creeps toward the dark recesses of my memory.

Food for thought: If even those of us who have a love for the subject and pride ourselves on independence and self-learning require extrinsic punishment/reward to pace the course appropriately, how will our less motivated students fare in such an environment?
 The blessing of self-pacing is that it's flexible -- you can fit the coursework around your busy work/life schedule.

The curse of self-pacing is that it's flexible -- you might not fit the coursework around your busy work/life schedule.

"Piled Higher and Deeper" #1388 by Jorge Cham. Used with permission.

Is it human nature, or do the students (and I?) just need to learn better self-discipline and time prioritization skills?  My take is that while self-pacing sounds liberating in theory, in practice I think it will prove ineffective for most students.  Agree?  Disagree?  Either way I'd love to hear from you -- comment below!

Galaxies and Cosmology -- First Experiences

(Posted by Dennis Ugolini, Associate Professor of Physics and Astronomy at Trinity University)

I am four weeks into the Coursera course "Galaxies and Cosmology," taught by Prof. George Djorgovski of the California Institute of Technology.  Fun fact:  I received my BS in Physics from Caltech in 1995, and was enrolled in Ay 21 (the basis for the online course) as a sophomore, twenty years ago with the same instructor.  Note that I said enrolled -- I had forgotten until recently that I dropped Ay 21 after a couple of weeks, having decided instead to take more optics and become a pure experimentalist.  So while the symmetry isn't perfect, at least the material is new to me, so I'm paying more attention.

The course is nine weeks long.  Each week's material is grouped thematically into two "chapters;" for example, the Week 1 chapters were History of Cosmology and Introduction to Relativity.  Lecture content is spread among 6-11 videos, each 8-12 minutes long, with an occasional multiple-choice question or two to reinforce concepts.  Most of the screen is filled with a PowerPoint slide (downloadable separately as a .pdf file), with Dr. Djorgovski shown at the upper left, seated and speaking to the camera, and the Caltech and course logo in the lower left.  The course website also recommends three textbooks and provides links to additional reading for each lecture. 

You get one point per video for viewing it, and up to ten points on a multiple-choice quiz each week.  Passing is 60%.  Watching every video and randomly guessing on each quiz gives you an expectation value right on 60%, while acing the quizzes but skipping the videos makes passing impossible.  So the course structure encourages you to (a) view the material and (b) just learn something along the way.

There is a discussion forum, as well as a teaching assistant who holds occasional "office hours," but I have not sat in on these yet.  At first the forums were dominated by people introducing themselves and forming study groups, and complaints about the course structure, particularly that you could only attempt each quiz once.  Oddly, most complaints came from people who scored 8/10 or close to it, but wanted to try again until they got it perfect.  Since then the forums have evolved towards questions on the material, with frequent answers from Dr. Djorgovski and the TA.  I was amused by the reaction to Dr. Djorgovski's note that the forums were not the place to discuss alternative theories for the origin of the universe, and that users who persisted in that would be banned.  Many people became frightened to ask questions, for fear that their misunderstandings would be construed as advancing alternative ideas.  Clearly they've never had to wade through a dozen "Einstein is Wrong!" posters at a conference.

Some thoughts after the first four weeks:

1.  The content is an abbreviated version of the Caltech course, but definitely not simplified.  There's a lot of calculus and some assumption of basic astronomy knowledge, and the pace is rapid.  But the quizzes cover only broad concepts with little math.  The idea again seems to be that they want you to learn something, and beyond that, you can go as deep as you'd like.

2.  I find that the concepts I remember the best are the ones addressed in the multiple-choice questions that interrupt each video.  No matter how simple the questions are, the fact that I had to stop and think, even for a moment, is reinforcement enough.

3.  I've toyed with creating video content in the past (for research-related outreach), but never followed through because I was worried about production values.  How much effort will it take to get fancy new graphics for each lecture?  Flying equations ala The Mechanical Universe?  Music?  This course has none of those things -- and I find I don't miss them like I thought I would.

4.  We only covered one chapter in week 4 because Dr. Djorgovski has the flu (feel better!).  Wait, the full course isn't already in the can?  He's making these on the fly as we go along?  That surprised me.

My interest in MOOC's is in providing a content bridge between high school and college, so that students with poor preparation (particularly in math) can catch up during the summer after senior year and arrive on campus ready to succeed in their intro classes.  I've seen too many students in physics and engineering doomed right out of the gate to a possible fifth year because they did not have the mathematical background to get through Mechanics.  So the most useful thing I've taken so far from this exercise are techniques for structuring a course that has just enough evaluation to make sure you have a pulse, while helping you get out of the course however much you put into it.

Additional Posts

One of the participants in our early efforts looking at MOOCs put the posts on her own blog.

http://moocsrus.blogspot.com/

Statistical Analysis of the Functional Programming Principles in Scala MOOC

(Post by Mark Lewis, Associate Professor of Computer Science at Trinity University)

It has now been a few weeks since my first MOOC completed. At the end of the course they did a survey asking for various information to help them plan potential future courses. Today I got an e-mail that mentioned they had done some analysis on the nearly 7500 responses to that survey. The following link was included in the e-mail.

http://docs.scala-lang.org/news/functional-programming-principles-in-scala-impressions-and-statistics.html

I haven't heard of any other MOOCs doing this type of analysis or making this type of data available. Granted, part of the reason they are making it available is so that students can get further experience. That doesn't really make sense for anything that isn't CS related and of a sufficient level of difficulty.

Two things that really jump out to me from this data are that this MOOC was a standout in terms of completion and it had an interesting population of students. These are inevitably correlated. Most MOOCs have a completion rate of about 10%. This one had a completion rate of 20%. I know that many people point to these low completion rates as a sign that MOOCs don't work well. However, I think they are part of the fact that MOOCs work very well at doing something that other forms of education simply can't do, they make learning highly available to anyone who is interested, even if they aren't willing to put in the full effort to finish a course.

The reason for the high completion rate is almost certainly related to the student population of this particular MOOC. The survey respondents showed a very highly educated group of students with the largest subgroup being those who have completed a masters degree, followed by those who have completed a bachelors degree. Basically, the people who finished this course, were doing it as continuing education and the majority had strong backgrounds to build on. My guess is that this isn't as strongly the case for other MOOCs and that is why they tend to have lower completion rates.

The blog post highlights some other interesting aspects of this course and how it was run. On the whole, I think this course is probably an early model for a highly successful MOOC. Those who want to use this type of structure for education should probably look to this particular offering as a model to base their efforts off of.