Thoughts on Our First Panel Discussion

(Post by Mark Lewis, Associate Professor of Computer Science at Trinity University)

On Monday 11/12/2012 we had the first panel discussion. The three faculty members at Trinity who are taking MOOCs this semester formed the panel. Attendance was rather good with most of the desks in the collaborative being filled for the majority of the discussion. Another thing that made the discussion a good one was that not everyone in the room, or even on the panel, was really convinced about the efficacy of MOOCs. This lead to more interesting and thought provoking discussions.

In this post I wanted to put down some follow-up ideas that I had related to this discussion. Of course, I was one of the people on the panel who felt that MOOCs could be effective and were worth watching out for. So this post contains my response to two main issues with MOOCs that were raised during the panel and some elaboration on why I fear MOOCs. I agree that the issues are significant problems with current MOOCs. What I want to point out is that they are also correctable.

Problems with Meet-Ups
One of the obvious shortcomings of a MOOC is the lack of actual physical contact with the professor or other students. While there really isn't much that can be done about contact with the professor (that is after all what makes the MOOCs scale so well), there are solutions when it comes to students. Many students in MOOCs schedule meet-ups with other students so that you can have the discussion that can be very vital to the learning process. For the MOOC I was part of, they even pointed out http://www.meetup.com/ as an option for getting together with other class members and for continuing to build a community after the class had ended.

I actually think that this approach works well in some places. Unfortunately, San Antonio is not currently one of those places. As a city, San Antonio has a sufficient population, but I don't believe it has a sufficient number of active MOOC takers. This is a problem that would be solved naturally if MOOCs were to gain more traction. Were they to really take off so that a decent percentage of people use them both as continuing education as well as post-secondary education, then every reasonably sized city in the US would likely have active meet-up groups that could form around the topics.

I could even see a model where the MOOCs look for "local experts" in the topic of the course in major metro areas. These people could probably be given some small compensation to get them to attend meet-ups regularly. Whether that is economically feasible is unclear, but I see this as pushing forward the trend I expect to see in teaching where people are broken into a few superstar content creators and large numbers of "learning coaches".

Shortcomings of Grading
One of the big discussion points of MOOCs is the grading. My CS course was done wonderfully. Automatic grading of code correctness is easy. The MOOC I was enrolled in went further by also checking style with ScalaCheck and giving points for that. They also integrated their tools very nicely so that it was easy for students to submit and get feedback. However, checking coding assignments is only one part of a complete evaluation for most Computer Science courses. That leads into the objection that was brought up in the discussions.

One of the other faculty was taking a Statistics course and the quizzes were all multiple choice. (Full disclosure, I have to admit that I normally use the term "multiple guess" so that tells you how I feel about that type of evaluation.) The problem that she had with this was the fact that she really cares more about the approach that students take more than the final result. She mentioned how seeing the actual work gives her a better ability to see what students are misunderstanding and to modify her teaching methods in the future to improve learning.

In this case, the desire was to see the work involved in doing math to solve problems. Right now the MOOCs do not include a method for doing this. However, this is something that I feel is just waiting for someone to create the right software. I can imagine a tool that includes a formula editor with additional intelligence and perhaps some notation so that students can indicate where one thing leads to another or what they are doing. It is certainly possible to verify that every step in the process is a valid one and to show that the start point and end points are what they should be. Given this, I don't think it is at all unreasonable to expect automatic grading for the work involved in mathematical derivations.

In the area of Computer Science, I have been thinking a fair bit of thinking about how I could set up automatic evaluation of the different types of questions that I like to use for assessment. Coding is one facet, but I also typically ask students to trace code and to do short answers. I don't yet have an approach for auto-grading short answer questions, but I do have an idea for auto-grading tracing questions which I wrote up here.

In addition, I have quite a few problems where students have to draw the data structures that result from performing various operations. By writing a custom drawing program, which includes only the constructs that students need to be able to draw data structures, it is possible to determine if what a student draws is equivalent to the correct answer and if not, how much it differs. One of my projects for this summer is to write a program that does these things so that I can have students do exercises beyond writing code that are automatically graded.

The advent of MOOCs is inevitably going to spur a lot of people to think about ideas like this for doing automatic evaluation of different types of problems that are significant for particular topics. Many of these will be doable well before we have really good short answer or essay auto-graders.

Fear and Economics

At the end of the panel we got into the question of whether people fear MOOCs and the impact that they might have on liberal arts schools. The room was definitely divided on this topic. I personally feel that MOOCs have to be taken seriously because of the reality of economics. As any economist will tell you, people buy on the margin. The price of a good is based on marginal utility, not absolute utility. For colleges, that means the real question is what benefits does a school provide above and beyond the lower priced competition, or how close can they be in quality to the higher priced competition.

Students receive many different things from a college education, but the reality is that the main thing they are paying for is the signaling to get them a better job. The stats clearly bear out the fact that a college degree is a good investment for most majors. (http://cew.georgetown.edu/whatsitworth/) However, that is largely because it is pretty much the only broadly recognized signaling option currently available. If MOOC certificates or badges were to gain traction as a signaling option, then the marginal cost becomes extremely significant.

The costs of a college education can vary quite a bit, but at many small, liberal arts schools, it looks something like four years + $160,000. The four years are significant because you do have to consider the opportunity cost of spending those years in college. The $160,000 needs to include room and board as well as tuition, books, and other fees. Small schools often have a residency requirement so tuition is only part of the total cost. Of course, students have to have food and shelter regardless of how they choose to educate themselves, so maybe some of that could be discounted, but if the comparison was to living at home with parents, taking MOOCs, and going to lots of meet-ups, the MOOC option really can come out at very close to $0. The flexibility of MOOCs might also allow students to get to the point where they have sufficient signaling in less than four years.

So the question is, is the marginal value of actually going to college worth the cost? As of today, I think that the answer is definitely yes, simply because MOOCs are new and they are not yet a valid signaling option. However, as I posted previously, I know of employers who already understand the value of MOOCs because they are requiring employees to take them. There is also acknowledgement of their signaling ability from the fact that MOOCs are able to work as job placement services today.

The reason I worry about how MOOCs will disrupt higher education is not because I think that MOOCs will be as good or better than traditional higher education. I don't. They can't recreate all the elements. I worry about them because they are priced so low that they don't have to match traditional higher education, they just have to get good enough to be considered a valid signaling mechanism. Once that happens, a reasonable fraction of parents and kids out of High School will pick to try that route instead of putting large sums of money into traditional higher ed. As it stands, many colleges are already struggling, either financially, or with admissions. The extra competition is going to drive those institutions that are in trouble under because they simply won't be able to attract students at the tuition rates that are needed to sustain their budgets.