Greetings! My name is Colin Fredericks. I work at HarvardX as a Senior Project Lead, which means that most of my job is helping Harvard professors make quality courses. However, over the past few months I’ve been working, mostly in my spare time, on adding partial credit to the Open edX platform.
I really like the Open edX platform, but I was always surprised that there was no way to award partial credit. Having taught college and high school it seemed like such an obvious oversight that I kept expecting to see it on the “upcoming features” list. After three years of waiting I decided to finally take matters into my own hands.
That was early April. I accomplished my original goal in two days, and then the project rapidly inflated into its current state. I was convinced to expand from a single problem type to as many as I could reasonably handle, and to write up a full-fledged proposal that ended up being 16 pages long.
While implementing this feature, I found out about this “test code” stuff, and “code quality checkers,” and other things that they (wisely) don’t tell people about in Computer Science II. I started from the edX Developer’s Guide, as best I understood it, with its pretty flowchart. I opened up a proposal ticket on JIRA, made a github repo for my own code, opened up my first pull request, did some more development, got a lot of good feedback (most of which I incorporated), learned how to rebase properly, opened a new and better pull request, and did some more coding. Finally, after about 15 weeks of development, I’m proud to say that the feature is merged into the edx-platform master branch!
Partial Credit: Feature Details
Here’s the quick rundown of what this feature currently looks like. Note that currently, authors can only enable partial credit by directly editing a problem’s OLX definition.
Multiple Choice and Dropdown problems have one type of partial credit available: “Points” style grading. The course author lists certain choices that are worth partial credit, and optionally sets how much credit they’re worth (the default is 50%). As a side effect, Dropdown problems have been upgraded so that more than one choice can be marked as correct.
Checkbox problems have two (mutually exclusive) types of partial credit available: “Halves” style, in which a perfect response gives 100% credit and each mistake cuts the credit in half, and “Every Decision Counts” style, in which each checkbox effectively turns into its own separate true/false question.
Here is an example of how a course author would enable the “Every Decision Counts” (or “EDC”) style partial credit in a Studio problem:
This is how the problem will appear to students:
Numerical problems also have two types available: “List” style, where the author lists answers that count for partial credit (for example in a math problem, partial credit answers might be ones that result from missing a negative sign or inverting a fraction), and “Close” style, that expands the given tolerance to allow for a “horseshoes and handgrenades” approach. Since the two styles are compatible, you can use both of them in the same problem if you like.
Here is how a course author can define a numerical problem with “list” style partial credit:
Here is how the problem will appear to students:
Finally, Custom Python problems can now return not just a point value, but an indicator of whether the problem is considered fully or only partially correct.
For more details on how to add partial credit to your problems, see the partial credit documentation.
I’m very pleased with how all this came out. The project took longer than I thought it would, but it’s also much more powerful. It provides a framework that makes it easier to add partial credit to other problem types in the future, something I really hope people will take advantage of.
What’s coming in v2?
Well, first, I have a v1.5 planned that cleans up some of the surrounding code, so that future contributors to the Capa module won’t have as much trouble as I did. After that – more problem types, of course! I’m also looking at making this feature available for common problems without using XML – that is, giving instructors the ability to use this feature from the Studio markdown editor. There’s some additional code generalization that could be done. Somewhere in the future there’s also the possibility of adding attempt-based grading… but let’s take things one step at a time.
I still don’t think of myself as a professional programmer, but thanks to this work I’m getting closer.
Colin Fredericks is a Senior Project Lead at HarvardX. In the rest of his spare time he enjoys cooking, writing roleplaying games, and being friends with all the animals.