I had a grader config that was off by a level - I'd put config into the grader section, but got no error - it was just silently skipped:
graders:
- name: "my grader"
type: "code"
# (this should be under `config`, not a root element)
assertions:
- "hello"
We should have more validation in place to detect obvious mistakes like this (for instance, there's no reason to have a code grader without any assertions.