Simple unified evaluation framework, using Gradescope Autograder
Overview#
The UoL Autograder framework builds on top of the Gradescope Autograder to provide a more streamlined way of defining exams to enable instructors to focus on the content of the exam, and giving high quality feedback.
Currently supported languages#
- Python
- C++
How it works#
This framework is designed to run a set of tests on student submitted code. This is achieved with a combination of built-in tests and using specific scripts to test certain aspects of the functionality of the student's code. These are called testers.
In addition to custom tester scripts, the system also has a set of built in tests, such as
- compilation checking
- static analysis
- code format
- comment density
The set of tests are configured in a configuration file that also sets the test parameters and score awarded for each test.
As an output a result.json file is generated that is compatible with the format Autograder expects.
How to use it#
To get started as an instructor, creating exams, visit
- Getting started page for installation instructions
- Creating exams page for instructions on how to create exams
- Examples pages for example tester configurations
- Packaging and uploading page for instruction on how to package and upload your exam
This codebase is created and maintained by student and staff at the University of Leeds, but any feedback and contribution is welcome. To get started as a developer, modifying the framework, visit
- Getting started page for environment configuration instructions, including how to run it in a dockerised environment
- Creating your own tester framework page for instructions on how to create new testers in the supported languages
- Issues page on GitLab to see what is in progress and what can be worked on. Also, feel free to raise new issues/requests.