Pre-requisites#

Before getting started, the framework requires a couple dependencies:

Linux#

Install all dependecies with apt-get:

apt-get install python3 python3-pip cloc cppcheck clang-format

OSX#

Install all dependencies with brew:

brew install python cloc cppcheck clang-format

Windows#

• Install Python
• Install CLOC
• Install cppcheck
• Install clang-format

Install the package#

The framework is distributed as a pip package. Install it by:

pip3 install UoL-Autograder

This will install all further dependencies.

Virtual environments are supported. The during evaluation the same environment will be used where the framework is running. Custom Python environment support is planned.

Running the framework#

If you have installed succesfully, the feedback command is added. Confirm by typing feedback -h into a command line.

To run an exam, type the following:

feedback <tested_file_location> <config_file_location>

To create these, see the Creating exams section.

Run parameters#

• -v - Verbose execution, see the details of the run. Default off
• -f <PATH> - Specify the location of the result file. Default is ./results.json
• -x - Clear the cache of compiled files. Only run this if you think this is impacting your tests. This will slow down test execution
• -c <PATH> - Copy the temporary directory where evaluation took place to the specified path after evaluation completed. Used to debug test execution
• -d - Disable result validation. Result files generated by testers are only deemed valid, if the appropriate validation logic is followed. If you're not using one of the tester templates and don't want to include validation, enable this option. Carefull, this enables the student to maliciously modify results.
• -p <n> - Set the precision of the results. Change this if you have a lot of questions, or low maximum score, which results in students who should score max get more or less than the maximum score. Default is 4

For developers#

Make sure you have the Pre-requisites installed, unless you want to use a docker development environment

Clone the repository:

git clone https://gitlab.com/tepl/uol-autograder.git
cd UoL-Autograder

Install dependencies:

pip3 install -r requirements.txt

Run the module with the same arguments as the installed version:

python3 -m feedback <tested_file_location> <config_file_location>

Run the tests:

pytest

Using a docker development environment#

If you don't want to install all the dependencies, and have docker running locally, use the developer docker image in the repository:

Clone the repository:

git clone https://gitlab.com/tepl/uol-autograder.git
cd UoL-Autograder

Build the docker image:

docker build --pull --rm -f "Dockerfile" -t uol-autograder-dev:latest .

Launch the container from the image, and mount the current directory:

docker run --rm --name feedback -v ${PWD}:/app -it uol-autograder-dev:latest bash

Inside the container in the /app directory you can run feedback like so: python3 -m feedback <tested_file_location> <config_file_location> or run tests like so: pytest

Visual Studio Code Dev Containers are also supported. Use the .devcontainer/devcontainer.json as a starting point, or use Dockerfile as the base