Plugins¶

A Python package can extend Karotte through entry points. Karotte reads them from every package installed next to it. Templates have their own entry point, which Template packages describes.

Registering a plugin¶

Declare entry points in your package's pyproject.toml. Each value has the form module:attribute. Karotte loads the attribute and uses it as described in the table above.

[project.entry-points."karotte.harness_secret_env"]
my_plugin = "my_plugin:HARNESS_SECRETS"

[project.entry-points."karotte.run_config_preprocessors"]
my_plugin = "my_plugin:preprocess"

from karotte.schemas.evaluation_run_config import EvaluationRunConfig

HARNESS_SECRETS = ["MY_SERVICE_TOKEN"]


def preprocess(config: EvaluationRunConfig) -> EvaluationRunConfig:
    return config.model_copy(update={"save_artifacts": False})

Install the package into the same environment as Karotte. For an environment's venv, add it as a dependency. For a tool install, use uv tool install karotte --with my-plugin or uvx --with my-plugin karotte ....

Order and merging¶

If only one value can win, Karotte sorts the entry points by name and takes the first one that loads. If the values add up, Karotte uses all of them.

When a plugin fails to load¶

If a plugin fails to load, Karotte skips it with a warning and carries on without it. Run config preprocessors are the exception. If one of them fails to load or raises an exception, the run fails.

The entry points¶

karotte.cli¶

The attribute is a typer.Typer app. It becomes the subcommand karotte <entry point name>. If an entry point has the same name as a built-in command, Karotte ignores it with a warning. Plugins never replace Karotte's own commands.

[project.entry-points."karotte.cli"]
mytool = "my_plugin.cli:app"

karotte.run_config_preprocessors¶

The attribute is a function that takes the parsed EvaluationRunConfig and returns one. karotte run calls it right after it parses --config, before it picks the runtime or loads the task. See Run config.

karotte.default_proxy_url¶

The attribute is a string. It's the URL that karotte run sends model calls to when you don't pass --proxy. If you pass --no-proxy, Karotte ignores it. See Run config for how proxies work.

karotte.harness_secret_env¶

The attribute is a list of environment variable names. Karotte leaves these variables out of the environment it gives the student's processes, such as bash tool calls and CLI agents. Use it for credentials that the harness needs but the student must not see. Karotte hides only the names you list, because tasks sometimes hand the student a secret on purpose.

karotte.platform_tooling_dirs¶

The attribute is a list of directories where the platform running Karotte mounts its own tooling into every container. During builds and graded runs, the language-toolchains template covers each of them with an empty, read-only tmpfs. That way a submission can't load an interpreter from them. Paths that don't exist are ignored.

karotte.age_delay_exemptions¶

The attribute is a list of package names. When Karotte runs uv outside a project (for karotte update and post_create.py), it applies a 7-day exclude-newer delay. Karotte itself and the packages in these lists are exempt from that delay. Inside an environment, the delay comes from the environment's own pyproject.toml.

karotte.update_migrations¶

The attribute is an object with three methods. karotte update calls them as follows:

Use it when a release renames things that the 3-way merge in Updating environments can't follow on its own.

karotte.default_hardware¶

The attribute is a string. It's the hardware name for tasks that don't set required_hardware. Karotte itself doesn't know any hardware names. Plugins define them. See Tasks and steps for the task property.

karotte.hardware_limits¶

The attribute is a function that takes a hardware name and returns a HardwareLimits. For hardware it doesn't know, it returns None. HardwareLimits sets the sandbox's memory, disk and CPUs, and marks hardware that a VM can't provide, such as a GPU. Tasks on such hardware run under docker by default, and VM runtimes refuse them.

from karotte.hardware import HardwareLimits

GIB = 1024**3


def hardware_limits(hardware: str) -> HardwareLimits | None:
    if hardware == "cpu-large":
        return HardwareLimits(memory_bytes=32 * GIB, disk_bytes=100 * GIB, cpus=8)
    if hardware == "gpu":
        return HardwareLimits(passthrough=True)
    return None

A hardware name that no plugin knows is not an error. Karotte just uses its defaults. Without an answer from a plugin, a VM gets 2 CPUs and 4 GiB for the sandbox. On other runtimes, the sandbox's memory comes from KAROTTE_SANDBOX_MEMORY_BYTES. If that variable isn't set, it comes from the sandbox's cgroup limit or RAM. The student gets this amount minus 1 GiB. If the variable isn't set and there's no working cgroup, the student has no memory limit. See Runtimes and Student resources.

karotte.container_run_args¶

The attribute is a function that takes the task and the runtime name (docker, podman, nerdctl, ...). It returns extra arguments for the engine's run command. You can use it to pass devices through, for example. Karotte adds these arguments for docker, podman, docker:gvisor and nerdctl.

karotte run calls every hook before it launches a container. If a hook raises an exception, Karotte refuses to launch and shows the exception's message.

def container_run_args(task, runtime: str) -> list[str]:
    if task.required_hardware != "gpu":
        return []
    if runtime != "docker":
        raise RuntimeError("GPU tasks need --runtime docker")
    return ["--gpus", "all"]