cerebrium.toml

One file per app, read by cerebrium deploy and cerebrium run. Two rules to internalise before editing one:

A key you leave out is set to the default, not left alone. A deploy replaces the whole app record. If a previous deploy or cerebrium apps scale set max_replicas = 20 and the TOML you deploy has no max_replicas, the app goes back to 1. Keep every value that matters in the file.
A key the CLI does not recognise is not rejected locally. A misspelled max_replica does not fail the deploy in the CLI, it just does nothing there. The whole file is uploaded verbatim, so the backend sees the key too and is free to reject it. Check spelling against the tables below rather than trusting a successful deploy. A TOML that does not parse at all does fail the deploy.

Defaults below are what the API applies when the key is absent. Where a published table disagrees, prefer these, and set anything that matters explicitly.

`[cerebrium.deployment]`

Key	Applied when omitted	Notes
`name`	required	App name.
`python_version`	`3.11`	`3.10`, `3.11`, `3.12` and `3.13` are accepted. Changing it forces a full rebuild.
`disable_auth`	`true`	`true` means the endpoint is public. Set `false` for anything real.
`include`	`["./*", "main.py", "cerebrium.toml"]`	Keep model weights out of it.
`exclude`	`[".*"]`
`shell_commands`	`[]`	Run at the end of the build.
`pre_build_commands`	`[]`	Run before dependencies install.
`docker_base_image_url`	`debian:bookworm-slim`	Changing it forces a full rebuild.
`use_uv`	`false`	uv instead of pip. Much faster on large dependency trees.
`deployment_initialization_timeout`	`600`	Seconds, 60 to 830. Raise it when weights load slowly.

cerebrium init scaffolds disable_auth = true. That is a convenience for a first curl, not a production default. Treat flipping it to false as part of the first real deploy.

`[cerebrium.runtime]`

Key	Applied when omitted	Notes
`container_runtime`	platform default	`"v1"` is runc, `"v2"` is gvisor. Any other value fails the deploy locally.

The sub-tables in the next two sections sit under the same heading and can be set alongside container_runtime.

`[cerebrium.runtime.custom]`

Only for a custom web server (FastAPI, ASGI, WebSockets, custom batching) or a Dockerfile build. Omit the section to use the default Cortex runtime.

Key	Applied when omitted	Notes
`port`	`8000`	Must match the port inside `entrypoint`.
`entrypoint`	required	A list (`["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]`) or a single string. The list form is unambiguous, prefer it.
`healthcheck_endpoint`	`""` (TCP ping)	Non-200 marks the instance unhealthy and restarts it.
`readycheck_endpoint`	`""` (TCP ping)	Non-200 removes the instance from routing.
`dockerfile_path`	unset	Build from a Dockerfile instead. The path must exist locally, and the image then owns its own dependencies and Python version.

`[cerebrium.runtime.deepgram]` and `[cerebrium.runtime.rime]`

Partner service runtimes. The table name is the partner, and an app uses one of them or neither. Omit both unless you are deploying that partner’s image.

Key	Applied when omitted	Notes
`port`	partner default	Port the partner service listens on.
`model_name`	partner default	Partner model identifier, for example `arcana` or `mist`.
`language`	partner default	Language code, for example `en`.

[cerebrium.runtime.deepgram]
model_name = "arcana"
language = "en"

`[cerebrium.hardware]`

Key	Applied when omitted	Accepted
`cpu`	`2.0`	Depends on the compute type and the plan. See `references/hardware.md`.
`memory`	`4.0`	GB of host RAM, not VRAM. Same limits apply.
`compute`	`"CPU"`	One identifier, or a preference-ordered list of up to 5 from the same family.
`gpu_count`	`1` when `compute` is an accelerator, `0` for `CPU`	Per-type maximum, capped again by the plan.
`provider`	platform picks	`aws`, `crusoe` or `nebius`.
`region`	platform picks	See `references/hardware.md`.

`[cerebrium.scaling]`

Key	Applied when omitted	Accepted	Notes
`min_replicas`	`0`	0 to 2000, capped by plan	Above 0 keeps warm capacity and bills for it.
`max_replicas`	`1`	1 to 2000, capped by plan	The most common cause of unexplained queueing. Raise it before load.
`replica_concurrency`	`1` on an accelerator, `100` on CPU	>= 1	In-flight requests per replica.
`scaling_metric`	`concurrency_utilization`	`concurrency_utilization`, `requests_per_second`, `cpu_utilization`, `memory_utilization`
`scaling_target`	`100`	1 to 100 for the utilization metrics, any positive rate for `requests_per_second`
`scaling_buffer`	`0`	>= 0	Extra replicas above what the metric asks for. Rejected with `cpu_utilization` or `memory_utilization`.
`cooldown`	`10`	0 to 3600 seconds	Time at reduced load before scaling down.
`response_grace_period`	`900`	16 to 43200 seconds	Also the ceiling on an async run, so 12 hours is the maximum.
`evaluation_interval_seconds`	`30`	6 to 300	Window metrics are evaluated over.
`load_balancing_algorithm`	`round-robin`	`round-robin`, `first-available`, `min-connections`, `random-choice-2`	`first-available` suits `replica_concurrency = 1` GPU work.
`compute_tier`	`interruptible`	`interruptible`, `protected`	`protected` is on-demand: higher availability, higher price.
`roll_out_duration_seconds`	`0`	0 to 21600	Gradual traffic shift to a new revision. Keep 0 while iterating.

Two combinations the API rejects outright:

cpu_utilization or memory_utilization with min_replicas = 0. These metrics need a running replica to measure, so set min_replicas to at least 1.
scaling_buffer with either of those two metrics.

Dependencies

[cerebrium.dependencies.pip]
torch = "==2.0.0"        # exact
transformers = "latest"
numpy = ">=1.26"

[cerebrium.dependencies.apt]
ffmpeg = "latest"

[cerebrium.dependencies.conda]
cudatoolkit = "11.7"

[cerebrium.dependencies.paths]
pip = "requirements.txt"
apt = "pkglist.txt"

`[cerebrium.experimental]`

[cerebrium.experimental]
checkpointing = true

Turns on memory and GPU checkpointing (beta, availability varies by account). The container also has to POST to http://169.254.169.253:8234/checkpoint once initialisation is done, so it is a config change plus a code change. See references/troubleshooting.md for where it sits in the cold-start order.

What forces a full rebuild

Batch these edits together, they are the slow ones:

python_version
docker_base_image_url
any [cerebrium.dependencies.apt] or [cerebrium.dependencies.conda] change

Pip-only changes and code changes are much cheaper.

Worked example

[cerebrium.deployment]
name = "llm-inference"
python_version = "3.12"
disable_auth = false
include = ["./*", "main.py", "cerebrium.toml"]
exclude = [".*"]
use_uv = true
deployment_initialization_timeout = 800

[cerebrium.hardware]
cpu = 8
memory = 32.0
compute = ["HOPPER_H100", "AMPERE_A100_80GB"]
gpu_count = 1
region = "us-east-1"

[cerebrium.scaling]
min_replicas = 0
max_replicas = 10
replica_concurrency = 1
scaling_metric = "concurrency_utilization"
scaling_target = 100
cooldown = 60
compute_tier = "protected"

[cerebrium.dependencies.pip]
vllm = "latest"

Private base images

docker login -u your-dockerhub-username     # not the bare `docker login` OAuth flow

Checklist before deploying

Every value that matters is in the file, including ones set by a previous apps scale
disable_auth is deliberate, and false if the endpoint is not meant to be public
max_replicas matches the traffic you expect, not the default 1
replica_concurrency matches the workload (1 for one-request-per-GPU inference)
compute uses an accepted identifier, and cpu, memory and gpu_count are inside that type’s limits and the plan’s (references/hardware.md)
weights load from /persistent-storage, not baked into the image or in include
custom runtime: the port in entrypoint equals port
secrets are added with cerebrium secrets add, never hardcoded

Getting Started

Container Images

GPUs and Compute Resources

Performance

Scaling apps

Deployments

Endpoints

Networking

Storage

Partner Services

Integrations

Other concepts

Config

cerebrium.toml

`[cerebrium.deployment]`

`[cerebrium.runtime]`

`[cerebrium.runtime.custom]`

`[cerebrium.runtime.deepgram]` and `[cerebrium.runtime.rime]`

`[cerebrium.hardware]`

`[cerebrium.scaling]`

Dependencies

`[cerebrium.experimental]`

What forces a full rebuild

Worked example

Private base images

Checklist before deploying

​cerebrium.toml

​[cerebrium.deployment]

​[cerebrium.runtime]

​[cerebrium.runtime.custom]

​[cerebrium.runtime.deepgram] and [cerebrium.runtime.rime]

​[cerebrium.hardware]

​[cerebrium.scaling]

​Dependencies

​[cerebrium.experimental]

​What forces a full rebuild

​Worked example

​Private base images

​Checklist before deploying

cerebrium.toml

`[cerebrium.deployment]`

`[cerebrium.runtime]`

`[cerebrium.runtime.custom]`

`[cerebrium.runtime.deepgram]` and `[cerebrium.runtime.rime]`

`[cerebrium.hardware]`

`[cerebrium.scaling]`

Dependencies

`[cerebrium.experimental]`

What forces a full rebuild

Worked example

Private base images

Checklist before deploying