Self-Hosted Model Gateway

LLMs

A self-hosted model gateway: your programs connect to this one endpoint and call different model providers the same way, and the provider keys live only on your own machine. A web admin panel lets you add providers and issue API keys. You bring it up on your own machine with Docker Compose, and the master key, the encryption key and the database password are generated at random by a script. Derived from a frozen snapshot of open-source model gateway software under the Apache-2.0 licence; the delivery adds an orchestration file that builds the image from source (two services, the gateway and PostgreSQL), a script that generates the secrets and the settings file, and a quick-start guide in Traditional Chinese.

Subscribe & download See how to start

SubscriptionUS$49 per month

Last run2026-11-02

In practice

Readiness4 / 5

What you need

Few changes needed4 / 5

Who it is for

Developers and small teams who use several models at once and want the provider keys kept in one place on their own machine.

How to start

$ sh scripts/quickstart/init-env.sh
$ docker compose up -d --build

Browse the files

Files in the product repository at the pinned commit. A few house files are left out of the zip.

File overview2,000+ paths

▸alembic131

env.py
script.py.mako
versions/

▸any-fetch41

LICENSE
README.md
pyproject.toml
scripts/
src/
tests/

▸any-search41

LICENSE
README.md
pyproject.toml
scripts/
src/
tests/

▸assets4

architecture.mmd
architecture.png
otari-demo.gif
otari-logo.svg

▸cli17

LICENSE
README.md
pyproject.toml
src/

▸demo31

catalog-smoke/
code-exec/
guardrails/
web-search/

▸deploy3

railway/

▸docs36

SUMMARY.md
access-control.md
agent-guardrails-reference.md
agent-guardrails.md
api-reference.md
chatgpt-subscription.md
code-execution-protocol.md
configuration.md
dashboard.md
deployment.md
…

▸integrations18

e2e/
langchain-otari/
pydantic-ai-otari/

▸loadtest9

.gitignore
README.md
ab.py
comparison.py
docker-compose.yml
fakeprovider.py
otari-config.yml
pytest.ini
run.sh

▸packaging1

homebrew/

▸plugins5

otari-router/

▸scripts30

check_alembic_heads.py
check_architecture.py
check_code_execution_conformance.py
check_railway_template.py
check_stale_regen_prs.py
demo_gif/
docker_liveness_check.sh
generate_openapi.py
generate_postman.py
homebrew_formula.py
…

▸src585

gateway/

▸tests636

conftest.py
fixtures/
integration/
unit/

▸web267

.gitignore
.storybook/
.vitest/
AGENTS.md
CLAUDE.md
README.md
biome.jsonc
design/
e2e/
index.html
…

▫AGENTS.md

▫ARCHITECTURE.md

▫ATTRIBUTION.md

▫CHANGELOG.md

▫CLAUDE.md

▫CODE_OF_CONDUCT.md

▫CONTRIBUTING.md

▫DOMAINS.md

▫Dockerfile

▫GOVERNANCE.md

▫LICENSE

▫LICENSE-99LL.md

▫Makefile

▫PRODUCT.md

▫QUICKSTART.md

▫README.md

▫RELEASE.md

▫SECURITY.md

▫SECURITY.upstream.md

▫SUPPORT.md

▫alembic.ini

▫cliff.toml

▫codecov.yml

▫config.example.yml

▫delivery-manifest.txt

▫docker-compose.build.yml

▫docker-compose.yml

▫openapitools.json

▫pyproject.toml

▫pytest.ini

▫uv.lock

▫Config and hidden files 114

Latest passing run

Clone → Checkout → Toolchain → Build images → Gateway ready and key auth

Pre-purchase Disclosure

What you are buying

A self-hosted model gateway: your programs connect to this one endpoint and can call different model providers, and the provider keys live only on your own machine. A web admin panel lets you add providers. After running the generation script you bring it up with Docker Compose.

Derived from open-source model gateway software under the Apache-2.0 licence, and not the original project itself. Our work: an orchestration file that builds the image from source, a script that generates the secrets, and a quick-start guide in Traditional Chinese.

Delivery is a frozen snapshot: one zip, taken from the product repository at revision 6053c71.

Known limitations

The service serves plain HTTP on port 8000 of the local machine; TLS and a reverse proxy are yours to bring
The admin panel is in English only
The delivery contains no model provider, and you add one in the admin panel with your own key
No model request was sent with a real provider key before delivery
If the encryption key is lost, the provider keys already stored cannot be opened
A backup has to keep the database dump and .env together
docker compose down -v deletes all providers, API keys, budgets and usage records
The first start builds the image from source, fetching from the registry, npm and PyPI during that time
The corepack and uv installed at build time are the latest versions of the day
Pinned to the delivered snapshot; it does not follow upstream.

The liability cap, the scope of what is promised and the statutory floor are governed by the Terms of Service.

Vetting record

Everything below was checked against the pinned commit b89a099. Every verdict carries its full reasoning, an evidence link and the checker; failures are published, not hidden — do not take our word for it, verify it yourself.

5 PASS0 WARN0 FAIL

Source ReviewPASS

Execution surface of the delivery reviewed item by item. Our own additions were read line by line: docker-compose.yml (the published port is bound to 127.0.0.1 only; the database is not published; it refuses to start when the master key, the encryption key, the independent secret or the database password is missing; the settings file is mounted read-only; the host name host.docker.internal is mapped so a local model service can be reached), scripts/quickstart/init-env.sh (openssl generates random values, .env is mode 600, existing .env and settings file are left alone), and the settings template. The upstream Python and front-end sources were not read file by file; the basis is the Apache-2.0 licence and container isolation.

Script SafetyPASS

The scripts on the buyer’s delivery path were each inspected: init-env.sh is ours (read line by line); the build goes through the Dockerfile (read line by line: a multi-stage build with the front end installed by pnpm from the lock file and the back end by uv from the lock file; corepack is installed from npm and pip and uv are upgraded at build time; no download piped straight into a shell). At start-up the gateway runs as a non-root user.

RecencyPASS

This product is a frozen snapshot and does not follow upstream: the upstream source is pinned to the main-line revision taken on the day the product repository was created (see the evidence link), and the product repository is that snapshot plus our own commits.

LicensePASS

SPDX:LicenseRef-99LL-Product

Upstream LICENSE was checked word for word (the evidence link points at the file itself at the pinned version and matches the copy inside the product repository): the full Apache License 2.0 text, copyright line naming Mozilla.ai. It allows commercial use, modification, derivative works, redistribution and sale; the duties include shipping the licence text, keeping the copyright and attribution notices, and marking modified files as changed; upstream has no NOTICE file (LICENSE and ATTRIBUTION.md ship in the zip). The decision does not rest on GitHub’s licence field. For the product as a whole the 99LL product licence template is attached: upstream’s LICENSE is kept as it is and LICENSE-99LL.md is added as the sale and liability terms for the part we built, with spdx recorded as LicenseRef-99LL-Product.

Dependency LicensesPASS

This records the declared side, not a scan result: the Python dependencies are declared in pyproject.toml and locked by uv.lock, the front end is declared in package.json and locked by pnpm-lock.yaml, and the packages themselves are fetched from their own repositories when the buyer builds; the base images come from a registry. These dependency trees were not scanned package by package for licences. Upstream has no submodule and no third-party dataset was copied in (see ATTRIBUTION.md).