llamactl API 1.0¶
llamactl is a control server for managing Llama Server instances.
Keys¶
GET /api/v1/auth/keys¶
List all API keys
Description
Returns a list of all API keys for the system user (excludes key hash and
plain-text key)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
POST /api/v1/auth/keys¶
Create a new API key
Description
Creates a new API key with the specified permissions and returns the plain-
text key (only shown once)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
key |
body | None | No | API key configuration |
Response 201 Created
Response 400 Bad Request
Response 500 Internal Server Error
DELETE /api/v1/auth/keys/{id}¶
Delete an API key
Description
Deletes an API key by ID
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
id |
path | None | No | Key ID |
Response 204 No Content
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
GET /api/v1/auth/keys/{id}¶
Get details of a specific API key
Description
Returns details for a specific API key by ID (excludes key hash and plain-
text key)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
id |
path | None | No | Key ID |
Response 200 OK
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
GET /api/v1/auth/keys/{id}/permissions¶
Get API key permissions
Description
Returns the instance-level permissions for a specific API key (includes
instance names)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
id |
path | None | No | Key ID |
Response 200 OK
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
Backends¶
GET /api/v1/backends/llama-cpp/devices¶
List available devices for llama server
Description
Returns a list of available devices for the llama server
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
GET /api/v1/backends/llama-cpp/help¶
Get help for llama server
Description
Returns the help text for the llama server command
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
POST /api/v1/backends/llama-cpp/parse-command¶
Parse llama-server command
Description
Parses a llama-server command string into instance options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
request |
body | None | No | Command to parse |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/backends/llama-cpp/version¶
Get version of llama server
Description
Returns the version of the llama server command
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
POST /api/v1/backends/mlx/parse-command¶
Parse mlx_lm.server command
Description
Parses MLX-LM server command string into instance options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
request |
body | None | No | Command to parse |
Response 200 OK
Response 400 Bad Request
POST /api/v1/backends/vllm/parse-command¶
Parse vllm serve command
Description
Parses a vLLM serve command string into instance options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
request |
body | None | No | Command to parse |
Response 200 OK
Response 400 Bad Request
System¶
GET /api/v1/config¶
Get server configuration
Description
Returns the current server configuration (sanitized)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
GET /api/v1/version¶
Get llamactl version
Description
Returns the version of the llamactl command
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
Instances¶
GET /api/v1/instances¶
List all instances
Description
Returns a list of all instances managed by the server
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
DELETE /api/v1/instances/{name}¶
Delete an instance
Description
Stops and removes a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 204 No Content
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/instances/{name}¶
Get details of a specific instance
Description
Returns the details of a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/instances/{name}¶
Create and start a new instance
Description
Creates a new instance with the provided configuration options
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name | |
options |
body | None | No | Instance configuration options |
Response 201 Created
Response 400 Bad Request
Response 500 Internal Server Error
PUT /api/v1/instances/{name}¶
Update an instance's configuration
Description
Updates the configuration of a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name | |
options |
body | None | No | Instance configuration options |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/instances/{name}/logs¶
Get logs from a specific instance
Description
Returns the logs from a specific instance by name with optional line limit
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
lines |
query | None | No | Number of lines to retrieve (default: all lines) | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /api/v1/instances/{name}/proxy¶
Proxy requests to a specific instance, does not autostart instance if stopped
Description
Forwards HTTP requests to the llama-server instance running on a specific
port
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Response 503 Service Unavailable
POST /api/v1/instances/{name}/proxy¶
Proxy requests to a specific instance, does not autostart instance if stopped
Description
Forwards HTTP requests to the llama-server instance running on a specific
port
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Response 503 Service Unavailable
POST /api/v1/instances/{name}/restart¶
Restart a running instance
Description
Restarts a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/instances/{name}/start¶
Start a stopped instance
Description
Starts a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/instances/{name}/stop¶
Stop a running instance
Description
Stops a specific instance by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Llama.cpp¶
GET /api/v1/llama-cpp/{name}/models¶
List models in a llama.cpp instance
Description
Returns a list of models available in the specified llama.cpp instance
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/llama-cpp/{name}/models/{model}/load¶
Load a model in a llama.cpp instance
Description
Loads the specified model in the given llama.cpp instance
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
model |
path | None | No | Model Name | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /api/v1/llama-cpp/{name}/models/{model}/unload¶
Unload a model in a llama.cpp instance
Description
Unloads the specified model in the given llama.cpp instance
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
model |
path | None | No | Model Name | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /llama-cpp/{name}/¶
Proxy requests to llama.cpp UI for the instance
Description
Proxies requests to the llama.cpp UI for the specified instance
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
query | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/apply-template¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/completion¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/detokenize¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/embeddings¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/infill¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/metrics¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /llama-cpp/{name}/props¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/props¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/reranking¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /llama-cpp/{name}/slots¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
POST /llama-cpp/{name}/tokenize¶
Proxy requests to llama.cpp server instance
Description
Proxies requests to the specified llama.cpp server instance, starting it on-
demand if configured
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Instance Name |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
Models¶
DELETE /api/v1/models¶
Delete a cached model
Description
Deletes a cached model by its repository and optional tag
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
node |
query | None | No | Node name to forward the request to | |
repo |
query | None | No | Repository | |
tag |
query | None | No | Tag |
Response 204 No Content
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
GET /api/v1/models¶
List cached models
Description
Returns a list of all models currently cached on the server. If node
parameter is specified, only returns models from that node. If no node is
specified, returns models from all nodes aggregated.
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
node |
query | None | No | Node name to query (if not specified, queries all nodes) |
Response 200 OK
Response 500 Internal Server Error
POST /api/v1/models/download¶
Download a model from a repository
Description
Initiates the download of a model from a specified repository and tag.
Returns a job ID to track progress.
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
node |
query | None | No | Node name to forward the request to | |
request |
body | None | No | Download request |
Response 202 Accepted
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
GET /api/v1/models/jobs¶
List all model download jobs
Description
Returns a list of all model download jobs with their details
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
node |
query | None | No | Node name to forward the request to |
Response 200 OK
Response 500 Internal Server Error
DELETE /api/v1/models/jobs/{id}¶
Cancel an ongoing model download job
Description
Cancels a model download job by its ID. Only jobs that are in progress can
be cancelled.
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
id |
path | None | No | Job ID | |
node |
query | None | No | Node name to forward the request to |
Response 204 No Content
Response 400 Bad Request
Response 404 Not Found
Response 409 Conflict
Response 500 Internal Server Error
GET /api/v1/models/jobs/{id}¶
Get details of a specific download job
Description
Returns the details of a download job by its ID
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
id |
path | None | No | Job ID | |
node |
query | None | No | Node name to forward the request to |
Response 200 OK
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
Nodes¶
GET /api/v1/nodes¶
List all configured nodes
Description
Returns a map of all nodes configured in the server (node name -> node
config)
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error
GET /api/v1/nodes/{name}¶
Get details of a specific node
Description
Returns the details of a specific node by name
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No | |
name |
path | None | No | Node Name |
Response 200 OK
Response 400 Bad Request
Response 404 Not Found
Response 500 Internal Server Error
OpenAI¶
POST /v1/¶
OpenAI-compatible proxy endpoint
Description
Handles all POST requests to /v1/*, routing to the appropriate instance
based on the request body. Requires API key authentication via the
Authorization header.
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 400 Bad Request
Response 500 Internal Server Error
GET /v1/models¶
List instances in OpenAI-compatible format
Description
Returns a list of instances in a format compatible with OpenAI API
Input parameters
| Parameter | In | Type | Default | Nullable | Description |
|---|---|---|---|---|---|
ApiKeyAuth |
header | string | N/A | No |
Response 200 OK
Response 500 Internal Server Error