Manage model masters

Model masters hold the default serving parameters for each model. You list, get, add, update, or archive them by sending requests to the app-node API from any machine that can reach your CosmicAC base URL. This guide uses curl to send the requests from a terminal. To register the supported models after you deploy, see Set up model masters. For what a model master is, see Model masters.

The base URL is the address where your CosmicAC UI is reachable, such as localhost, a server IP, or a domain name. It depends on how your team deployed CosmicAC and where you connect from. If you need it, ask your administrator. CosmicAC serves the app-node API under /api, so the requests below use <base-url>/api.

Prerequisites

You need the following before you start:

A running CosmicAC deployment. See Installation.
Your CosmicAC base URL.
A terminal with curl.

List model masters

List the active model masters. To list archived ones instead, set status=archived:

curl "<base-url>/api/v1/model-masters?status=active&limit=20&offset=0"

Each entry includes an id. Use this ID to get, update, or archive the model master in the requests below.

Get a model master

To get a single model master, send a GET request with its ID:

curl <base-url>/api/v1/model-masters/<id>

Add a model master

To add a new model master, send a POST request with its serving parameters, in the same format as Set up model masters. This example adds MiniMax M2.5:

curl -X POST <base-url>/api/v1/model-masters \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "INFERENCE_VLLM",
    "base_os_image": "Ubuntu 22.04 + CUDA 12.9",
    "disk_gb": 500,
    "cuda_driver_version": "CUDA 12.9",
    "model_name": "MiniMaxAI/MiniMax-M2.5",
    "runtime_image": "vllm/vllm-openai:v0.15.1",
    "data_type": "Auto",
    "quantisation": "None",
    "tensor_parallel": 4,
    "gpu_memory_utilisation": 0.85,
    "max_model_length": 131072,
    "max_concurrent_sequences": 64,
    "replica": 1,
    "require_auth_header": true
  }'

For validated values for supported models, see Recommended model parameters.

Runtime image format

Set runtime_image to a Docker image reference, such as vllm/vllm-openai:v0.15.1. CosmicAC serves the model on the image you name here.

The earlier label format, such as vLLM 0.15.0 + CUDA 12.9, no longer works. If a model master still uses that format, CosmicAC serves the model on a fallback image chosen by your deployment rather than the version you set.

The web interface offers the model master's value as the only runtime image for the model, and interactive cosmicac jobs create applies it without prompting. Non-interactive job creation takes the runtime image from --runtime-image instead.

Update a model master

To update a model master, send a PATCH request with only the fields to change:

curl -X PATCH <base-url>/api/v1/model-masters/<id> \
  -H "Content-Type: application/json" \
  -d '{ "disk_gb": 750, "replica": 2 }'

To change a model's runtime image, send a PATCH request with runtime_image:

curl -X PATCH <base-url>/api/v1/model-masters/<id> \
  -H "Content-Type: application/json" \
  -d '{ "runtime_image": "vllm/vllm-openai:v0.15.1" }'

Archive a model master

To archive a model master, send a DELETE request with its ID:

curl -X DELETE <base-url>/api/v1/model-masters/<id>

This archives the model master rather than deleting it. Archived model masters still appear when you list with status=archived.