Model Catalog Sync

The AI module keeps a local model catalog that mirrors the public Vercel AI Gateway model inventory and its provider endpoint metadata.

Controller Surface

• GET /models
• POST /models/refresh
• GET /models/refresh/status
• GET /models/:ownerId/:modelKey

These routes are implemented in source but are not live yet because the module is not mounted.

Refresh Workflow

POST /models/refresh starts an asynchronous refresh job and returns 202 Accepted.

The job performs the following steps:

1. Fetch the full model list from https://ai-gateway.vercel.sh/v1/models.
2. Upsert each model's base metadata locally.
3. For every discovered model, fetch endpoint details from .../models/:modelId/endpoints.
4. Persist provider capabilities, architecture, and sync outcome.
5. Mark locally known models as REMOVED when they no longer appear in the upstream catalog.

The sync loop deliberately waits 200 ms between models.

Read Behavior

• GET /models returns all locally stored models.
• GET /models/:ownerId/:modelKey reconstructs the model id as ownerId/modelKey.
• GET /models/refresh/status merges runtime progress counters with persisted counters and returns the list of model ids that are currently in SYNCED state.

Sync Statuses

Models move through these states:

• PENDING
• IN_PROGRESS
• SYNCED
• FAILED
• REMOVED

Failures store the last sync error string on the entity.

Data Captured Per Model

• descriptive metadata such as owner, name, description, and type
• capacity metadata such as contextWindow and maxTokens
• base model pricing and tiered pricing
• provider endpoint data, including per-provider pricing, status, latency, and supported parameters
• architecture modalities and last sync timestamp

Current Caveat

The in-memory refreshPromise guard only prevents overlapping refreshes inside one process. It does not coordinate across multiple API instances, so horizontally scaled deployments can still trigger concurrent refresh jobs.

Structured Error Contract

Code	HTTP	Meaning
AI.MODELS_SYNC_ALREADY_IN_PROGRESS	409	A refresh job is already running in this process.
AI.MODEL_NOT_FOUND	404	The requested stored model id does not exist.
AI.FAIL_TO_FETCH_VERCEL_AI_GATEWAY_MODELS	500	Upstream model-list fetch failed.
AI.FAIL_TO_FETCH_VERCEL_AI_GATEWAY_MODEL_ENDPOINTS	500	Upstream endpoint-details fetch failed.