Models on Your Own Device

Use the models running on your computer from external apps through the API

If you have installed the Onysoft AI Gateway Node app on your computer and connected it to your account, you can use the models running on that computer from external apps such as n8n, Cursor, LangChain, the OpenAI libraries or your own code. The endpoint and the key are the same as for every other model; the only difference is the node/ prefix in the model name.

How It Works

Your computer does not open any port to the outside; the connection is always made by the Node app towards the server. A request follows this path:

1. Your app sends the request to https://api.onysoft.com/v1/chat/completions, as usual.
2. The server checks that the key is a personal key you created in the dashboard and that the requested model is ready on one of your online devices.
3. The request is queued as a job owned by you.
4. The Node app on your computer picks the job up over an outgoing connection that it opens to the server itself and keeps renewing, and runs the model locally on your computer (127.0.0.1).
5. The result is sent back to the server and returned to your app in the same format as any other model.
6. Token counts and duration are written to your usage log; the charge is 0.

Your app (n8n, Cursor, your own code)
   │  ▲
   │  │   POST /v1/chat/completions  "model": "node/..."
   │  │   the answer comes back in the same format as other models
   ▼  │
api.onysoft.com   key and device check → queue
   ▲  │
   │  │   outgoing connection: the Node app picks up the job, sends the result back
   │  │   (no port is opened on your computer)
   │  ▼
Your computer     Onysoft AI Gateway Node → local model (127.0.0.1)

Model Name

Add node/ in front of the catalog ID. If the same model is ready on more than one of your devices, the request goes to one of them; to pick a specific device, append @<device_no>.

You can find the names and device numbers you can use in the id and device_model_id fields of the GET /v1/node/models response. The catalog ID after node/ is case-insensitive; write the prefix itself in lowercase. If your key has a model restriction, add node/* to its list to allow all device models.

Example Request

The endpoint, the key and the request body are the same as for other models; only the model field changes.

curl https://api.onysoft.com/v1/chat/completions \
  -H "Authorization: Bearer sk-ony-your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "node/qwen3-4b-q4_k_m",
    "messages": [
      {"role": "user", "content": "Merhaba! Kısaca kendini tanıtır mısın?"}
    ],
    "max_tokens": 300
  }'

Example Response

The response has the same format as for other models. id is the request ID (req_...), model contains node/<catalog_id> without the device number, and cost.amount is always 0. If the model produced reasoning text, the message also contains reasoning and reasoning_content; if it called a tool, tool_calls is present and finish_reason is tool_calls.

{
  "success": true,
  "data": {
    "id": "req_5c1e9a7d2b3f4e6a8c0d1e2f",
    "object": "chat.completion",
    "created": 1758542400,
    "model": "node/qwen3-4b-q4_k_m",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "Merhaba! Ben Qwen3, bilgisayarınızda çalışan bir dil modeliyim."
        },
        "finish_reason": "stop",
        "logprobs": null
      }
    ],
    "usage": {
      "prompt_tokens": 18,
      "completion_tokens": 16,
      "total_tokens": 34
    },
    "cost": {
      "amount": 0,
      "currency": "USD"
    }
  }
}

Supported Parameters

Only the following fields are passed to your device:

• messages, max_tokens (or max_completion_tokens), temperature, top_p, stop, seed, presence_penalty, frequency_penalty, response_format, tools and tool_choice. For streaming, stream and stream_options.include_usage are also taken into account.
• If you do not send max_tokens, the default is 2048; values above 8192 are lowered to 8192.
• If one of the devices that can serve the model runs it on the processor (CPU) and you did not send max_tokens, the default is 256 for non-streaming requests and 512 for stream:true requests. A value you set explicitly is left as is.
• n can only be 1; any other value returns 400 invalid_request.
• Fields not in this list are not passed to the device and are ignored.

Streaming

When you send stream:true, the response arrives as SSE chunks, just like with other models. However, the device sends the answer to the server only once it has generated all of it, so the chunks arrive back to back, in slices of about 40 characters, after the answer is complete on the device.

While the device is generating the answer, the connection is kept open: the server sends a : ping comment line every 5 seconds. SSE clients and the OpenAI libraries ignore these lines. Non-streaming requests have a shorter time limit, so we recommend stream:true for long answers.

: ping

: ping

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{"content":"Merhaba! Ben Qwen3, bilgisayarınızda çal"},"finish_reason":null,"logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{"content":"ışan bir dil modeliyim."},"finish_reason":null,"logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{},"finish_reason":"stop","logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542401,"model":"node/qwen3-4b-q4_k_m","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":16,"total_tokens":34},"cost":{"amount":0,"currency":"USD"}}

data: [DONE]

Chunk order: role, reasoning text if any (reasoning / reasoning_content), content, tool_calls if any, then finish_reason. If you sent stream_options.include_usage, a usage chunk follows. The last chunk always carries usage and cost, followed by [DONE].

Errors that happen after the stream has started (the device did not pick the request up in time, a timeout, a device error) do not arrive as an HTTP status code but as an error chunk inside the stream, and the stream closes with [DONE]. Errors about the key, the model name or the request body are returned before the stream starts, as a normal JSON error response.

data: {"error":{"code":"node_timeout","message":"..."}}

data: [DONE]

Listing Your Device Models

GET /v1/node/models returns the models loaded on the devices connected to your account. If the same model is loaded on two of your devices, you get two rows (with different node_id values). A model can be used right now only if its ready field is true. This endpoint also works with personal keys only.

curl https://api.onysoft.com/v1/node/models \
  -H "Authorization: Bearer sk-ony-your-api-key"

{
  "success": true,
  "object": "list",
  "data": [
    {
      "id": "node/qwen3-4b-q4_k_m",
      "object": "model",
      "created": 1758369600,
      "owned_by": "cihazim",
      "name": "Qwen3 4B",
      "purpose": "genel",
      "context_window": 8192,
      "node_id": 12,
      "node_name": "MacBook Pro",
      "device_model_id": "node/qwen3-4b-q4_k_m@12",
      "online": true,
      "sharing": true,
      "model_status": "hazir",
      "ready": true,
      "last_seen_at": "2026-09-22T14:05:31+03:00",
      "pricing": {
        "input_per_million_tokens": 0,
        "output_per_million_tokens": 0,
        "currency": "USD"
      }
    }
  ],
  "total": 1
}

Error Codes

Error responses have the same format as on the other endpoints: {"success": false, "error": {"code": "...", "message": "..."}}. Error messages are in Turkish.

Limits

• Only personal keys you created on the API Keys page of the dashboard can be used; project and partner keys get 403 node_owner_only.
• At most 4 requests at the same time per user (queued and running requests are counted together).
• Without streaming, the device has 30 seconds to pick the request up and 120 seconds to produce the answer; the total wait never exceeds 160 seconds. With stream:true these limits are 60 and 180 seconds (300 seconds if the model runs on the processor).
• The context window is the model's context_window value (8192 tokens if none is listed). The prompt length is estimated at roughly 4 characters = 1 token.
• Messages can be at most 256 KB and the whole request body at most 512 KB; you can define at most 64 tools and at most 4 stop strings.

Privacy and Security

• Your request goes only to your own devices; it is never routed to another user's device. Nobody else can send requests to your device this way either.
• Your computer does not open any port to the outside. The connection is always made by the Node app towards the server, so no extra setup is needed behind a router, NAT or firewall.
• As with other models, every request is written to your usage logs together with token counts, duration and the request/response content.

Pricing

Using your own device is free; nothing is deducted from your balance. Requests appear in your usage log with an amount of 0, and token counts and duration are still recorded.

Frequently Asked Questions

Why is there no separate endpoint?

OpenAI-compatible tools such as n8n, Cursor, LangChain and the OpenAI libraries expect /v1/chat/completions for chat. Because we use the same endpoint, you do not have to change the address or the key in these tools; you only change the model name. The node/ prefix makes it explicit that the request goes to your device, not to a catalog model.

What happens if my computer is off?

If none of your devices is online with the model ready, the request returns 503 node_unavailable. If your computer has only just shut down, the server still counts it as online for 45 seconds; in that case the request is queued and the error arrives once the pickup time runs out (30 seconds without streaming, 60 seconds with streaming). In a streaming request, this error arrives inside the stream as an error chunk. The request is never moved to another device or another model.

Can I use it with a project key?

No. Device models only work with the personal keys you create on the API Keys page of the dashboard; project and partner keys get 403 node_owner_only.

Can someone else use my device this way?

No. A request goes to the devices of the owner of the key that sent it. A request sent with your key only runs on your devices, and a request sent with someone else's key only runs on theirs.