Models on Your Own Device

Use the models running on your computer from external apps through the API

If you have installed the Onysoft AI Gateway Node app on your computer and connected it to your account, you can use the models running on that computer from external apps such as n8n, Cursor, LangChain, the OpenAI libraries or your own code. The endpoint and the key are the same as for every other model; the only difference is the node/ prefix in the model name.

How It Works

Your computer does not open any port to the outside; the connection is always made by the Node app towards the server. A request follows this path:

1. Your app sends the request to https://api.onysoft.com/v1/chat/completions, as usual.
2. The server checks that the key is a personal key you created in the dashboard and that the requested model is ready on one of your online devices.
3. The request is queued as a job owned by you.
4. The Node app on your computer picks the job up over an outgoing connection that it opens to the server itself and keeps renewing, and runs the model locally on your computer (127.0.0.1).
5. The answer is sent back to the server and returned to your app in the same format as any other model; for streaming requests, chunks arrive as the device generates them (see “Streaming” below).
6. Token counts and duration are written to your usage log; the charge is 0.

Your app (n8n, Cursor, your own code)
   │  ▲
   │  │   POST /v1/chat/completions  "model": "node/..."
   │  │   the answer comes back in the same format as other models
   ▼  │
api.onysoft.com   key and device check → queue
   ▲  │
   │  │   outgoing connection: the Node app picks up the job, sends the result back
   │  │   (no port is opened on your computer)
   │  ▼
Your computer     Onysoft AI Gateway Node → local model (127.0.0.1)

Model Name

Add node/ in front of the catalog ID. If the same model is ready on more than one of your devices, the request goes to one of them; to pick a specific device, append @<device_no>.

You can find the names and device numbers you can use in the id and device_model_id fields of the GET /v1/node/models response. The catalog ID after node/ is case-insensitive; write the prefix itself in lowercase. If your key has a model restriction, add node/* to its list to allow all device models.

Example Request

The endpoint, the key and the request body are the same as for other models; only the model field changes.

curl https://api.onysoft.com/v1/chat/completions \
  -H "Authorization: Bearer sk-ony-your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "node/qwen3-4b-q4_k_m",
    "messages": [
      {"role": "user", "content": "Merhaba! Kısaca kendini tanıtır mısın?"}
    ],
    "max_tokens": 300
  }'

Example Response

The response has the same format as for other models. id is the request ID (req_...), model contains node/<catalog_id> without the device number, and cost.amount is always 0. If the model produced reasoning text, the message also contains reasoning and reasoning_content; if it called a tool, tool_calls is present and finish_reason is tool_calls.

{
  "success": true,
  "data": {
    "id": "req_5c1e9a7d2b3f4e6a8c0d1e2f",
    "object": "chat.completion",
    "created": 1758542400,
    "model": "node/qwen3-4b-q4_k_m",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "Merhaba! Ben Qwen3, bilgisayarınızda çalışan bir dil modeliyim."
        },
        "finish_reason": "stop",
        "logprobs": null
      }
    ],
    "usage": {
      "prompt_tokens": 18,
      "completion_tokens": 16,
      "total_tokens": 34
    },
    "cost": {
      "amount": 0,
      "currency": "USD"
    }
  }
}

Supported Parameters

Only the following fields are passed to your device:

• messages, max_tokens (or max_completion_tokens), temperature, top_p, stop, seed, presence_penalty, frequency_penalty, response_format, tools, tool_choice and reasoning_effort (only low, medium or high; other values are not passed to the device; it has an effect on models that support a reasoning level, for example gpt-oss). For streaming, stream and stream_options.include_usage are also taken into account.
• If you do not send max_tokens, the default is 2048; values above 8192 are lowered to 8192.
• If one of the devices that can serve the model runs it on the processor (CPU) and you did not send max_tokens, the default is 256 for non-streaming requests and 512 for stream:true requests. A value you set explicitly is left as is.
• n can only be 1; any other value returns 400 invalid_request.
• Fields not in this list are not passed to the device and are ignored.

Streaming

When you send stream:true, the response arrives as SSE chunks, just like with other models. With the current app versions (main app 0.4.5, distributed app 0.6.2 and later), the device sends the model’s reasoning text and the answer to the server as it generates them, so chunks reach your app and the dashboard chat while generation is still running. With these versions, closing the connection also stops generation on the device. Older versions send the answer only once it has been generated in full; the chunks then arrive back to back, in slices of about 40 characters.

As long as no data is flowing, the connection is kept open: the server sends a : ping comment line every 5 seconds. During streaming, long comment lines made of spaces may also arrive to flush intermediate buffers. SSE clients and the OpenAI libraries ignore these lines starting with :. Non-streaming requests have a shorter time limit, so we recommend stream:true for long answers.

: ping

: ping

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{"content":"Merhaba! Ben Qwen3, bilgisayarınızda çal"},"finish_reason":null,"logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{"content":"ışan bir dil modeliyim."},"finish_reason":null,"logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542400,"model":"node/qwen3-4b-q4_k_m","choices":[{"index":0,"delta":{},"finish_reason":"stop","logprobs":null}]}

data: {"id":"req_5c1e9a7d2b3f4e6a8c0d1e2f","object":"chat.completion.chunk","created":1758542401,"model":"node/qwen3-4b-q4_k_m","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":16,"total_tokens":34},"cost":{"amount":0,"currency":"USD"}}

data: [DONE]

Chunk order: role, reasoning text if any (reasoning / reasoning_content), content, tool_calls if any, then finish_reason. If you sent stream_options.include_usage, a usage chunk follows. The last chunk always carries usage and cost, followed by [DONE].

Errors that happen after the stream has started (the device did not pick the request up in time, a timeout, a device error) do not arrive as an HTTP status code but as an error chunk inside the stream, and the stream closes with [DONE]. Errors about the key, the model name or the request body are returned before the stream starts, as a normal JSON error response.

data: {"error":{"code":"node_timeout","message":"..."}}

data: [DONE]

Listing Your Device Models

GET /v1/node/models returns the models loaded on the devices connected to your account. If the same model is loaded on two of your devices, you get two rows (with different node_id values). A model can be used right now only if its ready field is true. This endpoint also works with personal keys only.

curl https://api.onysoft.com/v1/node/models \
  -H "Authorization: Bearer sk-ony-your-api-key"

{
  "success": true,
  "object": "list",
  "data": [
    {
      "id": "node/qwen3-4b-q4_k_m",
      "object": "model",
      "created": 1758369600,
      "owned_by": "cihazim",
      "name": "Qwen3 4B",
      "purpose": "genel",
      "context_window": 8192,
      "node_id": 12,
      "node_name": "MacBook Pro",
      "device_model_id": "node/qwen3-4b-q4_k_m@12",
      "online": true,
      "sharing": true,
      "model_status": "hazir",
      "ready": true,
      "last_seen_at": "2026-09-22T14:05:31+03:00",
      "pricing": {
        "input_per_million_tokens": 0,
        "output_per_million_tokens": 0,
        "currency": "USD"
      }
    }
  ],
  "total": 1
}

Error Codes

Error responses have the same format as on the other endpoints: {"success": false, "error": {"code": "...", "message": "..."}}. Error messages are in Turkish.

Limits

• Only personal keys you created on the API Keys page of the dashboard can be used; project and partner keys get 403 node_owner_only.
• At most 4 requests at the same time per user (queued and running requests are counted together).
• Without streaming, the device has 30 seconds to pick the request up and 120 seconds to produce the answer; the total wait never exceeds 160 seconds. With stream:true these limits are 60 and 180 seconds (300 seconds if the model runs on the processor).
• The context window is the model's context_window value (8192 tokens if none is listed). The prompt length is estimated at roughly 4 characters = 1 token.
• Messages can be at most 256 KB and the whole request body at most 512 KB; you can define at most 64 tools and at most 4 stop strings.

Privacy and Security

• Your request goes only to your own devices; it is never routed to another user's device. Nobody else can send requests to your device this way either.
• Your computer does not open any port to the outside. The connection is always made by the Node app towards the server, so no extra setup is needed behind a router, NAT or firewall.
• As with other models, every request is written to your usage logs together with token counts, duration and the request/response content.

Program Security

Download Onysoft AI Gateway Node only from the download page in your dashboard; do not use copies from other sources. The app accesses your computer and your account only as far as it needs to do its job:

• macOS: The package is signed with an Apple Developer ID belonging to Onysoft Veri Merkezi A.Ş. and notarized by Apple. macOS checks this signature itself when the app opens; on first launch you only get the standard “downloaded from the internet” confirmation.
• Windows: The package is not yet signed with a code-signing certificate, so Windows SmartScreen may show a “Windows protected your PC” warning. If you downloaded the file from the download page and the SHA256 checksum matches, continue with “More info” → “Run anyway”.
• SHA256 verification: The download page shows the SHA256 checksum of every package. Compare it with shasum -a 256 <file> on macOS or certutil -hashfile <file> SHA256 on Windows. If the checksum does not match, do not open the file; delete it and report it to info@onysoft.com.
• Your password and API keys are never stored in the app: The app connects to your account with a one-time device code that you approve in your browser, and receives an access key for that device only. The key is kept in the Keychain on macOS and in the operating system’s encrypted store (DPAPI) on Windows, and is never written to the log file; the server keeps only an irreversible hash of it. When you sign out of the app, the key is deleted from your computer and invalidated on the server (this needs an internet connection at sign-out).
• Verified files: The model engine (llama.cpp) and the model files are downloaded only over HTTPS from their official sources, and are not used unless they match the published SHA256 checksum.
• Other people’s data never reaches your device: Other customers’ requests are not routed to any device. Your device runs only your own requests and short test requests that Onysoft sends to check that the device works correctly.

Modifying the app, attempting to decompile it or repackaging it is prohibited by the Acceptable Use Policy (except where mandatory law permits). If you believe you have found a security vulnerability, please report it to info@onysoft.com.

App Updates

The app checks for a new version at startup and at regular intervals while it runs; when a new version is out, it tells you in its window and offers to open the download page. The app never updates itself; you download and install the update. Simply install the new version over the old one: your settings, device connection and downloaded models are kept.

If every device on which the model is ready runs an unsupported version, the request returns 503 node_update_required. In the GET /v1/node/models response, such a device’s rows show update_required: true and model_status: "surum_eski", and ready is false.

Installing Models on Another Disk

Model files can take tens of GB. If your system disk is full, you can install models on a second or external disk: in the app, click the “Boş disk” (free disk) row on the hardware card and pick a disk from the list (the app’s interface is in Turkish). Models are installed in the OnysoftNode/models folder on the selected disk (OnysoftNodeDagitik/models for the distributed edition); the app itself, its settings and its log file always stay in the application folder.

• Only local, writable disks attached to the computer are listed; network drives, CD/DVD drives and read-only volumes are not. Disks formatted as FAT show a warning, because model files larger than 4 GB cannot be written to them.
• When you change the disk, installed models are moved to the new disk: they are copied first, the setting changes only after copying finishes, and the old files are deleted last. You can cancel the move; if it is interrupted, the models stay intact on the old disk.
• The disk cannot be changed while sharing is on or while a model is being installed; turn sharing off first.
• If the selected disk is not connected, the hardware check shows “Yetersiz” (insufficient) with the reason, and the disk list opens automatically; you can reconnect the disk or choose another one.

Distributed Edition: One Model on Two Devices (beta)

Onysoft AI Gateway Node Dağıtık (distributed edition) is a separate app that runs a model which does not fit in a single computer’s memory (for example gpt-oss-20b, which does not fit on an 8 GB graphics card alone) on two of your own devices together. There is no model selection in the distributed app; you only open the device for computing (graphics card or processor). You choose and install the model in the dashboard based on the group’s total capacity, and the model’s layers are split between the two devices. Once the group is ready, the model is used as the main device’s model in the dashboard chat and in the API under the name node/<catalog_id>. The main app continues as it is; if the model fits on a single computer, the main app is faster.

Limits

• The model is chosen in the dashboard based on the group’s total capacity; only catalog models can be installed. Models that do not fit in the group’s memory cannot be selected. When the model changes, both devices ask for approval again.
• A group has at most 2 devices: the main device, which downloads the full model, and the helper device, which downloads only its own share of layers. A device can be in only one group at a time.
• Only devices linked to your own account with the distributed app installed can be in the same group.
• The setup we have measured running across two devices: gpt-oss-20b, with a Windows computer with an NVIDIA graphics card (CUDA) as the main device and an Apple Silicon Mac (Metal) as the helper. For now the helper can only be an Apple Silicon Mac; Windows graphics cards (CUDA, Vulkan) and processors have not been verified as helper devices yet. The dashboard does not install a model with a helper type that has not yet been tested for that model, and shows the reason in the list. A computer with only a processor can also be the main device (this setup has not yet been tried on two separate computers). Both devices must run the current version of the distributed app and the same runtime version.
• Data passing through the relay server is limited to 100 GB per month (at most 20 GB in a single session); a direct connection on the same local network does not count toward this limit. On different networks each generated token uses ~35–45 KB of data (a 1,000-token answer ~35–45 MB); long inputs (for example a document to summarize) also transfer additional data between the devices while they are being processed. While the devices are idle there is ~40 KB of control traffic per minute, and each time the group model loads ~40–70 MB is transferred once. If one of the devices is on mobile data, this data comes out of your mobile plan.
• Customer requests are never sent to groups; a group is only for your own requests (dashboard chat and your personal API key).
• Every token needs one round trip between the devices, so speed depends on the connection. In our measurement (gpt-oss-20b; a Windows main device with an NVIDIA RTX 4060 8 GB graphics card + a 16 GB MacBook with an Apple M5 chip as the helper), the same local network produced ~18 tokens per second; a short question took ~3.8 s and summarizing a 2,033-token text took 9.8 s in total. On different networks (helper on mobile data, connection through the relay server) the speed dropped to ~4 tokens per second and the first part of the answer arrived after ~12 s; long inputs make this wait longer. If you do not set max_tokens for a group model, the answer length is limited based on the measured speed; use stream:true for long answers.

Group Connection Security

• Devices on the same local network connect to each other directly; otherwise the connection goes through the Onysoft relay server. In both cases the connection is end-to-end encrypted (TLS 1.3, device certificates are verified mutually). The relay only carries the encrypted data and cannot see its content; only the amount of data transferred and the duration are recorded. With a direct connection, a small control connection to the relay server also stays open. You do not need to open a port on your router; for a direct connection the app listens on the local network only briefly while the connection is being set up (Windows may ask once for firewall permission).
• You choose the group’s model in the dashboard; until you give your explicit consent with “Install and open”, the group stays off and the devices download nothing.
• Local approval on every device: Before a device joins a group or downloads anything, the app asks for your approval on that device as well. The approval screen shows the group name, the device’s role, the other device’s name, role and security fingerprint, this device’s own fingerprint, the model name, the size to be downloaded to this device and the memory this device will use. A device you have not approved downloads nothing and does not connect to the group; even if someone gains access to your web account, they cannot add your device to a group without your approval.
• Compare fingerprints: The fingerprints in the app (for example ABCD-EF01-2345-6789) must exactly match those shown on the Device Groups page in the dashboard. If they differ, do not approve; turn the group off and renew its security secret.
• If the group’s model changes, a device is added to or removed from the group, or a device’s role changes, approval is requested again. A device on which you rejected the group does not connect to it; if you rejected it by mistake, use the “Kararı geri al” (undo decision) button in the app. If you later withdraw your approval, the group connection closes immediately.

Pricing

Using your own device is free; nothing is deducted from your balance. Requests appear in your usage log with an amount of 0, and token counts and duration are still recorded.

Frequently Asked Questions

Why is there no separate endpoint?

OpenAI-compatible tools such as n8n, Cursor, LangChain and the OpenAI libraries expect /v1/chat/completions for chat. Because we use the same endpoint, you do not have to change the address or the key in these tools; you only change the model name. The node/ prefix makes it explicit that the request goes to your device, not to a catalog model.

What happens if my computer is off?

If none of your devices is online with the model ready, the request returns 503 node_unavailable. If your computer has only just shut down, the server still counts it as online for 45 seconds; in that case the request is queued and the error arrives once the pickup time runs out (30 seconds without streaming, 60 seconds with streaming). In a streaming request, this error arrives inside the stream as an error chunk. The request is never moved to another device or another model.

Can I use it with a project key?

No. Device models only work with the personal keys you create on the API Keys page of the dashboard; project and partner keys get 403 node_owner_only.

Can someone else use my device this way?

No. A request goes to the devices of the owner of the key that sent it. A request sent with your key only runs on your devices, and a request sent with someone else's key only runs on theirs.