Laya or Jev? We Installed the Open-Source Typed Decision Model on Our Own Server and Tested It in Turkish

In two weeks typed decision models went from a category to a race. First came the closed-source Jev, then Laya arrived claiming to do the same job under an Apache 2.0 licence with open weights. Most of the comparisons circulating online are a single chart: a local model's milliseconds placed next to a cloud model's network round trip.

That comparison tells us nothing. Local being faster than the internet is the expected result. The three questions we actually had were different: does it work in Turkish? does it really run on our own server? and which one for which job?

We installed both and measured them against our own labelled data. We found something we did not expect: changing the format of the question moved the same model's accuracy on the same data from 8/20 to 20/20. That finding matters more than either model, because the same trap is waiting for anyone using typed decisions.

Two Models, the Difference in One Sentence

Both do the same thing: they produce no text, they return a typed, probabilistic answer to the question you ask. Three question types are shared — noul (yes/no probability), choice (option plus confidence), score (a score across labelled levels).

	Laya	Jev
Licence	Apache 2.0 (code + weights)	Closed, via API
Size	421M / 322M parameters	Not disclosed
Runs on	Your own hardware	The provider's cloud
Architecture	Encoder based, single pass	Not disclosed
Language	Multilingual checkpoint covers 100+	English first (its own docs warn)

Laya ships three separate checkpoints: a general English one, a multilingual one, and one trained specifically for typed decisions. That split brings the first trap with it — we will get there shortly.

Installation: 16 Cores, 1.2 GB, Zero Surprises

We assumed installing Laya on a shared web server would be impossible. We were wrong. The install needs no root privileges; the Python virtual environment goes into user space.

The real numbers from our own server:

Item	Value
Install size (including CPU build of PyTorch)	1.2 GB
Model load time	17 seconds (single checkpoint)
All three checkpoints (preload=True)	5.3 GB of memory
Site response time during install	0.49 s (unchanged)

Worth noting: the ~1 GB mentioned in the repository is for a single checkpoint. preload=True loads all three and holds 5.3 GB resident. If you run it as a service, load only the checkpoint you actually serve.

Latency measured on CPU (no GPU, 16 cores):

Checkpoint	Time per question
Multilingual (322M)	52 ms
Typed decisions (421M)	444 ms

The First Trap: The Router Looks at Language, Not Task

Laya's Router class is presented as sending "each request to the right checkpoint". We ran our first measurement with the defaults and the results were dreadful: on a 446-model classification task it gave 399 of them the same label.

The reason is simple and documented, but easy to miss: by default the router looks only at language, not at the task (auto_task_detection defaults to off). So when you send English text it goes to the general English checkpoint rather than the one trained for typed decisions.

The fix is to name the checkpoint explicitly:

from laya import Router

router = Router(default="multilingual", max_loaded=1)
router.load("multilingual")

# Name the checkpoint EXPLICITLY
result = router.predict(text, questions, model="multilingual")

This is where we caught our own mistake mid-measurement, and it taught the same lesson once more: when a model gives you a bad result, first check that you are using it correctly.

The Real Finding: Question Format Moved Accuracy from 8/20 to 20/20

We tested with our actual job: the question "is this customer asking for a human?" on support tickets. We built 20 Turkish support messages, five of which clearly requested a human and fifteen of which did not. We labelled them by hand.

Same model, same 20 messages, only the way the question is written differs:

Question format	Correct
Turkish instruction + criteria defined	8 / 20
Turkish instruction, no criteria	14 / 20
English instruction, no criteria	20 / 20

That is not a marginal improvement; it is the difference between an unusable result and a perfect one. And the 20/20 in the last row holds at every threshold between 0.5 and 0.9, because the probabilities separate cleanly: the five messages requesting a human land between 0.976 and 0.996, the other fifteen between 0.000 and 0.166.

Two practical rules follow:

• Write the instruction in English even when your text is Turkish. The model is multilingual, but it is markedly more accurate when it reads the instruction in English.
• Leave criteria out on binary questions. Counter-intuitive, but the measurement is clear: adding criteria reduced accuracy. If the instruction itself is clear enough the model uses it; extra criteria add noise.

The failure pattern was instructive too. In the wrong format the model gave "Thanks, the issue is resolved" a human-request probability of 0.88, and a pure technical bug report 0.97. In the right format both dropped below 0.04.

Laya and Jev Side by Side

The same 20 Turkish messages, each model asked in its own best format:

Measurement	Laya (multilingual)	Jev
Binary decision (noul) accuracy	20 / 20	20 / 20
Category (choice) accuracy	9 / 20	13 / 20
Dissatisfaction (score) mean error	0.91 / 3	0.21 / 3
Latency per question	52 ms	1,059 ms
Runs on	Our own server	Provider cloud

The table tells a clear story: on binary decisions both are perfect, but Laya does it 20x faster and on your own hardware. On the other two question types Jev is distinctly better.

The gap on the score question is especially sharp. Whatever the message, Laya squeezed the score between 1.06 and 1.96: "Thanks, the issue is resolved" got 1.29, "This is a disgrace!" got 1.81. A 0.5 gap is not enough to set a threshold on. Jev measured the same test with a 0.21 error; it genuinely discriminates.

What Laya Cannot Do: Know What the Text Does Not Say

We tried a second real task: reading the descriptions of 446 models in our catalogue and answering "should this model be listed in a corporate catalogue?". We had a hand-built label set of 15 models.

	Laya	Jev
Caught of the 15 labelled	no separation	14 / 15
False alarms across 431 normal models	—	0

On this task Laya could not separate the risky group (mean 1.77) from the normal one (mean 1.73). We do not count that against Laya, because the task is unfair: whether a model is positioned for adult content usually is not written in its description; it lives in the publisher's reputation. A 421-million-parameter encoder does not carry that world knowledge.

The right distinction: Laya classifies the text itself, Jev can also decide using knowledge beyond the text. Intent, sentiment, category and urgency belong to the first group; judgements requiring world knowledge belong to the second.

Cost: Free Is Not Always Cheap

Running Laya has no token cost — but it is not free either. The real cost lines are:

• Memory. ~1.7 GB for a single checkpoint, 5.3 GB for all three. That memory stays allocated.
• Cold start. 17 seconds to load. Which means you need a process that keeps the model resident, and you need to keep that process alive.
• Operations. If the process dies the endpoint returns 503. We wrote a watchdog that checks every minute; when we killed the process on purpose it came back in 16.7 seconds.

So "open source equals free" holds only at small volumes. If you make a few hundred decisions a day the cloud model is cheaper. At tens of thousands a day the local model pulls clearly ahead.

Current catalogue prices:

Model	Input (1M tokens)	Output	Supported types
convai/laya-multilingual	$0.042	Free	noul
typesafe/jev	$0.063	Free	noul, choice, score

Which One, When

The split we ended up using ourselves:

Job	Choice	Why
Yes/no gate, high volume	Laya	20/20 accuracy, 52 ms, no token cost
A decision in a flow where a user is waiting	Laya	No network round trip; even the slowest request stays under half a second
Multi-option classification	Jev	13/20 against 9/20 in our measurement
Measuring degree or severity	Jev	Laya's score head does not separate
Judgement needing knowledge beyond the text	Jev	A small encoder has no world knowledge
Data must not leave your server	Laya	The request never goes out

In practice the two are not mutually exclusive. The pattern we built: the cheap, fast binary gate runs first, and anything ambiguous or needing a degree goes to a second stage.

Using Them on Onysoft

Both models are in the catalogue and work through the same endpoint. You do not need a separate provider contract or a server of your own:

curl https://api.onysoft.com/v1/decisions \
  -H "Authorization: Bearer sk-ony-YOUR-KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "convai/laya-multilingual",
    "state": "I want to speak to a real person, not a bot.",
    "questions": {
      "wants_human": {
        "type": "noul",
        "instructions": "Does the user ask to talk to a human agent instead of the bot?"
      }
    }
  }'

The response is not text you have to parse, it is a number you can use directly:

{
  "success": true,
  "data": {
    "model": "convai/laya-multilingual",
    "answers": {
      "wants_human": { "type": "noul", "noul": 0.9626, "confidence": 0.9626 }
    },
    "usage": { "input_tokens": 65, "output_tokens": 0 },
    "latency_ms": 52
  }
}

You can ask up to 32 questions in one request; all are answered in a single pass. Only input tokens are billed. See the typed decisions API documentation for details.

Summary

Measuring both models against our own data, this is how it came out:

• Both score 20/20 on binary decisions. The difference is not accuracy; Laya does it 20x faster and on your own hardware.
• Jev is clearly ahead on choice and score questions. Laya's score head did not separate at all in our test.
• The most important finding is not about either model: writing the instruction in English and leaving criteria out moved the same model from 8/20 to 20/20 on Turkish text.
• Installation is lighter than assumed (1.2 GB, no root needed) but memory is heavier than assumed (5.3 GB for three checkpoints).
• A small encoder cannot know what the text does not say. Judgements needing world knowledge still require a large model.

Other articles in this series: what typed decision models are and typed decision model or chat model.

Browse the catalogue on /models, or create a free account to try it.