Is Jev the future of spam filtering?

This month TypeSafe AI came out of stealth with Jev, a hosted model that cannot chat: it takes a typed question and returns a probability. “Is this email spam?” is the obvious question to ask it, so I pitted it against Klar’s own model. It is billed per token, which came to 11 cents per thousand of my messages.

Jev catches more of my spam than the model I ship, for eleven cents per thousand messages. It is not a fair fight: Klar’s filter runs on any Apple silicon Mac and nothing it reads leaves the machine. Jev runs in a data centre.

Jev catches more, and misses newsletters and cold pitches

I took the 5,000 messages I measure Klar on: 2,500 spam (2,456 from a public 2026 corpus, 44 from my own junk folder) and 2,500 legitimate, drawn in proportion from my 2025 receipts, notices and opted-in newsletters. Person-to-person mail stayed on the Mac. Three models read them: Jev, the model Klar ships today, and the one it replaced last year.

On the same messages, Jev catches 151 spam that Klar misses and misses 11 that Klar catches (the same messages both ways, so this is not chance: p below 10^-30). On receipts and notices it flags 6 where Klar flags 18.

I read all six. Two are spam that had been filed as legitimate (a forwarded “final hours” offer, a “Metamask Team” notice through a newsletter platform); Klar flags the second too. Four are legitimate mail that looks like phishing, such as an exchange changing its bank details and a bank’s recruitment site on a SaaS domain.

Jev’s misses are two families. It scores the newsletters I junked (an AI-tools digest, a game’s seasonal mail) 0.1 to 0.5: I subscribed once, and its definition is unsolicited bulk mail. B2B pitches (“business inquiry”, “get back to me”) land on its scale at 0.43 to 0.63; Klar, trained on thousands of them, catches 9 of the 11 at 0.99.

Klar catches both families because it was trained on thousands of them. Jev has no open weights to fine-tune, so the only way to have both is to run the two together.

Jev’s threshold here is 0.640, the lowest at which it wrongly flags no more legitimate mail than Klar does at its own threshold of 0.99. I set it on half the 5,000 and report on the other half. Below 0.6 its scores run low on my mail, which is why the threshold is not Jev’s own 0.5.

Jev repeats itself: asked the same 500 messages five times, it moved a score by 0.006 on average and changed side on 2 rows in 500. Under 0.02 is noise.

I also handed Jev Klar’s verdict and score with one question, “is that verdict wrong?”, on the 2,240 messages Klar flagged with confidence. It unflags 15 of the 43 false positives and not one true spam (0.09 on average where Klar is right, 0.80 where Klar is wrong). A wanted message Klar is sure about is the one mistake no threshold can undo. This reaches those.

Jev cares what it reads, not how I ask

Then I sent it the wrong thing on purpose. Sending only the subject and the sender catches 0.98 of the spam it catches on the full message, at a fifth of the price, and no message content leaves the server at all.

Rephrasing the question does not move it either. Each probe went to the 120 hardest messages (the 72 the main run got wrong, the 48 nearest its threshold) and forty easy ones where a change can break something, counted as hard messages fixed and broken.

What it reads is another matter. Jev believes a “this message failed DMARC” line written in the body: 14 of 38 legitimate messages near the threshold rise by more than 0.1. Sent the raw source, which I do not send, it reads an authentication header the sender wrote himself as a pass: 15 of 82 spam messages near the threshold cross to “not spam”.

Jev does not care how I ask, only what it reads. Most of its lead over Klar is that it reads more: on Klar’s 128-token view it loses 31 hard rows and 9 easy ones.

Written to fool Jev, and one in five hundred got through

I wrote thirty-five messages for the shapes a filter meets and the tricks a sender tries. Among them: an authenticated “new login” alert, a “PayPaI” that authenticates its own lookalike domain, apple.com with a Cyrillic a, the boss on a free-mail address asking for gift cards, an attachment named Invoice_2026-09.pdf.exe. Thirty-two have a side, three of them controls, and Jev gets twenty-seven.

The same receipt and phish in thirteen languages, English to Russian, Japanese, Arabic and Turkish: Jev scores every receipt 0.02 to 0.03 and every phish 0.94 to 0.98.

One of five hundred rewrites got under the threshold. Those are the hundred spam messages Jev is surest about, each rewritten five ways with every link and the ask kept: formal, condensed to a third, in French, with a claimed prior relationship, in a receipt’s layout. Klar catches 96 of the 100 originals and misses 69 of the rewrites.

Everything else a sender can do for free (unicode tricks, HTML tags, capitals, an unsubscribe header, a Reply-To at a big brand) moves nothing, or moves the wrong way for him. The one lever is on the other side: a corporate signature block pushes borderline legitimate mail toward spam by about 0.1.

The colleague who gets past Jev

Two misses at the top of the table point the same way, so I put thirty real spam messages inside a colleague’s mail. Quoted under one line of her own, 11 of 30 are still spam to Jev; forwarded with “is this real?”, 26 of 30; forwarded bare, 30 of 30. Twenty words of hers bring it to 8 of 30, a hundred to 2 of 30.

The spammer cannot do the same from his own address: his pitch over six hundred quoted words of a genuine thread stays spam 30 of 30, and a hundred words of office prose over the quoted pitch is still spam 24 of 30. What counts is who wrote the newest part and how much they wrote. A quote under a pitch buys the spammer nothing.

My first count of that reply was 23 of 30, not 11: a reviewer found the build had put her address and her authentication into the message and left the spam’s envelope sender and date in place. With every field hers, 11 of 30. The other twelve were Jev reading an envelope that disagreed with its authentication, which a spammer cannot fake from his own address.

Not the teacher

I also tried Jev as a labelling teacher against Claude Haiku, which labels my training corpus with a human pass behind it. On the 170 messages I judged by hand, in four classes, Haiku agrees with me 123 times and Jev 115, and they fail in opposite directions: Haiku calls legitimate marketing spam, Jev calls 15 of 19 spam “marketing”.

So no: Jev stays out of the teacher’s chair. Where the two disagree is the queue a human should read.

Where this leaves us

Did I fool it? Barely: one rewrite in five hundred, a hundred words of the spammer’s own prose over a quote, and a planted authentication header in raw source I do not send. What gets past it is a real person quoting the spam, a hole in what I send rather than in the judge.

Jev is hosted with no open weights. But a model that answers one narrow question with a probability need not be large: the open ones that copy its shape run from 0.3 to 4 billion parameters, and an ordinary Mac already runs Klar’s 278-million-parameter encoder. Will a Jev-like model run on a consumer machine, for one narrow problem like spam, within a few years? That question decides Klar’s next shape.

Yes on a server, today, behind Klar’s own model. On your Mac, not yet. If a Jev-like model gets there, everything stays private on the machine, the case I made for smaller models that own one task. If it cannot, the second opinion stays what it will be in Klar Plus: an optional cloud second line of defence, off by default, for the mail Klar is unsure about, and never in the free app on your Mac.

The Klar score here is the model’s, not the product’s

Every Klar number on this page is one model alone: the encoder, at its threshold of 0.99. Klar does more with your mail. The verdict folds that score with a small model that learns from your own corrections (FTRL), then with checks on who sent the message rather than what it says:

• Who signed it: SPF, DKIM and DMARC, read from the headers.
• Who it claims to be: a sender named after a bank or a brand, from a domain that brand does not own.
• Where its links really point.
• Whether it answers a message you sent, which keeps a reply out of junk.

So Jev was measured against Klar’s model, not against Klar. The full verdict on these panels is a different number, and the layer that makes it is public, with the engine, at github.com/klar-im/engine.

He gets the last word

How it was run, and what it does not prove

• Every message went out scrubbed (credentials, one-time codes, card numbers, addresses, crypto addresses, keys, my name) in Jev’s own message format: subject, body and sender headers. The raw log of every request and reply stays with me.
• Every number is from the half of the messages I kept for reporting, split by a hash of each one. One request per message asked sixteen questions: the broad one, a four-class choice, Jev’s six, seven of mine, and a four-level score of how wanted the mail is.
• These are my numbers on my corpus, which is not your inbox. I say the same of every vendor, myself included.
• The public spam panel is a public corpus and reads as an upper bound; a model may have trained on it.
• One run of one model version, jev-1.13.0, on one day; Jev’s API moved between my reading of its docs and the run.
• Nothing here runs on a Mac. Klar’s free Mac app and Klar for Messages on iPhone send nothing off the device; the hosted second opinion is an option of Klar Plus, still in development, off until you switch it on.

I measured throughput from Lyon, where an empty call to Jev’s US West host is 584 ms of the 680. The model’s own share is under 100 ms and does not move with concurrency. The wait is the ocean, not the model.

All posts