Putting a decision model to work on my X feed

When TypeSafe released Jev, I wrote an overview of decision models: what they are, how they differ from language models, and why their makers call them a System 1 for software. The claims were easy to repeat: a judgment in a third of a second, almost no cost per call, honest confidence. I wanted to see how one behaves on real input, so I tried it on something I do every day.

That something was my X feed. X is one of the places where I keep track of the latest news and developments in tech, mostly through search and lists that already pick the subjects for me. It is also known for a high share of posts that aren’t worth the attention: requests to like and repost, rage bait, vague hype that says nothing, launches with a link in bio. Posts like these spread because they work: in a study of 2.7 million posts, each word referring to the other political side raised a post’s odds of being shared by 67%. Skipping one takes a second, but on a long feed those seconds add up to more attention on the noise than on the news. Muting words doesn’t help, because noise has no keyword: a new model’s name turns up in careful benchmarks and in giveaways alike, so muting it hides both. Telling them apart takes the quick judgment decision models claim to offer, and asking a language model about every post in a scrolling feed would be too slow and costly. It looked like a fair test.

I left topics out of it. Search and lists already choose the subject, and they do it well. What they can’t do is judge how a post is written, so I asked Jev about the noise instead. Each post gets four questions: does it mainly fish for likes, reposts, replies or follows, is it mainly written to provoke anger, does it share little real information, and is it mainly selling something. Jev calls these noul questions. Each is written as an instruction with one sentence for yes and one for no, and the answer is the probability of yes. Here is the rage question, exactly as it is sent:

{
  type: "noul",
  instructions:
    "Is `post.text` mainly written to provoke anger or outrage?",
  criteria: {
    true:
      "`post.text` is mainly written to provoke anger or outrage.",
    false:
      "`post.text` is not mainly written to provoke anger or outrage.",
  },
}

The post itself goes along as context: the author’s name and handle, the text, any quoted text and the image descriptions. Jev reads it once and scores all four questions in the same pass. What comes back is four numbers and the cost of the call, so there is no text to parse and no answer that falls outside the question.

To see what those numbers look like without using anyone’s real posts, I wrote seven of my own, the kind my feed is full of, and sent them to Jev through the same code. This is the rage-bait one:

Two scores passed their limits, so the post collapses into a one-line bar that names the bigger miss, “Rage bait 0.95”. The closing “Share before they delete it” is what pulled engagement bait over as well. Here are all seven, against the limits I settled on:

The four noisy posts collapsed, each for the reason I would have given myself, and the three I’d want to read stayed open. The joke and the hot take scored higher than the CI write-up, but nowhere near a limit. Each call took between 286 and 484 milliseconds from my laptop and cost $0.000026, about 3 cents for a thousand posts. I sent the CI post three times to see how stable the answers are, and no score moved by more than 0.01.

Those limits are where the model stops and my own code takes over. Jev only gives scores, and the rule for what to do with them fits in one sentence: a post collapses when any score reaches its limit. I set them by scoring a few dozen posts I wrote for the purpose and placing each limit in the gap between the noise and the posts worth reading. Low substance got the highest limit, because noise of every kind scores high on it, and so can a good joke.

Keeping the decision outside the model mattered more than I expected. When a limit is wrong, I change a number instead of rewriting a prompt. It is also why I pin one dated version of Jev: the limits were measured against it, and a new version means measuring again.

The limits also had to keep learning, because what counts as noise is personal. In the overview, I described decision models handing their unsure cases to a slower System 2. On my feed, that System 2 is me. A collapsed post stays in place as a bar, one click from opening. When I keep one, each limit it crossed rises a little. When I hide a post that stayed open, the limit it came closest to drops a little, but only if the post came near it. If it wasn’t close, I probably hid it for its topic, and nothing is learned. Jev itself never changes. The limits drift, one click at a time, toward what I actually want to read.

Everything so far was about text. TypeSafe has since announced image input for Jev as an experimental feature, so I now send each post’s first photo along with its text. The announcement warns that every model, Jev included, does worse on images than on text, and my first tests agreed: a photo shifts the scores enough that limits set on text don’t carry over. So I treat photo judgments as unproven. Keeping or hiding a post judged with a photo doesn’t change the limits yet, and I’ll keep calibrating and checking them on real use before I rely on them.

Looking back, much of what I learned about Jev is about wording. A decision model has no prompt around its questions, so the questions are the whole interface, and one word can move an answer. A post that shares a benchmark and ends with “reply with your numbers” went over the engagement bait limit when I asked whether it fishes for replies, and stayed under it when I asked whether it mainly does. The questions also overlap: the rage post scored high on bait too, which is why each quality has its own limit and the bar names only one reason.

The speed claim held up. At about a third of a second, a post blurs for a moment while it is judged, then opens or collapses. Scores are kept for a week, so scrolling back or reloading doesn’t call Jev again. I haven’t tested the stronger claim, that a 0.9 is right nine times in ten. For this use, the scores only had to separate noise from the rest, and on my posts they did.

Kahneman’s warning, which I quoted in the overview, holds too: what you see is all there is. Jev sees the post, not the person behind it or the thread around it, so a reply that only makes sense in its thread is judged as if it stood alone. That is the main reason nothing is ever hidden outright. A wrong call costs me a second and a click, and I never lose a post I would have wanted.

What started as a test became the way I read X. Within a week it grew around the same goal of a calmer feed. Focus mode shows one post at a time. X’s menu and trending column stay out of view until I reach for them. Keys move between posts, and a list keeps what I have already read. I called it Xift.

It is free and open source on GitHub, and it runs in Firefox with your own OpenRouter key. For now it installs from the source; a listing on Firefox’s add-on site will follow. Only the post being judged leaves the browser, and it goes only to Jev, through OpenRouter. If you are weighing decision models for your own work, I’d suggest the same route: pick a small judgment you make every day and hand it over. A week of that taught me more about what these models can do than reading their benchmarks did.