Checking Relevance with Verdict4s

A few years ago I set up a site that curates Scala-related articles (see Scala News) from the blogs of well-known Scala developers using RSS. This involves fetching all new articles from a list of bloggers’ RSS feeds. One problem with this was that even though these bloggers mostly write about Scala, occasionally they don’t. Some of the posts were also sales pitches or recruitment-focused.

I wanted to filter out irrelevant articles, and my attempt to solve this was quite simplistic. Let’s take a look at a code snippet of it:

private def isAboutScala(entry: SyndEntry): Boolean = {
    val hasRelevantCategory = Option(entry.getCategories)
      .map(_.asScala.toList)
      .getOrElse(List())
      .exists(category => {
        val name = category.getName.toLowerCase
        name.contains("scala") || name.contains("sbt")
      })

    val hasRelevantTitle =
      Option(entry.getTitle)
        .map(_.toLowerCase)
        .getOrElse("")
        .contains("scala") ||
        Option(entry.getTitle).map(_.toLowerCase).getOrElse("").contains("sbt")

    val hasRelevantDescription = Option(entry.getDescription)
      .map(_.getValue.toLowerCase)
      .getOrElse("")
      .contains("scala") ||
      Option(entry.getDescription)
        .map(_.getValue.toLowerCase)
        .getOrElse("")
        .contains("sbt")

    hasRelevantCategory || hasRelevantTitle || hasRelevantDescription
  }

You can clearly see the shortcomings of this. As a result, I had to vet the articles manually myself, which was inefficient and kind of defeated the whole point of automatically fetching the latest articles using RSS.

Discovering Decision Making Models

Back when I started with Scala, a long time ago, there was a company called Typesafe, as I am sure many will remember. It later changed its name to Lightbend, and more recently to Akka.

A couple of weeks ago I noticed a lot of conversation on social media and AI-related forums about TypeSafe AI. My immediate reaction was nostalgia, thinking it was the same company going back to its original name for the AI era 😁

That wasn’t the case. It was a new company with a new model called Jev, a new type of model that is optimised for making decisions, as opposed to current large language models that generate text and that you have a conversation with.

In practice, that means you don’t chat with Jev at all. You give it some information along with a question that has a fixed set of possible answers, like “Is this email spam, yes or no?” or “Which team should handle this support request?” Jev picks one of those answers and tells you how confident it is. It’s a bit like the difference between asking an advisor to write you an essay and asking them to tick a box. Because Jev can only choose from the options you give it, it can’t wander off or make things up, and it answers in a fraction of a second at a small fraction of the cost. That makes it well suited to the quick, repetitive judgment calls software has to make thousands of times a day.

Well, deciding whether an article or blog post is relevant is a decision made by answering a question, which makes Jev a perfect fit for the problem I am trying to solve.

Introducing Verdict4s

This got me excited about using Jev, the decision-making model, to improve how I filter out irrelevant articles. So I did what all good Scala developers do: I created a new library you can use to bring decision models into your Scala applications.

Verdict4s is a Scala 3 client for typed decision models. It supports Jev and OpenAI Decisions, and it runs on both the JVM and Scala.js. You ask typed questions and get typed answers back: choices, scores and probabilities, never free text you’d have to parse. If a choice question uses your own enum, the answer comes back as that enum. You don’t need casts or lookups by string key.

It uses Iron refined types to carry the services’ limits. A probability must be between 0 and 1, and a choice can have at most 255 options. Some invalid values won’t compile at all, for example Probability(1.5). The rest are caught before any request is sent.

It comes as three modules, so you only take what you need:

  • verdict4s: a working client in one line, with its own HTTP transport. It runs in any cats-effect Async effect, so IO works, and so does ZIO through its cats interop.
  • verdict4s-client: bring your own http4s Client[F] if you want to choose the backend.
  • verdict4s-core: only the types and codecs, with no effect system, so you can use it with Future or plain Java HTTP.

So whatever effect system you use, or none, there’s a way in. The details are in the docs.

How I use it to decide whether articles are relevant

I use Jev to ask three questions:

// All three questions go to jev in a single request.
  private[scalanews] val relevanceQuestions = Ask(
    (
      Question.noul(
        "Is this article about the Scala programming language or its ecosystem?",
        "It is mainly about Scala itself or Scala libraries, frameworks, tools or build tools (such as sbt, Mill or Scala CLI).",
        "It is mainly about something else, and mentions Scala only in passing or not at all."
      ),
      Question.noul(
        "Is this article just an announcement of a new version of a Scala library or application?",
        "It only announces a release: a version number, a list of changes or release notes, with little else.",
        "It has substance beyond announcing a release, such as explanation, tutorial, design discussion or opinion."
      ),
      Question.noul(
        "Is this article mainly sales or recruitment content?",
        "It mainly promotes a paid product, service, training or consultancy, or advertises jobs or hiring, rather than sharing knowledge.",
        "It mainly shares knowledge, experience or opinion, even if the author works for a company or mentions its products in passing."
      )
    )
  )

As part of Jev’s response you get a probability between 0 and 1 for each question: how likely it is that the answer is yes (the decision!).

These are the thresholds I have set for each question. An article is kept when “about Scala” is at least 0.3, and dropped if either of the other two reaches 0.7:

 // jev answers each question with P(yes). Excluding a real Scala post loses it
  // silently while including an off-topic one only costs a reader a skip, so
  // both thresholds lean towards keeping articles. Calibrated on real feeds:
  // off-topic posts scored at most 0.12 for "about Scala" while Scala
  // community news (e.g. Typelevel governance posts) scored 0.41-0.66; release
  // announcements scored 0.84-0.94 for "just an announcement" and substantive
  // posts at most 0.31. The sales/recruitment threshold isn't calibrated yet;
  // like the announcement one it only drops articles jev is fairly sure of.
  private[scalanews] val aboutScalaThreshold = 0.3
  private[scalanews] val announcementThreshold = 0.7
  private[scalanews] val promotionalThreshold = 0.7

And this self-explanatory snippet shows how the response and thresholds are used to decide whether to include the article or not:

/** Relevant when jev judged it about Scala, not just a release announcement
    * and not sales or recruitment content. Decided from stored probabilities,
    * so retuning the thresholds needs no new jev calls.
    */
  private[scalanews] def relevant(
      aboutScala: Probability,
      announcement: Probability,
      promotional: Probability
  ): Boolean =
    (aboutScala.value: Double) >= aboutScalaThreshold &&
      (announcement.value: Double) < announcementThreshold &&
      (promotional.value: Double) < promotionalThreshold

You can see the full code here: Relevance.scala

How has it worked out?

I ran it over the 83 articles Scala News collected from 16 blogs between January and August 2026. Jev kept 43 of them and filtered out the other 40:

  • 30 release announcements, such as “Scala 3.8.4 is now available!” and “sbt 2.0.0”.
  • 9 off-topic posts, including four chapters of a Rust Book series and a resume page. Every one of these mentions Scala somewhere, so my old keyword check would have happily let them all through.
  • 1 promotional post: “Announcing Scala Days 2026”. This one is debatable, as it’s a community conference rather than a sales pitch.

That’s 40 articles I no longer have to spot and remove by hand, which is exactly what I was hoping for.

What’s next

Because I store Jev’s probabilities rather than just a yes or no, I can retune the thresholds at any time without asking Jev again. That’s handy, as I’m not done tuning yet. The sales and recruitment threshold is still a guess, and the Scala Days post shows it needs calibrating against real feeds, the same way I did for the other two.

Try it yourself

If your code makes the same kind of decision over and over, such as routing, triage, moderation or filtering, give verdict4s a try:

// Mill
def mvnDeps = Seq(mvn"com.softinio::verdict4s::0.2.0")

// sbt
libraryDependencies += "com.softinio" %%% "verdict4s" % "0.2.0"

The docs will get you started, and the code is on GitHub. I would love to hear what you use it for, and pull requests are always welcome.