“Make it a bar chart” is an expensive request if you send it through a full agent run.

The chart already exists, and your agent has the context in the thread. But a small change can still mean waiting for a model to reason through the request and work out which tools to call.

I've been working on moving these follow-ups out of the LLM loop with Jev, TypeSafe’s decision model. It powers Lightdash’s Fast mode, which dropped median response time from 17.5 seconds to 1.7 seconds.

Here’s how we got there.

How does Jev make a chart edit faster?

My approach was to use Jev to interpret the request, then let our existing code apply the change. For each follow-up, Jev returns probabilities for a fixed set of options. Lightdash uses these to identify the action you requested and the fields or values it needs. Those decisions can be evaluated in parallel in a single request.

For “make it a bar chart,” Jev identifies the action as “change chart type” and the type as “bar.” Lightdash’s code uses those selections to prepare the edit. If it passes the confidence and completeness checks below, Lightdash applies it without a full agent run.

Simplified follow-up flow. Confidence checks determine whether a request can use each shortcut.

How much faster are responses using Jev?

To measure the difference, we ran 4 scripted conversations with 25 follow-ups on our own analytics instance. Each message went to two threads side by side: one with Fast mode on and one running the normal agent. Both used claude-opus-5-5.

Here are the results:

72% of follow-ups never called the agent LLM. Agent LLM token usage fell by about 87%, or 76% when including Jev’s tokens.

Here’s a separate side-by-side example, with both threads using gpt-6-luna:

Here’s how long each response took in the example above, turn by turn:

Those six follow-ups took 5.5 seconds with Fast mode, compared with 59.8 seconds without it. Fast mode used 0 agent LLM tokens and 53k Jev tokens, compared with 970k agent LLM tokens for the normal agent. Both threads ended on the same chart.

How does Lightdash check Jev’s decisions?

Speed only helps if the edit matches what you asked for. So, Fast mode has three safeguards:

• Jev only chooses from known options. These include fields in your explore, values returned by your warehouse, and numbers in your prompt.

• Each choice must clear a confidence threshold. The minimum probability is 0.45 for the kind of edit, 0.5 for the field or option, and 0.6 for a filter value. Below that, the agent takes over.

• A separate check tests the whole request. Before anything changes, Jev checks whether the planned edit covers everything you asked for. If it doesn’t, the request goes to the agent.

What did we learn from testing?

The timings were encouraging, but the classification tests and iterative improvements showed me where Jev needed work: it missed some filter values already on the chart, and twice a request handed back to the agent saved a "working on it" message as the final answer.

This left me with a few practical lessons:

1. Test the path users take. A harness that calls a different endpoint will pass while the real one breaks.

2. Measure noise first. Identical runs can disagree on 5 of 30 prompts. Only count changes that repeat.

3. Check the experiment itself. If both sides of a comparison share results, you aren’t measuring them independently.

4. Track usage as well as errors. A feature can stop firing without a single error. Track how often it runs, not just whether it fails.

How can you try Jev in Lightdash?

If you're a Lightdash customer, ask us to switch on Fast mode for you. Let us know how it works and help us improve Lightdash Agents!