Five questions collapse into one request; a filter and a join justify a second

Key ideas

  • Five questions about one customer = one request, not five.
  • A second stage needs a real dependency: an earlier answer, new candidates, new context, or a new model.
  • Fewer requests, same number of recorded judgments.

Five questions about a customer do not make five model calls. In Deduced they make one, unless one of them cannot be asked until another is answered.

The packing rule

The compiler groups questions that can be answered from the same state with compatible model settings into a single request. Their answers do not become hidden context for each other. TypeSafe’s documentation is explicit that sibling answers stay separate, and the trace keeps one recorded judgment per question either way.

If a later question actually needs an earlier answer, that is a dependency. It goes to the next reducer.

What justifies a second reducer

  1. A computed answer feeds the next question. A rubric that consumes returned signal values has a true dependency.
  2. The candidate set changes. Only the customers who passed a filter get the expensive follow-up.
  3. New context is retrieved. The ticket example screens on recent tickets, then joins account history for the customers that were kept. The second reducer reads data the first one never had.
  4. The model changes. A stage pinned to a different model is a different request.

Capacity is a fifth reason. When a question set does not fit the provider’s request limit the compiler splits it and records why.

One question per request4 customers × 5 questions = 20 requests
Cancel intentConcernSeverityReasonExplicit intent
Pack questions that share the same stateIllustrative run: 4 customers screened, 2 selected for follow-up

Customer signals

Cancel intent
Noul
Concern
Choice
Severity
Score
4 customers · 4 requests
Filterp(intent) ≥ 0.65
JoinAdd account history

Investigate reasons

Reason
Choice
Explicit intent
Noul
2 customers · 2 requests
6 model requestsThe second stage reads new context for the selected customers.
Three questions share the first request. Two follow-up questions share the second. Filtering and joining are code steps, with no model calls.

The ticket example, counted

  1. Group by customer. 5 tickets, 4 customers. No model call.
  2. Customer signals. One request per customer, three questions inside. 4 requests.
  3. Keep high-risk customers. p(cancel) at or above 0.65. 2 kept, 2 excluded. No model call.
  4. Attach customer history. 3 history records joined on the customer key. No model call.
  5. Why might they leave? Two questions over the joined state. 2 requests.

Six requests for five questions answered across four customers. If the second reducer’s questions could be answered from the first state, the compiler would fuse them and send four requests, not six. A speculative question is cheap when the state is already in the request. A gate is worth it when the follow-up needs data the first stage did not have, or when the follow-up is expensive and most units will not need it.

Why it matters at scale

An upstream provider’s request and token limits constrain throughput. Adding workers does not remove an account quota. Packing compatible questions can reduce the number of requests, but the effect on runtime depends on input length, model latency and the limits of the selected provider.

What packing never does

  • It never merges two questions into one vaguer prompt. Atomic questions stay atomic.
  • It never puts unrelated records into one state to save a call. Shared-array packing is a separate optimisation that needs its own quality measurements, and the docs warn that unrelated context lowers accuracy.
  • It never drops a trace. Fewer requests, the same number of recorded judgments, each mapped to the request that carried it.

Explore the interactive example, or join the waitlist to try your own questions when early access opens.