Deduced

Score interview transcripts

Every interview scored against one rubric, with the quote behind each score.

Illustrative workflow · Sample data · Coming soon

Unit
One interview transcript plus the role’s rubric
Ask
Three competency scores. The evidence turn. Did they answer?
Example result
Scores with distributions, and the one answer to read

Fireflies · Files · 2 model stages · 7 questions · updated Sep 21, 2026

The flow

212 interviews source snapshot v4
212 Attach the rubric join by role_id, no model call
212 Score competencies reducer requests, 5 questions each
131 Keep problem solving ≥ 2.0 filter on the rubric's own scale
131 Attach the resume join no model call
131 Must-haves reducer requests, 2 questions each
131 Outputs output scored, with the quote

Same frozen rubric for every candidate. Must-haves only for those above the bar. 343 requests for 212 interviews. Every score links to a rubric version and a turn.

Follow one record

Interview I-0412, senior support engineer

snapshot v4, 42 minutes, 6 questions asked

  • Q2Walk me through a time a customer's problem turned out to be different from what they reported.
  • A2A customer said exports were slow. I reproduced it and the export was fine; their VPN was rewriting headers. I wrote up the header trace, sent it to their IT, and added a check to our diagnostics page so support could see it next time.
  • Q3What would you have done if IT had pushed back?
  • A3I had the packet capture, so I would have walked them through it live. If it still stalled I would escalate on our side with the evidence attached rather than argue.

Rubric for role R-17, version 2

left key role_id, right key role_id

role
Senior support engineer
problem_solving
Below the bar: guesses. Meets with gaps: reproduces but stops at the symptom. Meets: isolates the cause with evidence. Exceeds: also prevents recurrence.
communication
Below the bar: unclear. Meets with gaps: clear but incomplete. Meets: clear and complete. Exceeds: adapts to the audience.
ownership
Below the bar: defers. Meets with gaps: acts when asked. Meets: acts and follows through. Exceeds: changes the system.

Request 33 of 212

model jev-1.13.0, attempt 1, succeeded

problem_solvingordinal
2.70
  • Below the bar0.02
  • Meets with gaps0.28
  • Meets0.68
  • Exceeds0.02

confidence 0.66, provider reported

communicationordinal
3.08
  • Below the bar0.00
  • Meets with gaps0.08
  • Meets0.76
  • Exceeds0.16

confidence 0.74, provider reported

ownershipordinal
2.24
  • Below the bar0.06
  • Meets with gaps0.64
  • Meets0.30
  • Exceeds0.00

confidence 0.62, provider reported

evidence_turncategorical
A2
  • A20.71
  • A30.24
  • A40.03
  • A10.02

the answer the score rests on

answeredbinary
93.0%
  • true0.93

answered the questions asked

2.70 is at or above 2.0, kept

the scores stay with the row either way

field
answers.problem_solving.expected
comparison
gte 2.0
value
2.70
result
matched collection

Resume joined on candidate_id

no model call

  • 2023 to nowTier 2 Support Engineer, Ridgeline Logistics. Python tooling for log triage used by the whole team; on-call rotation.
  • 2020 to 2023Support Specialist, Fernwood Health. Zendesk macros, escalations to engineering.

Request 21 of 131

model jev-1.13.0, attempt 1, succeeded

python_productionbinary
88.0%
  • true0.88

resume: team-wide Python tooling

led_incidentbinary
34.0%
  • true0.34

handled incidents; leading one is not evidenced

One row the hiring manager reads with the quote open

every score links to the rubric version and the evidence turn

interview
I-0412, senior support engineer
problem solving
2.70, Meets (0.68)
communication
3.08, Meets (0.76)
ownership
2.24, Meets with gaps (0.64)
evidence
A2 (0.71): the VPN header trace answer
must-haves
Python in production yes 0.88; led an incident no 0.34
trace
7 nodes, 2 attempts, rubric v2 frozen in the run
1 of 7

What comes out

Hiring round, senior support engineerrun 5d20b7c4, 131 rows above the bar, 81 in the excluded collection
InterviewProblem solvingCommunicationOwnershipEvidencePython in prodLed incident
I-03883.413.123.05A40.960.91
I-04122.703.082.24A20.880.34
I-04012.662.312.90A30.520.78
I-03772.412.882.12A20.910.12
I-04202.082.022.55A50.230.61

Synthetic rows in the app's Outputs layout. Scores are expected positions on the four-level rubric; the distributions are one click away.

Details, if you want them

whyWhy this shape

Why this shape

  • One rubric, frozen. Every candidate scored against the same words.
  • Ordinal scores. 2.7 means between two levels, and you can see how sure.
  • The evidence turn. Read one answer, not the transcript.
  • Must-haves only above the bar. The other 81 keep their scores.

What it replaces

  • Six people remembering what was said.
  • A 3 that means a 4 to someone else.
  • Not the interviewer. People decide.
questionsThe 7 questions, verbatim
  • Score competencies problem_solvingordinal, score

    Using `rubric.problem_solving` as the scale, how does the candidate’s reasoning in `transcript` rate?

    Below the barMeets with gapsMeetsExceeds
  • Score competencies communicationordinal, score

    Using `rubric.communication` as the scale, how clearly does the candidate explain their thinking in `transcript`?

    Below the barMeets with gapsMeetsExceeds
  • Score competencies ownershipordinal, score

    Using `rubric.ownership` as the scale, how much personal responsibility for outcomes does the candidate show in `transcript`?

    Below the barMeets with gapsMeetsExceeds
  • Score competencies evidence_turncategorical, choice

    Which of the candidate’s answers in `transcript` best evidences the problem-solving score?

    A1A2A3A4A5A6
  • Score competencies answeredbinary, noul

    Did the candidate answer the questions actually asked in `transcript`, rather than a different question?

  • Must-haves python_productionbinary, noul

    Do `transcript` and `resume` together show the candidate has written Python that ran in production?

  • Must-haves led_incidentbinary, noul

    Do `transcript` and `resume` together show the candidate has led the response to a customer-facing incident?

buildHow each stage is built, as JSON
1 join Attach the rubric
A join on role_id puts the rubric text into each interview’s state. The rubric is a snapshot, so every candidate in the run is scored against the same words; editing it later creates a new version and does not touch this run.

stage definition

{
  "id": "with_rubric",
  "input": "source:interviews",
  "kind": "join",
  "left_key": [
    "role_id"
  ],
  "name": "Attach the rubric",
  "right": "source:rubrics",
  "right_key": [
    "role_id"
  ]
}
2 reducer Score competencies
One request per interview, five questions inside. The three competency scores are ordinals over the rubric’s four levels, so an expected score of 2.7 means “between meets-with-gaps and meets, leaning meets”, and the distribution shows how sure that is. evidence_turn picks the candidate’s answer that best supports the problem-solving score; answered catches the answer that dodged the question.

stage definition

{
  "id": "score",
  "input": "with_rubric",
  "kind": "evaluate",
  "model": "jev",
  "name": "Score competencies",
  "questions": {
    "answered": {
      "instructions": "Did the candidate answer the questions actually asked in `transcript`, rather than a different question?",
      "type": "noul"
    },
    "communication": {
      "criteria": [
        "Below the bar",
        "Meets with gaps",
        "Meets",
        "Exceeds"
      ],
      "instructions": "Using `rubric.communication` as the scale, how clearly does the candidate explain their thinking in `transcript`?",
      "type": "score"
    },
    "evidence_turn": {
      "criteria": {
        "A1": "The candidate's first answer",
        "A2": "The candidate's second answer",
        "A3": "The candidate's third answer",
        "A4": "The candidate's fourth answer",
        "A5": "The candidate's fifth answer",
        "A6": "The candidate's sixth answer"
      },
      "instructions": "Which of the candidate's answers in `transcript` best evidences the problem-solving score?",
      "type": "choice"
    },
    "ownership": {
      "criteria": [
        "Below the bar",
        "Meets with gaps",
        "Meets",
        "Exceeds"
      ],
      "instructions": "Using `rubric.ownership` as the scale, how much personal responsibility for outcomes does the candidate show in `transcript`?",
      "type": "score"
    },
    "problem_solving": {
      "criteria": [
        "Below the bar",
        "Meets with gaps",
        "Meets",
        "Exceeds"
      ],
      "instructions": "Using `rubric.problem_solving` as the scale, how does the candidate's reasoning in `transcript` rate?",
      "type": "score"
    }
  },
  "state_path": []
}
3 filter Keep problem solving ≥ 2.0
A comparison on the expected problem-solving score. It gates the second pass; it does not reject anyone. Candidates below the line keep their scores and their evidence in the excluded collection, where the hiring manager can still read them.

stage definition

{
  "comparison": "gte",
  "field": [
    "answers",
    "problem_solving",
    "expected"
  ],
  "id": "above_bar",
  "input": "score",
  "kind": "filter",
  "name": "Keep problem solving ≥ 2.0",
  "value": 2
}
4 join Attach the resume
The must-haves need the resume as well as the transcript, so the resume snapshot is joined on candidate_id only for the candidates above the bar.

stage definition

{
  "id": "with_resume",
  "input": "above_bar",
  "kind": "join",
  "left_key": [
    "candidate_id"
  ],
  "name": "Attach the resume",
  "right": "source:resumes",
  "right_key": [
    "candidate_id"
  ]
}
5 reducer Must-haves
Two binary questions with the transcript and the resume in the state. Each must-have is its own proposition, so a candidate can be strong on one and weak on the other, and you can see which.

stage definition

{
  "id": "must_haves",
  "input": "with_resume",
  "kind": "evaluate",
  "model": "jev",
  "name": "Must-haves",
  "questions": {
    "led_incident": {
      "instructions": "Do `transcript` and `resume` together show the candidate has led the response to a customer-facing incident?",
      "type": "noul"
    },
    "python_production": {
      "instructions": "Do `transcript` and `resume` together show the candidate has written Python that ran in production?",
      "type": "noul"
    }
  },
  "state_path": []
}
6 definition The whole workflow, as saved

Saving publishes an immutable version. A run freezes this definition, the snapshot IDs and each reducer’s model profile, so the trace can always show exactly what ran. Edits create a new version; old runs keep theirs.

jevreduce.local.v1

{
  "name": "Interview transcripts scored against a rubric",
  "schema_version": "jevreduce.local.v1",
  "sources": {
    "interviews": "snap_fireflies_interviews_v4",
    "resumes": "snap_resumes_v4",
    "rubrics": "snap_rubrics_v2"
  },
  "stages": [
    {
      "id": "with_rubric",
      "input": "source:interviews",
      "kind": "join",
      "left_key": [
        "role_id"
      ],
      "name": "Attach the rubric",
      "right": "source:rubrics",
      "right_key": [
        "role_id"
      ]
    },
    {
      "id": "score",
      "input": "with_rubric",
      "kind": "evaluate",
      "model": "jev",
      "name": "Score competencies",
      "questions": {
        "answered": {
          "instructions": "Did the candidate answer the questions actually asked in `transcript`, rather than a different question?",
          "type": "noul"
        },
        "communication": {
          "criteria": [
            "Below the bar",
            "Meets with gaps",
            "Meets",
            "Exceeds"
          ],
          "instructions": "Using `rubric.communication` as the scale, how clearly does the candidate explain their thinking in `transcript`?",
          "type": "score"
        },
        "evidence_turn": {
          "criteria": {
            "A1": "The candidate's first answer",
            "A2": "The candidate's second answer",
            "A3": "The candidate's third answer",
            "A4": "The candidate's fourth answer",
            "A5": "The candidate's fifth answer",
            "A6": "The candidate's sixth answer"
          },
          "instructions": "Which of the candidate's answers in `transcript` best evidences the problem-solving score?",
          "type": "choice"
        },
        "ownership": {
          "criteria": [
            "Below the bar",
            "Meets with gaps",
            "Meets",
            "Exceeds"
          ],
          "instructions": "Using `rubric.ownership` as the scale, how much personal responsibility for outcomes does the candidate show in `transcript`?",
          "type": "score"
        },
        "problem_solving": {
          "criteria": [
            "Below the bar",
            "Meets with gaps",
            "Meets",
            "Exceeds"
          ],
          "instructions": "Using `rubric.problem_solving` as the scale, how does the candidate's reasoning in `transcript` rate?",
          "type": "score"
        }
      },
      "state_path": []
    },
    {
      "comparison": "gte",
      "field": [
        "answers",
        "problem_solving",
        "expected"
      ],
      "id": "above_bar",
      "input": "score",
      "kind": "filter",
      "name": "Keep problem solving ≥ 2.0",
      "value": 2
    },
    {
      "id": "with_resume",
      "input": "above_bar",
      "kind": "join",
      "left_key": [
        "candidate_id"
      ],
      "name": "Attach the resume",
      "right": "source:resumes",
      "right_key": [
        "candidate_id"
      ]
    },
    {
      "id": "must_haves",
      "input": "with_resume",
      "kind": "evaluate",
      "model": "jev",
      "name": "Must-haves",
      "questions": {
        "led_incident": {
          "instructions": "Do `transcript` and `resume` together show the candidate has led the response to a customer-facing incident?",
          "type": "noul"
        },
        "python_production": {
          "instructions": "Do `transcript` and `resume` together show the candidate has written Python that ran in production?",
          "type": "noul"
        }
      },
      "state_path": []
    }
  ]
}
limitsWhat it will not do
  • A score is a position on your rubric, not a hiring decision.
  • Rubric versions are frozen per run. Replay to rescore.
  • Transcription errors are scored as said.
  • Measure consistency on a labelled sample first.
  • No written feedback. Jev returns scores and a turn.
relatedRelated pages

Bring your next question.

Join the waitlist to explore your own data with Deduced.

Join the waitlistExplore examples