Michael Khalfin

Michael Khalfin

Ph.D. student, Industrial and Systems Engineering · UW–Madison

I build loops where AI agents draft, cheap checks verify, and experts answer only the questions that change the answer.

Email GitHub LinkedIn CV
The expert panel and the LLM floating above the floor of consistent opinions
Ph.D. researchWhen can a cheap LLM judge stand in for experts?
Sandia National LabsVerifying AI-written code
A tablet page: my hand-drawn sketch beside Claude's typeset reading of it
Personal, used dailyClaude ↔ reMarkable loop
First page of the data collection prioritization working paper
Working paperWhich costly data are worth collecting?
Weight patterns of three hidden neurons
Interactive demoInside a tiny neural network
Research, RiceChoosing on the spot
Chess at Széchenyi Bath, Budapest
About meChess in a Budapest bath, and other stories

I like hearing from people building AI systems that work alongside experts.

© Michael Khalfin

Research

2026–present · Ph.D. research

When can a cheap AI judge stand in for experts?

Expert feedback is scarce; an LLM judge is cheap but biased. I combine a small weekly stream of expert answers with many LLM answers to keep a deployed model accurate.

The floor of consistent opinions, with the expert panel and the LLM floating above it
On and off the floor. The floor holds every opinion a perfectly consistent judge could have. The expert panel (blue) and the LLM (amber) float above it. Drop each to the floor and its inconsistency is gone; what's left along the floor is the LLM's bias.
A tilted sheet above the floor; its height over a point is that judge's verdict
A verdict is a height: one comparison is one tilted sheet over the floor.
One week: many cheap LLM answers and one expensive panel answer update one estimate
One week: many cheap answers, one expensive one, one estimate.
Two cycles of preferences among replies A, B, C: one sums to zero, one does not
Consistent preferences sum to zero around a loop. A real panel's don't.
Share of the distance covered against number of LLM answers, flattening quickly
Past the knee, more LLM answers buy almost nothing.
A weighting is a spot on the floor One comparison draws one set of contour lines Many comparisons define a distance A second space: lists of verdicts The panel sits above the plane; drop it and read back Back onto the triangle: the target What the data tells us: an uncertainty ellipse The cheap judge sits a gap away from the panel
1 / 8 A weighting is a spot on the floor · the whole picture, built one page at a time. Claude sends me these on my tablet.
First page of the working paper

2025–2026

Which costly data are worth collecting?

Pick the measurements that pin down the numbers a decision rests on. No prior, no noise model, and it runs on off-the-shelf solvers. Working with Claude, I scaled it to a 6.6-million-variable school-outbreak model.

Working paper (Sections 1–2), PDF

One formulation, eleven problems.

Public health
School outbreakWho was infected, and when? Test a person for a time window.
Water contaminationWhere and when did it enter? Sample at finer tolerances.
Biology
Food websWho eats whom? Buy gut-content and diet studies.
Metabolic fluxFlows inside a cell? Buy isotope tracers.
Haplotype phaseWhich copy carries each allele? Assays comparing two sites.
Engineering
Traffic tomographyFlows by vehicle class? Sensors of increasing resolution.
Model-based diagnosisWhich component failed? Choose test probes.
Unmetered solarLoad vs. solar split? When and where to meter.
Economics & supply chains
Interbank contagionWho owes whom? Legal data requests.
Suppressed travel demandDemand at service never offered? Surveys, traces, pilots.
Supply-chain provenanceWho shipped each part? Declarations, audits, lab tests.
take take best seen while watching watch then take what beats it arrivals, one at a time →

Rice, 2024–2025 · with Sebastian Perez-Salazar

Choosing on the spot

Options arrive one at a time, and each must be taken or passed for good (the k-secretary problem). I derived closed-form guarantees for simple threshold rules, 1.3x better than the previous best.

Projects

AI agent writes the transpiler 51 checks must match reference code each failure becomes a new constraint MODEL BUILD TIME, 42M VARIABLES standard 117 min transpiled under 1 min

Sandia National Laboratories, 2025–present

Verifying AI-written code

I directed AI agents to write two Pyomo-to-Gurobi transpilers, provably correct for the target class of models. The output had to match a reference on 30 reduced instances and 21 small models that each test one modeling construct.

A page from my research notebook on the reMarkable

Pages from my research notebook.

Personal · used daily

Claude ↔ reMarkable loop

I wanted to work through papers on paper, rather than chatting with a screen. So I built a loop: Claude sends prompts to my reMarkable, reads my handwritten notes, and decides what to ask next.

Input weight patterns of the three first-layer hidden neurons

What each of the three hidden neurons looks for in a 7×7 digit (blue: pushes up, red: pushes down).

Share of test digits certified robust, by digit and perturbation size

Share of 600 test digits certified robust at each perturbation size.

Rice, 2025 · with Charlie Cruz and Burton Alexander · COMP 414

Inside a tiny neural network

Certified a small MNIST classifier's robustness with an LP relaxation, and found hidden neurons you can read.

Open the demo · desktop Sandia talk slides, PDF

Interactive demo

Which decisions does an optimal algorithm need?

The best possible rule for the k-secretary problem comes out of a linear program as a table of decisions.

Open the demo · desktop

Publications

CV

Open CV (PDF)

About me

Peony farm in Homer, Alaska

Harvesting flowers in Alaska

Several weeks on a small peony farm in Homer, AK: picking, cleaning, and packing flowers. Rustic, peaceful, restful.

Chess at Széchenyi Bath, Budapest

Chess at Széchenyi Bath

"Chess tourists" from Denmark and Moldova in a Budapest thermal bath. A friend, three Kazakh students, and I took them on for hours.

Cycling in Hungary

Biking in Hungary

I thought we were taking the train to Eger. We cycled about 30 miles instead, and drank a whole liter of grape juice on the way.

Bavarian breakfast in Regensburg

Bavarian breakfast

A friend's 21st birthday in Regensburg. After a long night out: sausage, pretzels, and coffee.

Polar plunge near Stockholm

Polar plunge

33°F water in a village near Stockholm, right after a dry sauna. A test of discipline that feels great.

Books I love