From the SREGym team

Blog

Experiments, findings, and field notes on building reliable SRE agents.

Latest writing

Research

Jev-as-a-Judge for SRE

We tested how closely Jev could match an LLM judge when grading SRE diagnosis reports.

Yukun He · Saad Mohammad Rafid Pial · Jackson Clark · Yiming Su · Tianyin Xu

Read article
Jev grades each report. Reports with uncertain scores go to the LLM judge, while Jev decides the rest.
Research

Decision Models Lead, LLMs Review: Fast and Accurate SRE Diagnosis

Decision models lead the investigation; LLMs review the conclusions and write the report.

Yukun He · Saad Mohammad Rafid Pial · Yiming Su · Jackson Clark · Tianyin Xu

Read article
The decision model leads an investigation loop with read-only tool calls. When the investigation ends, the LLM reviews the root cause, the decision model selects supporting evidence, and the LLM writes the diagnosis.
Community

SREGym at Frontier Data Summit: Community, Conversations, and What’s Next

Conversations about AI for SRE, our growing community, and a look ahead to SREGym 2.0.

Jackson Clark · Tianyin Xu

Read article
Frontier Data Summit’s featured-poster card for SREGym, presented by Jackson Clark on October 8, 2026, in San Francisco.
Research

Jev-Driven SRE Diagnosis: What Worked and What Failed

A Jev-driven pipeline chooses where to investigate and which cluster evidence supports a diagnosis.

Yiming Su · Saad Mohammad Rafid Pial · Jackson Clark · Tianyin Xu

Read article
Jev selects a component from a cluster snapshot, then reviews collected evidence. A supported cause produces a diagnosis result; otherwise, investigation continues with another component.
Research

Turning Failure Postmortems into SRE-Benchmark Problems

Students from UIUC++ SRSE created 30+ problems in SREGym from postmortem reports of noteworthy system failures.

Tianyin Xu · Saad Mohammad Rafid Pial · Munim Thahmid · Mohammad Elsharqawy · Mahdiya Rahman Sukanya · Sai Hari Krishnan · Sumaiya Islam · Tanzim Hossain Romel · Abdallah Adel · Varuni HK · Jackson Clark

Read article
From postmortems to SREGym problems: reports of real incidents and their causes become reproducible faults for testing SRE agents.
Research

Can Jev Make SRE Agents More Reliable?

We tested TypeSafe AI's Jev as a decision layer for SRE agents.

Jackson Clark · Saad Mohammad Rafid Pial · Yiming Su · Tianyin Xu

Read article
The agent investigates and repairs; Jev ranks and reviews. Pass rate: 40% without Jev, 48% with Jev.