Jason Poulos

Research Scientist · AI Evaluations & Agents · Machine Learning

poulos [AT] berkeley.edu

CV · Resume

Bio

I am an independent researcher whose work lies at the intersection of machine learning, causal inference, and policy. I hold a Ph.D. in Political Science with a Designated Emphasis in Computational Science and Engineering from UC Berkeley and have held postdoctoral appointments in machine learning and data science.

My current research evaluates frontier AI models by designing evaluation frameworks that measure model capabilities and translate safety concerns into measurable tests. I have published studies on causal inference with multi-valued treatments, Bayesian frameworks for adversarial machine learning, and deep learning-based missing-data imputation and counterfactual prediction.

Highlights: I co-authored Terminal-Bench (ICLR 2026), contributing eight benchmark tasks, including a genome-assembly task featured in the task spotlight, and Humanity's Last Exam (Nature, 2026). Co-authored work on Bayesian adversarial machine learning received the Premio SEIO–Fundación BBVA 2026.

News

Jul 2026"Adversarial Machine Learning: Bayesian Perspectives" (JASA, 2023), with David Ríos Insua, Roi Naveiro, and Víctor Gallego, received the Premio SEIO–Fundación BBVA 2026 for Best Contribution from Statistics and Operations Research to Data Science.
Jan 2026Humanity's Last Exam, a benchmark of expert-level academic questions to assess AI capabilities, is published in Nature.
Jan 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces is accepted at ICLR 2026.

Publications

Most recent publications on Google Scholar.

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Y. Shin, Thomas Walshe, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, E. K. Buchanan, Jenia Jitsev, Marianna Nezhurina, Di Lu, Orfeas M. Mastromichalakis, ..., Ludwig Schmidt.

International Conference on Learning Representations, 2026

Paper Preprint Website Blog

A benchmark of expert-level academic questions to assess AI capabilities

Center for AI Safety, Scale AI & HLE Contributors Consortium

Nature, 2026

Paper Preprint Website

State-Building through Public Land Disposal? An Application of Matrix Completion for Counterfactual Prediction

Jason Poulos

Statistics and Public Policy, 2024

Paper Preprint Code

Targeted Learning in Observational Studies with Multi-Level Treatments: An Evaluation of Antipsychotic Drug Treatment Safety

Jason Poulos, Marcela Horvitz-Lennon, Katya Zelevinsky, Thomas Huijskens, Pooja Tyagi, Jiaju Yan, Jordi Diaz, Tudor Cristea-Platon, Sharon-Lise Normand

Statistics in Medicine, 2024

Paper Preprint Code

Adversarial Machine Learning: Bayesian Perspectives

David Rios Insua, Roi Naveiro, Víctor Gallego, Jason Poulos

Journal of the American Statistical Association, 2023

Awarded the Premio SEIO-Fundación BBVA (2026) for Best Contribution from Statistics and Operations Research to Data Science.

Paper Preprint Code

Are Deep Learning Models Superior for Missing Data Imputation in Surveys? Evidence from an Empirical Comparison

Zhenhua Wang, Olanrewaju Akande, Jason Poulos, Fan Li

Survey Methodology, 2022

Paper Preprint Code

RNN-Based Counterfactual Prediction, with an Application to Homestead Policy and Public Schooling

Jason Poulos, Shuxi Zeng

Journal of the Royal Statistical Society (C), 2021

Paper Preprint Code

Character-Based Handwritten Text Transcription with Attention Networks

Jason Poulos, Rafael Valle

Neural Computing & Applications, 2021

Paper Preprint Code

Estimating Population Average Treatment Effects from Experiments with Noncompliance

Kellie Ottoboni, Jason Poulos

Journal of Causal Inference, 2020

Paper Preprint Code

Missing Data Imputation for Supervised Learning

Jason Poulos, Rafael Valle

Applied Artificial Intelligence, 2018

Paper Preprint Supplementary Material Code

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Y. Shin, Thomas Walshe, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, E. K. Buchanan, Jenia Jitsev, Marianna Nezhurina, Di Lu, Orfeas M. Mastromichalakis, ..., Ludwig Schmidt.

International Conference on Learning Representations, 2026

Paper Preprint Website Blog

A benchmark of expert-level academic questions to assess AI capabilities

Center for AI Safety, Scale AI & HLE Contributors Consortium

Nature, 2026

Paper Preprint Website

Revisiting Diabetes Risk of Olanzapine versus Aripiprazole for Serious Mental Illness Care

Denis Agniel, Sharon-Lise Normand, John Newcomer, Katya Zelevinsky, Jason Poulos, Jeannette Tsuei, Marcela Horvitz-Lennon

BJPsych Open, 2024

Paper

State-Building through Public Land Disposal? An Application of Matrix Completion for Counterfactual Prediction

Jason Poulos

Statistics and Public Policy, 2024

Paper Preprint Code

Targeted Learning in Observational Studies with Multi-Level Treatments: An Evaluation of Antipsychotic Drug Treatment Safety

Jason Poulos, Marcela Horvitz-Lennon, Katya Zelevinsky, Thomas Huijskens, Pooja Tyagi, Jiaju Yan, Jordi Diaz, Tudor Cristea-Platon, Sharon-Lise Normand

Statistics in Medicine, 2024

Paper Preprint Code

Antipsychotics and the Risk of Diabetes and Death among Adults with Serious Mental Illnesses

Jason Poulos, Sharon-Lise Normand, Katya Zelevinsky, John Newcomer, Denis Agniel, Haley Abing, Marcela Horvitz-Lennon

Psychological Medicine, 2023

Paper Code

Adversarial Machine Learning: Bayesian Perspectives

David Rios Insua, Roi Naveiro, Víctor Gallego, Jason Poulos

Journal of the American Statistical Association, 2023

Awarded the Premio SEIO-Fundación BBVA (2026) for Best Contribution from Statistics and Operations Research to Data Science.

Paper Preprint Code

Gender Gaps in Frontier Entrepreneurship? Evidence from 1901 Oklahoma Land Lottery Winners

Jason Poulos

Journal of Historical Political Economy, 2023

Paper Preprint Code

Are Deep Learning Models Superior for Missing Data Imputation in Surveys? Evidence from an Empirical Comparison

Zhenhua Wang, Olanrewaju Akande, Jason Poulos, Fan Li

Survey Methodology, 2022

Paper Preprint Code

RNN-Based Counterfactual Prediction, with an Application to Homestead Policy and Public Schooling

Jason Poulos, Shuxi Zeng

Journal of the Royal Statistical Society (C), 2021

Paper Preprint Code

Character-Based Handwritten Text Transcription with Attention Networks

Jason Poulos, Rafael Valle

Neural Computing & Applications, 2021

Paper Preprint Code

Amnesty Policy and Elite Persistence in the Postbellum South: Evidence from a Regression Discontinuity Design

Jason Poulos

Journal of Historical Political Economy, 2021

Paper Preprint Code

Estimating Population Average Treatment Effects from Experiments with Noncompliance

Kellie Ottoboni, Jason Poulos

Journal of Causal Inference, 2020

Paper Preprint Code

Land Lotteries, Long-Term Wealth, and Political Selection

Jason Poulos

Public Choice, 2019

Paper Code

Missing Data Imputation for Supervised Learning

Jason Poulos, Rafael Valle

Applied Artificial Intelligence, 2018

Paper Preprint Supplementary Material Code

Projects

Transformer Models for Diabetes Treatment Failure 2023–2024

Interpretable transformer neural networks trained on over 10 million clinical events from Mass General Brigham electronic health records, combining static and temporal patient features to predict diabetes treatment failure (HbA1c ≥ 7% within 12 months). Compares tabular transformers with a GPT-style autoregressive transformer on patient event streams.

Code Slides

Doubly Robust Causal Inference with Multi-Valued Treatment Regimes and RNNs

Extends the longitudinal targeted minimum loss-based estimator (LTMLE) to multi-valued treatment regimes by modeling treatment assignment with a multinomial model, and integrates recurrent neural networks as an alternative to sequential g-computation. In simulations, the multinomial model improves efficiency over a series of binomial regressions, and LTMLE with RNNs reduces average bias relative to ensemble machine learning. Motivated by the safety of antipsychotic treatment regimes for adults with serious mental illnesses.

Code

Vitæ

Full CV: PDF. Industry resume: PDF.

Jason Poulos
poulos [AT] berkeley.edu