Peter Shaw

Publications

AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
Xing Han Lù, Amirhossein Kazemnejad, Nicholas Meade, Arkil Patel, Dongchan Shin, Alejandra Zambrano, Karolina Stańczak, Peter Shaw, Christopher J. Pal, Siva Reddy
CoLM 2025 · Website

ALTA: Compiler-Based Analysis of Transformers
Peter Shaw, James Cohan, Jacob Eisenstein, Kenton Lee, Jonathan Berant, Kristina Toutanova
TMLR 2025 · Code

ProtEx: A Retrieval-Augmented Approach for Protein Function Prediction
Peter Shaw, Bhaskar Gurram, David Belanger, Andreea Gane, Maxwell L Bileschi, Lucy J Colwell, Kristina Toutanova, Ankur P Parikh
MLCB 2025 (Oral) · Code

Robust Preference Optimization through Reward Model Distillation
Adam Fisch, Jacob Eisenstein, Vicky Zayats, Alekh Agarwal, Ahmad Beirami, Chirag Nagpal, Peter Shaw, Jonathan Berant
TMLR 2025

BAGEL: Bootstrapping Agents by Guiding Exploration with Language
Shikhar Murty, Christopher Manning, Peter Shaw, Mandar Joshi, Kenton Lee
ICML 2024

Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D'Amour, DJ Dvijotham, Adam Fisch, Katherine Heller, Stephen Pfohl, Deepak Ramachandran, Peter Shaw, Jonathan Berant
CoLM 2024

From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces
Peter Shaw^*, Mandar Joshi^*, James Cohan, Jonathan Berant, Panupong Pasupat, Hexiang Hu, Urvashi Khandelwal, Kenton Lee, Kristina Toutanova
NeurIPS 2023 (Spotlight) · Code

QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations
Chaitanya Malaviya, Peter Shaw, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
ACL 2023 · Outstanding Paper Award · Code · Dataset

Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Kenton Lee^*, Mandar Joshi^*, Iulia Turc, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, Kristina Toutanova
ICML 2023 · Code

Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing
Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, Fei Sha
COLING 2022

Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing
Linlu Qiu, Peter Shaw, Panupong Pasupat, Tianze Shi, Jonathan Herzig, Emily Pitler, Fei Sha, Kristina Toutanova
EMNLP 2022

Improving Compositional Generalization with Latent Structure and Data Augmentation
Linlu Qiu^*, Peter Shaw^*, Panupong Pasupat, Paweł Krzysztof Nowak, Tal Linzen, Fei Sha, Kristina Toutanova
NAACL 2022 · Code

Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks
Wang Zhu, Peter Shaw, Tal Linzen, Fei Sha
arXiv 2021

Visually Grounded Concept Composition
Bowen Zhang, Hexiang Hu, Linlu Qiu, Peter Shaw, Fei Sha
EMNLP 2021 (Findings)

Systematic Generalization on gSCAN: What is Nearly Solved and What is Next?
Linlu Qiu, Hexiang Hu, Bowen Zhang, Peter Shaw, Fei Sha
EMNLP 2021 · Code

Graph-Based Decoding for Task Oriented Semantic Parsing
Jeremy Cole, Nanjiang Jiang, Panupong Pasupat, Luheng He, Peter Shaw
EMNLP 2021 (Findings)

Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations
Jonathan Herzig, Peter Shaw, Ming-Wei Chang, Kelvin Guu, Panupong Pasupat, Yuan Zhang
arXiv 2021 · Code

Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?
Peter Shaw, Ming-Wei Chang, Panupong Pasupat, Kristina Toutanova
ACL 2021 · Code

Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing
Alane Suhr, Ming-Wei Chang, Peter Shaw, Kenton Lee
ACL 2020 · Code

Answering Conversational Questions on Structured Data without Logical Forms
Thomas Mueller, Francesco Piccinno, Peter Shaw, Massimo Nicosia, Yasemin Altun
EMNLP 2019

Generating Logical Forms from Graph Representations of Text and Entities
Peter Shaw, Philip Massey, Angelica Chen, Francesco Piccinno, Yasemin Altun
ACL 2019

Self-Attention with Relative Position Representations
Peter Shaw, Jakob Uszkoreit, Ashish Vaswani
NAACL 2018 · Code

^* indicates equal contribution.

---

About

Broadly, my research focuses on machine learning. I have worked on evaluating and improving generalization in neural models, developing retrieval-augmented systems, and combining neural models with structured components and environments.

Previously, I worked on query understanding for Google Search, where I led the development and deployment of neural models for semantic parsing. Before joining Google in 2015, I was the technical lead and manager of the embedded software team at MakerBot, developing 3D printers and scanners. I graduated summa cum laude from Villanova University with a combined Bachelor's and Master's in Electrical Engineering.