Postdoctoral Researcher
AI2 & University of Washington

Yanai Elazar

I am a postdoctoral researcher (and Young Investigator) on the AllenNLP team at AI2, and the University of Washington, and a Rothschild Fellow. Before joining AI2 and UW, I did my PhD (2022) in Computer Science in the NLP lab at Bar-Ilan University, where I was lucky to be a Google PhD Fellow.

I am broadly interested in Natural Language Processing (NLP) and Machine Learning (ML). In particular, I am interested in understanding what makes models work, how they work, and why. These days I am focusing on the data such models are trained on. Moreover, I'm interested in Causal Inference (CI) and integrating ideas from CI into NLP.

I'm happy to talk about research in general, and my own work in particular. If you have any questions about one of my papers, or my overall research, feel free to reach out!


Co-Organizing the 1st Data Contamination workshop at ACL 2024
August 2024
Attending a Dagstuhl Seminar on 'Trustworthiness and Responsibility in AI - Causality, Learning, and Verification'
March 2024
Giving talks at LMU Munich and Edinburgh NLP. Come say hi!
March 2024
Giving talks at USC, UCLA, and UCSB. Come say hi!
Febrauary 2024
IAAI Best Thesis Award Runner Up
December 2023
December 2023
Attending SoCal NLP. Come say hi!
November 2023
Invited talk at EPFL
November 2023
Wrote a 502 page about my academic failures
October 2023
Our work (turned into WIMBD) was featured in The Washington Post
April 2023
Invited talk at Georgetown University
April 2023
A new blog post about the behind the scenes of interviews
February 2023



The Bias Amplification Paradox in Text-to-Image Generation
Preethi Seshadri, Sameer Singh, Yanai Elazar
NAACL 2024
paper long code

A Survey on Data Selection for Language Models
Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, Colin Raffel, Shiyu Chang, Tatsunori Hashimoto, William Yang Wang
paper resource

Calibrating Large Language Models with Sample Consistency
Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar, Niket Tandon, Marianna Apidianaki, Mrinmaya Sachan, Chris Callison-Burch

OLMo: Accelerating the Science of Language Models
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi
paper code resource models
Press: TechCrunch Axios Forbes GeekWire SD Times VentureBeat Fast Company

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
paper code resource

What's In My Big Data?
Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, Hanna Hajishirzi, Noah A. Smith, Jesse Dodge
ICLR 2024
paper code demo
Press: Marktechpost

Estimating the Causal Effect of Early ArXiving on Paper Acceptance
*Yanai Elazar, *Jiayao Zhang, *David Wadden, Bo Zhang, Noah A. Smith
CLeaR 2024
paper code


Paloma: A Benchmark for Evaluating Language Model Fit
Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hannaneh Hajishirzi, Noah A. Smith, Kyle Richardson, Jesse Dodge
paper code resource

Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals
Yanai Elazar, Bhargavi Paranjape, Hao Peng, Sarah Wiegreffe, Khyathi Raghavi, Vivek Srikumar, Sameer Singh, Noah A. Smith

A taxonomy and review of generalization research in NLP
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Rita Frieske, Ryan Cotterell, Zhijing Jin
Nature Machine Intelligence 2023
paper journal project-page

Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation
Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, Yanai Elazar
Findings of ACL 2023
paper long code poster

CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm
Hongming Zhang, Yintong Huo, Yanai Elazar, Yangqiu Song, Yoav Goldberg, Dan Roth
Findings of EACL 2023
paper long code


Lexical Generalization Improves with Larger Models and Longer Training
Elron Bandel, Yoav Goldberg, Yanai Elazar
Findings of EMNLP 2022
paper short code poster

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg
paper models

Text-based NP Enrichment
*Yanai Elazar, *Victoria Basmov, Yoav Goldberg, Reut Tsarfaty
TACL 2022
paper journal code resource demo project-page slides video


Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes
Ofer Sabo, Yanai Elazar, Yoav Goldberg, Ido Dagan
TACL 2021
paper journal code video

Back to Square One: Bias Detection, Training and Commonsense Disentanglement in the Winograd Schema
Yanai Elazar, Hongming Zhang, Yoav Goldberg, Dan Roth
EMNLP 2021
paper long code slides video

Contrastive Explanations for Model Interpretability
Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel, Yanai Elazar, Yejin Choi, Yoav Goldberg
EMNLP 2021
paper long code video

Measuring and Improving Consistency in Pretrained Language Models
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
TACL 2021
paper journal code resource slides video

First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
Benjamin Muller, Yanai Elazar, Benoît Sagot and Djamé Seddah
EACL 2021
paper short code

*Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
Yanai Elazar, Shauli Ravfogel, Alon Jacovi, Yoav Goldberg
TACL 2021
(*) previous version that appeared on arxiv was named: "When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions", which we changed to the current title to better reflect our contributions.
paper journal code slides video


It’s not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT
Hila Gonen, Shauli Ravfogel, Yanai Elazar, Yoav Goldberg
Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, at EMNLP 2020
paper long code poster

The Extraordinary Failure of Complement Coercion Crowdsourcing
Yanai Elazar, Victoria Basmov, Shauli Ravfogel, Yoav Goldberg, Reut Tsarfaty
Workshop on Insights from Negative Results in NLP, EMNLP 2020
paper short slides video

Do Language Embeddings Capture Scales?
Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, Dan Roth
Findings of EMNLP 2020
paper long code

Unsupervised Distillation of Syntactic Information from Contextualized Word Representations
*Shauli Ravfogel, *Yanai Elazar, Jacob Goldberger, Yoav Goldberg
Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, at EMNLP 2020
paper long code slides

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, Yoav Goldberg
ACL 2020
paper long code video

Evaluating Models' Local Decision Boundaries via Contrast Sets
Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
Findings of EMNLP 2020
paper long resource

oLMpics -- On what Language Model Pre-training Captures
Alon Talmor, Yanai Elazar, Yoav Goldberg, Jonathan Berant
TACL 2020 (presented at EMNLP 2020)
paper journal code video


Adversarial Removal of Demographic Attributes Revisited
Maria Barrett, Yova Kementchedjhieva, Yanai Elazar, Desmond Elliott, Anders Søgaard
EMNLP 2019
paper short

How Large Are Lions? Inducing Distributions over Quantitative Attributes
Yanai Elazar, Abhijit Mahabal, Deepak Ramachandran, Tania Bedrax-Weiss, Dan Roth
ACL 2019
paper long code resource demo slides

Where's My Head? Definition, Dataset and Models for Numeric Fused-Heads Identification and Resolution
Yanai Elazar, Yoav Goldberg
TACL 2019 (presented at EMNLP 2019)
paper journal code resource demo slides video

Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations
Yehezkel S. Resheff, Yanai Elazar, Moni Shahar, Oren Sar Shalom
paper long


Adversarial Removal of Demographic Attributes from Text Data
Yanai Elazar, Yoav Goldberg
EMNLP 2018
paper long code slides video


From Interviewee To Interviewer

Behind the scences of the interviewing process

Attending ACL 2020

My strategy for attending my first virtual conference.

Remote Servers

How to setup your environment to seemingly work with remote servers.