Mechanistic Interpretability#

Slides from the lecture covering mechanistic interpretability methods for LMs (logit lens, residual stream, activation patching, circuit analysis, sparse autoencoders) can be found here.

Additional materials#

If you want to dig a bit deeper, here are (optional!) supplementary readings. More papers discussed in the lecture are provided in the slides.

• The logit lens blogpost

• Merullo et al. (2024) Language Models Implement Simple Word2Vec-style Vector Arithmetic

• Elhage et al. (2021) A Mathematical Framework for Transformer Circuits

• Meng et al. (2022) Locating and Editing Factual Associations in GPT, aka ROME paper mentioned in the lecture

• Vig et al. (2020) Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias

• Heimersheim & Nanda (2024) How to use and interpret activation patching

• Wang et al. (2023) Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

• Causal Scrubbing: a method for rigorously testing interpretability hypotheses

• Merullo et al. (2024) Circuit component reuse across tasks in transformer language models

• Yu et al. (2023) Characterizing Mechanisms for Factual Recall in Language Models