Topic

Interpretable AI

Models whose reasoning people can read and check: Bayesian models with meaningful parameters, grammars and programs that encode domain knowledge, and methods for probing what neural networks have learned.

5 episodes

About Interpretable AI on Interpretable

Interpretable AI is the subject the show is named for. In this podcast it means models whose reasoning a person can read and check, and the five episodes in this topic show five different ways to get there. Episode 002 uses a Bayesian model whose parameters have meaning: PyR0 attributes the growth of SARS-CoV-2 lineages to individual mutations, with uncertainty. Episode 003 uses a grammar: a set of production rules learned from data that a chemist can read one by one. Episode 004 offers a model of human reasoning, value-guided construal, precise enough to make predictions and test them in experiments.

Episode 005 turns the question around and asks how to know what an AI model of code has learned, by lining up its internal representations with fMRI recordings of programmers. Episode 000, from the host, Omar Costilla Reyes, explains the idea behind the name: when part of a model is written as a program, scientists can read, test, and correct it. Each episode is anchored to a published paper, listed below with its venue, year, and link, so the claims on this page can be checked at the source.

Episodes

References

  1. Shashank Srikant, Benjamin Lipkin, Anna Ivanova, Evelina Fedorenko, Una-May O'Reilly (2022). Convergent Representations of Computer Programs in Human and Artificial Neural Networks(opens in a new tab). NeurIPS 2022, Advances in Neural Information Processing Systems 35: 18834–18849. Discussed in Your Brain on Code: What Brains and AI Models Both Learn.
  2. Mark K. Ho, David Abel, Carlos G. Correa, Michael L. Littman, Jonathan D. Cohen, Thomas L. Griffiths (2022). People construct simplified mental representations to plan(opens in a new tab). Nature, 606: 129–136. https://doi.org/10.1038/s41586-022-04743-9. Discussed in The Simplified Mental Maps We Plan With.
  3. Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, Wojciech Matusik (2022). Data-Efficient Graph Grammar Learning for Molecular Generation(opens in a new tab). ICLR 2022 (oral). Discussed in Designing Molecules With Rules You Can Read.
  4. Fritz Obermeyer, Martin Jankowiak, Nikolaos Barkas, Stephen F. Schaffner, Jesse D. Pyle, Leonid Yurkovetskiy, Matteo Bosso, et al. (2022). Analysis of 6.4 million SARS-CoV-2 genomes identifies mutations associated with fitness(opens in a new tab). Science, 376 (6599): 1327–1332. https://doi.org/10.1126/science.abm1208. Discussed in Which Mutations Make a Virus Spread? Reading 6.4 Million Genomes.
  5. Jennifer J. Sun, Megan Tjandrasuwita, Atharva Sehgal, Armando Solar-Lezama, Swarat Chaudhuri, Yisong Yue, Omar Costilla-Reyes (2022). Neurosymbolic Programming for Science(opens in a new tab). NeurIPS 2022 Workshop on AI for Science. Discussed in Why Interpretable? Code as the Language of Science.