Published in ICML 2024 Workshop on LLMs and Cognition, 2024
We investigate the dynamics of the predictive distribution across the layers of LLMs instructed to lie and tell the truth using information theory and logit lens.
Published in Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, 2023
We find linear representations in transformers trained to solve mazes
Published in University of Sussex, 2022
I investigated directed functional connectivity in human electrocorticographic (ECoG) data during visual perception motivated by predictions from predictive coding.
Published in AI alignment forum, 2024
Inspired by singular learning theory, we study the behaviour of SGD around degenerate minima in toy loss landscapes
Published in Apart Hackathon on mechanistic interpretability, 2023
We explore the use of automated circuit discovery to extract a circuit predicting gender pronouns in GPT2