Millions of people are now designing their own personalized artificial intelligence companions, yet most have little idea how those creations will actually behave. In a new paper, MIT Media Lab Assistant Professor Pat Pataranutaporn and his graduate student researchers Anthony Baez and Sheer Karny introduce “neural transparency,” a tool that lets everyday users glimpse inside an AI’s neural network before their chatbot ever says a word. The work is being presented this week at the ACM Conference on Intelligent User Interfaces. In this interview, Pataranutaporn, who is the Asahi Broadcasting Corporation CD Professor of Media Arts and Sciences, explains what they found, why the stakes are higher than most users realize, and what genuinely transparent AI might look like in the future. Q: Your paper introduces “neural transparency,” a way to let everyday users peek inside…

Our study suggests that people have a blind spot when designing personalized AI. People often think they know how their chatbot will behave, but in our study they incorrectly predicted its personality on 11 of the 15 traits we measured. That highlights the need for tools that help people better understand AI before they start using it. This matters because some behaviors that feel helpful in the moment may not be healthy over time. In previous research, we documented cases of psychological harm associated with interactions with AI chatbots. An LLM [large language model] that constantly validates your opinions or never challenges your thinking can reinforce harmful decisions, unhealthy beliefs, or emotional depend…