Who Owns Your Thoughts? Part 1

In the spring of 2025, OpenAI launched their new Memory feature in ChatGPT. Like everyone else, I was dead tired of starting from scratch in every chat, and I felt tempted to try it but also skeptical about what it entailed. After a few days of consideration, I finally pressed the "Enable Memory" button, feeling a bit like I had sold a piece of my soul to Silicon Valley. When I was ready to leave the platform eight months later, I didn't really know how to do so without losing that memory and all the thoughts I had painstakingly shared over the previous two years. That feeling - of first investing something personal and then being unable to take it with you - triggered first irritation and then curiosity regarding the field of AI memory.

What happens to your thoughts once you share them with AI? Are they still yours - and can you take them back? That is the subject of this 3 piece essay. For some this may be familiar territory, but I have found, through research and through building AI memory systems myself, that there is a great deal people have got wrong about this - myself included. Maybe you do too.

For those unfamiliar with what I am describing: the chat memory feature is what makes a chatbot feel like it knows you based on your conversation history. Switch to a different provider and it feels like meeting a stranger - you have to explain yourself all over again. It is the chatbot equivalent of Spotify's personalised playlist: so convenient and so tailored to you that leaving feels like a genuine loss.

The feature became centre of attention in the spring of 2026 when thousands of users decided to migrate from ChatGPT to Claude in protest against OpenAIs agreement with the American Ministry of Defence. Up until this point the memory feature had served like a moat for people wanting to leave the chat providers. Anthropic acted on the momentum and quickly launched a memory migration tool to bridge this moat. Other leading lab Google quickly followed suit with a similar function in Gemini (US only at the time of writing)[1] and the user verdict was immediate and merciless as Claude bypassed ChatGPT in weekly downloads.

It was like thousands of people going through what I did months earlier. Taking control of our digitally stored memories are important to us and being able to transfer from one walled garden to another was presented by many as a story of liberation in the context of “consumers voting with their feet” to support Anthropic’s ethical battle.

But while the news cycle focused on the giant-versus-giant drama, a quieter story went largely untold: the systematic and largely consensual outsourcing of our most intimate data - happening every day, at scale, and built on technology whose fundamental premises most users only vaguely understand.

In Who Owns Your Thoughts I zoom in on this issue. In part 1 I build the case for why the human-like design of AI and the digital habits we have already formed are driving us to share increasingly intimate data. Part 2 unpacks AI memory as a technical concept - demystifying the experience. I explain how today's memory is not genuine machine memory but a constructed illusion - a magic trick performed in the layers surrounding the AI model. In the third and final part I answer the question "What happens to your thoughts". I look at the current state of affairs in AI memory research and I close with the governance and legal challenges that remain stubbornly unsolved.

If you make it all the way to the end, it is my hope that you will understand a great deal more about how you and your data fit into the current AI ecosystem.

Ready? Let's dive in.

The Machine That Feels Human

For most people, first impressions of AI were of an advanced search engine that occasionally lied with the confidence of a management consultant. The technology has since evolved into a frequent conversation partner, woven into both work and private life.

The chatbot framing has stuck - even for people who know there is more to it. Andrej Karpathy, a founding member of OpenAI, argued in a recent interview that public understanding of AI is shaped almost entirely by the products built on top of the models, not by the underlying technology. The person who has only ever used ChatGPT perceives a personality, a memory, a relationship. The engineer who has worked with a raw base model knows it is, at bottom, a statistical engine that outputs the most probable continuation of whatever text it has received.

This gap between perception and reality is not a minor misunderstanding. It is the precondition for every other problem described in this essay. People who believe they are talking to someone make very different decisions than people who know they are submitting text to a statistical system. And if you want to build machines that feel human - machines that feed on human-like data  then….well, we will get to that.

In my research, I have spoken with a range of people about their AI habits. I have met professionals who have deep and intimate conversations with Microsoft Copilot about personal matters. One CTO, migrating from one chat provider to another, confessed that he made the mistake of looking into the underlying data - and was shocked by the private contents he found in this corporately owned dataset. He felt like he was spying on close colleagues, and quickly gave up the exercise altogether.

The time we spend talking to machines is growing, the intimacy of the data we produce has never been higher, and public understanding of the technology and its implications is still catching up.

The Big Memory Mystery

What really happens to the things we confess to a chatbot? Does AI remember everything? Could someone else read it? These are questions I have encountered repeatedly in conversations with friends and colleagues.

The confusion is understandable - the experience of these systems does not make the answer obvious. So let us be direct:

AI models don’t remember anything you tell them.

This is not a design flaw awaiting a patch. It is a structural property of the dominant AI architecture today. A large language model's knowledge is encoded in billions of static weight matrices, frozen at training time and never again updated.

If that seems counterintuitive, recall the early days of ChatGPT, which had no contemporary knowledge and would confidently deny that the war in Ukraine was ongoing - even though the conflict had begun months before the model was released. Chatbots have since improved and can now search the web, so it may seem like they have acquired a form of updating memory. But the fundamental constraint remains: LLMs do not learn beyond their training period.

This is one of the most important unsolved problems in AI research. If LLMs could genuinely learn from ongoing interactions, it would mark a dramatic step towards Artificial General Intelligence. So far, attempts to unfreeze models by updating their weights mid-deployment have caused what researchers call catastrophic forgetting - an abrupt collapse of previously learned knowledge.

More recent attempts have drawn on biology for inspiration. The human brain is a recurring reference point in AI research, and a paper from March 2026 observes that in their current design, LLMs never rest - they process every token as if permanently awake. The researchers propose a biologically inspired Sleepgate framework, in which AI models adopt something analogous to a sleep cycle, mapping the three core mechanisms of sleep-dependent memory consolidation - synaptic downscaling, selective replay, and active forgetting - onto computational modules operating on the model's key-value cache. A companion paper, literally titled 'Language Models Need Sleep,' demonstrates that sleep-inspired consolidation improves long-context performance. A fascinating reminder of how far we remain from machines that genuinely remember despite the feeling that AI will soon do everyhing humans do - better.

In Part 2 of Who owns you thoughts? I take a deeper look under the hood at how models actually remember today in the absence of any real memory capability