Who Owns Your Thoughts? Part 2

What happens to your thoughts once you share them with AI? Are they still yours - and can you take them back? That is the subject of this essay in three parts. This is part 2 of 3  

In Part 1 I built the case for why the human-like design of AI and the digital habits we have already formed are driving us to share increasingly intimate data. Now it is time to unpack AI memory as a technical concept and demystify the experience.

By now we have established that AI models cannot retain anything you share. But AI systems can - and that is a distinction most people miss entirely.

What we experience in a chatbot is an application. It sits in the outermost layer of the AI architecture, handling input and output to and from the model. To make machines feel human and contextually relevant, engineers must construct various workarounds in this application layer to compensate for the model's frozen state. The three most common approaches are described below.

1. The Context Window - AI's Short-Term Memory

During a chat session, an AI model sees only what is currently in this conversation - its context window. The moment it sends you a reply, the model retains no trace of what was discussed. The next time you write, all previous messages are re-sent along with your new question, creating an increasingly large request as the session grows.

There is a ceiling on how much a model can handle: currently around 200,000 tokens, or roughly 50,000 words. Beyond roughly half that limit, quality begins to degrade - the model starts omitting information, typically from the middle of long conversations, while retaining the start and the end. This is why experienced users keep sessions to a manageable length: relevance stays high, and costs stay low, since the price of a session rises with every token sent.

2. RAG - Retrieval-Augmented Generation

A second approach stores relevant documents - in the case of chatbots, full conversation histories - in specialised vector databases. This allows the model to search semantically for context beyond the current session and retrieve relevant fragments to augment its in-session knowledge. RAG has long been a standard approach for giving AI models access to company-specific or domain-specific knowledge, and it underpins the memory features of many modern chatbots and customer service tools.

When well designed, it can function as a powerful extended knowledge base. But it has no understanding of sequence or causality, memory resets when the session ends, and - as anyone who has tried to build a RAG-enabled service will tell you - it can be painfully slow and finicky to get right.

3. Persistent Memory Systems

This brings us to the more recent generation of memory systems, developed by leading AI labs and dedicated memory frameworks such as Mem0. These systems - still scaffolding built around the AI model rather than native capabilities - are designed to make memory persist across sessions and actively shape future interactions.

They work by extracting and storing atomic facts from conversations: "User is planning a trip to Barcelona in May" or "User works at Acme Corp." Think of it as a set of sticky notes on a board - a compressed, bullet-point profile of you as a user at any given moment. When deeper memory is required, these systems fall back to searching available chat sessions via RAG.

This was the memory that Claude and Gemini rolled out import functions for, when they were picking up the churned users from ChatGPT. In essence, it is just a highly compressed profile; a bulletpoint list. It works something like going on a date with someone who has studied your social media, read your old messages, and is now deploying those notes to seem like they know you well. It can feel remarkably natural. But it is not memory. It is inference from data.

Is the Memory Feature Really What You Should Worry About?

Let us return to my original moment of doubt from part 1: the decision to enable memory in ChatGPT, which can feel like crossing a line of privacy. As you may have already guessed, that decision is less consequential than it feels - because the intuition behind it has the causality backwards.

Enabling or disabling the memory feature does not determine what data the company stores. It only determines which sticky notes the model references in future sessions. The conversation data is already being retained in the application layer the moment you hit send. The memory toggle is just your experience I the application but the data pipeline operates independently as wee shall see below.

When you sign up for a chatbot service what follows goes something alone the lines of this

So how likely is it that a private conversation you had yesterday with Claude ends up shaping the next Chat GPT release?

The reassuring news is that your raw conversation does not land directly in a neural network as training data. It is filtered, de-identified, and used primarily to shape behavioural signals rather than to teach the model new facts. But for some users, that may not be reassurance enough. What if you simply want to keep your thoughts to yourself? Can we opt out? 

The Terms You Never Bothered to Read

The answer is: kind of.

First, any conversations conducted within enterprise systems are exempt from the above process. Corporate deployments cannot, for obvious reasons, feed their data to the labs. The tradeoff is that you as a user cannot easily export your data either, since data export tends to be an administrator privilege. Your most relevant concern there is your global IT department and the overall governance of your company's AI systems.

But it is a different matter for private users who essentially hand over their data the minute they sign up. A revealing signal of the direction the industry is heading: Anthropic - previously positioned as the privacy-forward alternative, explicitly promising not to train on consumer conversations - controversially reversed course in August 2025, shifting users to an opt-out model. While this drew public criticism, one could argue Anthropic was simply aligning with what had already become industry standard. Unless you discontinue the service, the providers retain wide-ranging rights over the data you share. xAI is the most candid about this.

All providers terms of service as per April 2026:

In some respects, this is not new territory. For over a decade, billions of people have used Gmail, whose terms permit Google to process email content to improve its products. Few users have read or understood those terms. What differs with AI chatbots is not the legal structure but the intimacy of the content: email is often functional and transactional, while AI chat increasingly contains something much closer to our inner monologue.

In the third and final installment of Who owns your thoughts? we will look at where all this harvested data goes and the legal and technical challenges we face if we want to identify, retrieve or forget it.