Chapter 17: Chapter 17: Memory and Context Management in AI Applications

The Memory Problem at Scale

Every AI model call starts fresh. The model has no memory of your previous conversations unless you explicitly provide that history as part of the current context. For a single-turn interaction, this is fine. For an application that serves the same user over weeks and months, this is a fundamental limitation.

Real user-facing AI applications need:

Session memory: What happened earlier in this conversation
User memory: What this user has told us about themselves across all sessions
World memory: Facts and context that the application needs to function (product catalog, documentation, policies)
Episodic memory: Specific past interactions that are relevant now

Each type requires a different storage and retrieval approach.

---

Memory Architecture 1: Full Context Injection (Session Memory)

The simplest approach: store the entire conversation history and inject it into every call.

javascript

class ConversationMemory {
  constructor(maxTokens = 50000) {
    this.messages = [];
    this.maxTokens = maxTokens;
  }
  
  addMessage(role, content) {
    this.messages.push({ role, content, timestamp: Date.now() });
    this.pruneIfNeeded();
  }
  
  pruneIfNeeded() {
    while (this.estimateTokens() > this.maxTokens && this.messages.length > 2) {
      this.messages.splice(1, 1);
    }
  }
  
  estimateTokens() {
    return this.messages.reduce((sum, m) => sum + m.content.length / 4, 0);
  }
  
  getMessages() {
    return this.messages;
  }
}

When this works: Short-to-medium conversations. Context windows up to 1M tokens make this viable for much longer than it used to be.

When it breaks: Very long-running conversations, applications where users return days later, and cases where old conversation history is irrelevant noise.

---

Memory Architecture 2: Summarisation + Compression

As conversations grow, summarise old content and compress it:

```javascript async function compressConversation(messages, keepLast = 10) { if (messages.length <= keepLast) return messages; const toCompress = messages.slice(0, messages.length - keepLast); const toKeep = messages.slice(messages.length - keepLast); const summary = await client.chat.completions.create({ model: 'gpt-4o-mini',

messages: [ ...toCompress, { role: 'user', content: 'Summarise the key information, decisions, and context from this conversation in 3-5 bullet points. This summary will replace the conversation history.' } ], max_tokens: 500, }); const summaryMessage = { role: 'system', content: 'Previous conversation summary:\n' + summary.choices[0].message.content, }; return [messages[0], summaryMessage, ...toKeep]; }

code

This approach maintains the essence of long conversations without the full token cost.

---

## Memory Architecture 3: Vector-Based Long-Term Memory

For cross-session memory — remembering things across different conversations over time — use vector embeddings stored in a database.

javascript class LongTermMemory { constructor(supabase, userId) { this.supabase = supabase; this.userId = userId; } async remember(content, category = 'general') { const embedding = await createEmbedding(content); await this.supabase.from('user_memories').insert({ user_id: this.userId, content, category, embedding, created_at: new Date().toISOString(), }); } async recall(context, limit = 5) { const contextEmbedding = await createEmbedding(context); const { data } = await this.supabase.rpc('match_memories', { user_id: this.userId, query_embedding: contextEmbedding, match_threshold: 0.7, match_count: limit, }); return data || []; } async buildMemoryContext(userMessage) { const memories = await this.recall(userMessage); if (memories.length === 0) return ''; return 'Relevant information from previous conversations:\n' + memories.map(m => '- ' + m.content).join('\n') + '\n\n'; } }

code

---

## Memory Architecture 4: Explicit User Profile

For structured user data, maintain a user profile document that grows over time:

javascript async function updateUserProfile(userId, conversationExcerpt) { const currentProfile = await getProfile(userId); const updatedProfile = await client.chat.completions.create({ model: 'gpt-4o-mini', messages: [{ role: 'user', content: 'Based on this conversation, update the user profile. Output the updated profile as JSON. Add new information, update changed information, keep unchanged fields.\n\nCurrent profile:\n' + JSON.stringify(currentProfile, null, 2) + '\n\nNew information from conversation:\n' + conversationExcerpt }], response_format: { type: 'json_object' }, }); await saveProfile(userId, JSON.parse(updatedProfile.choices[0].message.content)); } ```

This approach builds a growing picture of each user's preferences, knowledge level, and goals that improves every subsequent interaction.

---

Choosing the Right Memory Architecture

Use Case	Architecture	Why
Single-session chatbot	Full context injection	Simple, no infrastructure
Long support conversations	Compression + injection	Handles length without complexity
Personalised assistant	Vector long-term memory	Cross-session personalisation
User preference tracking	Explicit user profile	Structured, queryable
Enterprise knowledge base	RAG with vector search	Domain knowledge at scale

Chapter 17: Memory and Context Management in AI Applications

The Memory Problem at Scale

Memory Architecture 1: Full Context Injection (Session Memory)

Memory Architecture 2: Summarisation + Compression

Choosing the Right Memory Architecture

Finished reading this chapter?

Download Printable Chapter Checklist

The Memory Problem at Scale

Memory Architecture 1: Full Context Injection (Session Memory)

Memory Architecture 2: Summarisation + Compression

Get advanced blueprint resources

Choosing the Right Memory Architecture

Finished reading this chapter?

Download Printable Chapter Checklist