Memory & Context

Learn how Twig AI agents maintain conversation context and use memory to provide coherent, contextual responses across multiple interactions.

What is Memory?

Memory in AI agents refers to the ability to:

• Remember previous messages in a conversation

• Maintain context across multiple turns

• Reference earlier parts of the dialogue

• Provide contextually relevant responses

Without memory, each query is treated independently. With memory, agents understand the conversation flow.

How Memory Works

Conversation Flow

Without Memory (Stateless):

User: "What are your pricing plans?"
Agent: "We have Free, Pro, and Enterprise plans..."

User: "What's included in the middle one?"
Agent: "I'm not sure what you're referring to." ❌

With Memory (Stateful):

User: "What are your pricing plans?"
Agent: "We have Free, Pro, and Enterprise plans..."
[Memory stored: discussing pricing plans]

User: "What's included in the middle one?"
Agent: "The Pro plan (middle tier) includes..." ✅
[Memory used: understands "middle one" = Pro plan]

Memory Storage

Session-Based Memory:

• Stored for duration of conversation

• Cleared when session ends

• Tied to sessionId

• Best for: Live conversations, chat widgets

User-Level Memory:

• Persists across sessions

• Stored per user

• Can span days/weeks

• Best for: Returning users, ongoing projects

Ephemeral Memory:

• Exists only during single request

• No persistence

• Used for: Single-query endpoints

Memory Components

1. Conversation History

Stores the actual messages:

{
  "memory": [
    {
      "role": "user",
      "content": "What are your pricing plans?",
      "timestamp": "2024-01-15T10:00:00Z"
    },
    {
      "role": "assistant",
      "content": "We have Free, Pro, and Enterprise...",
      "timestamp": "2024-01-15T10:00:02Z"
    },
    {
      "role": "user",
      "content": "What's included in the middle one?",
      "timestamp": "2024-01-15T10:00:30Z"
    }
  ]
}

2. Context Summary

Compressed representation of conversation:

Summary: User is inquiring about pricing plans,
specifically interested in the Pro plan features.
Has not yet asked about billing or payment methods.

Key Entities:
- Plans: Free, Pro, Enterprise
- Focus: Pro plan
- Context: Feature comparison

Intent: Product research, possibly pre-purchase

3. Entity Tracking

Tracks mentioned entities:

{
  "entities": {
    "products": ["Free plan", "Pro plan", "Enterprise"],
    "features": ["SSO", "API access"],
    "intent": "product_inquiry",
    "stage": "consideration"
  }
}

Memory Configuration

Basic Settings

{
  "memory": {
    "enabled": true,
    "type": "session",           // session, user, or ephemeral
    "maxTurns": 10,              // Number of turns to remember
    "maxTokens": 2000,           // Max memory size in tokens
    "ttl": 3600                  // Time-to-live in seconds
  }
}

Advanced Settings

{
  "memory": {
    "enabled": true,
    "type": "user",
    "maxTurns": 20,
    "maxTokens": 4000,
    "ttl": 86400,                // 24 hours
    
    // Automatic summarization
    "summarizeAfter": 10,        // Summarize after 10 turns
    "keepRecentTurns": 5,        // Keep last 5 turns full
    
    // Privacy
    "encryptAtRest": true,
    "allowUserDelete": true,
    "piiDetection": true,
    
    // Persistence
    "persistAcrossSessions": true,
    "syncAcrossDevices": true
  }
}

Memory Strategies

1. Full History (Small Conversations)

Best for: Short conversations (< 10 turns)

Configuration:

{
  "maxTurns": 10,
  "summarize": false
}

Pros:

• Complete context

• No information loss

• Best accuracy

Cons:

• Token usage grows

• Slower with long conversations

2. Windowed History (Medium Conversations)

Best for: Moderate conversations (10-20 turns)

Configuration:

{
  "maxTurns": 20,
  "keepRecentTurns": 5,
  "summarizeOlder": true
}

Example:

Turns 1-15: [Summarized]
"User discussed pricing and features. 
Interested in Pro plan for team of 50."

Turns 16-20: [Full history]
User: "How do I upgrade?"
Agent: "You can upgrade by..."
User: "What's the downtime?"
Agent: "Upgrades are instant..."

Pros:

• Balanced approach

• Recent context preserved

• Manageable token usage

Cons:

• Some detail lost in summary

• Slight complexity

3. Summarized History (Long Conversations)

Best for: Extended conversations (20+ turns)

Configuration:

{
  "maxTurns": 50,
  "keepRecentTurns": 3,
  "aggressiveSummarization": true
}

Example:

Summary of turns 1-47:
"User onboarding for Enterprise plan.
Discussed: SSO setup, API integration, team training.
Current focus: Data migration timeline."

Recent turns 48-50: [Full history]

Pros:

• Handles very long conversations

• Low token usage

• Maintains key context

Cons:

• Loss of detail

• Summary quality matters

Context Enhancement

Memory-Enhanced Queries

Cedar and Cypress strategies use memory to improve retrieval:

Original Query:

"What about pricing?"

With Memory Context:

Previous: "Tell me about the Enterprise plan"
Enhanced Query: "What is the pricing for the Enterprise plan?"

Impact:

Without Memory:
- Generic pricing results
- May not be relevant

With Memory:
- Targeted Enterprise pricing
- More accurate response

Entity Resolution

Memory helps resolve ambiguous references:

Turn 1:
User: "I need help with Google Drive integration"
[Memory: topic=google_drive_integration]

Turn 2:
User: "How do I authenticate?"
[Resolved: "How do I authenticate Google Drive integration?"]

Turn 3:
User: "What about the sync frequency?"
[Resolved: "What is the sync frequency for Google Drive?"]

Session Management

Creating Sessions

# Let SDK/client generate session ID
curl -X POST https://api.twig.so/api/chat \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "prompt": "What are your plans?",
    "agentId": "agent-123",
    "sessionId": "session-' $(uuidgen)'"
  }'

Continuing Sessions

# Use same sessionId for follow-up
curl -X POST https://api.twig.so/api/chat \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "prompt": "Tell me more about the Pro plan",
    "agentId": "agent-123",
    "sessionId": "session-abc-123"  # Same ID
  }'

Ending Sessions

# Optional: explicitly end session
curl -X DELETE https://api.twig.so/api/sessions/session-abc-123 \
  -H "Authorization: Bearer YOUR_API_KEY"

Sessions auto-expire based on TTL.

Memory in Different RAG Strategies

Redwood (Standard RAG)

Memory Usage:

• Basic memory storage

• Not used for query rewriting

• Available in context

Impact: Minimal

Cedar (Context-Aware)

Memory Usage:

• Active memory analysis

• Used for query rewriting

• Enhanced context

Example:

Memory: ["What is pricing?", "We have 3 tiers...", "What about support?"]
New Query: "Is it included?"
Cedar Rewrites: "Is support included in the pricing plans?"

Impact: High - Memory directly improves query quality

Cypress (Advanced)

Memory Usage:

• Comprehensive memory integration

• Query expansion considers history

• Context-aware reranking

Impact: Highest - Memory enhances all stages

Privacy & Security

PII Detection

Automatically detect and handle sensitive information:

{
  "memory": {
    "piiDetection": true,
    "piiHandling": "MASK",  // MASK, REDACT, or BLOCK
    "piiTypes": [
      "email",
      "phone",
      "ssn",
      "credit_card",
      "address"
    ]
  }
}

Example:

User Input: "My email is [email protected]"
Stored Memory: "My email is [REDACTED_EMAIL]"

Encryption

{
  "memory": {
    "encryptAtRest": true,
    "encryptionAlgorithm": "AES-256",
    "keyRotation": "quarterly"
  }
}

User Control

{
  "memory": {
    "allowUserDelete": true,      // User can clear history
    "allowUserExport": true,      // User can download history
    "requireConsent": true,       // Ask before storing
    "retentionPeriod": 90         // Days to keep
  }
}

Best Practices

1. Choose Appropriate Memory Type

✅ Session: Chat widgets, customer support ✅ User: Returning customers, ongoing projects ✅ Ephemeral: Public APIs, privacy-sensitive ❌ Don't use user memory for anonymous users

2. Set Reasonable Limits

✅ maxTurns: 10-20 for most cases ✅ Summarize after 10-15 turns ✅ TTL: 1 hour for sessions, 24 hours for user ❌ Don't store unlimited history

3. Handle Long Conversations

✅ Implement summarization ✅ Keep recent turns in full ✅ Monitor token usage ❌ Don't truncate abruptly

4. Respect Privacy

✅ Detect and mask PII ✅ Encrypt sensitive data ✅ Allow user deletion ✅ Clear expired sessions ❌ Don't store sensitive data unnecessarily

5. Test Memory Behavior

✅ Test multi-turn conversations ✅ Verify entity resolution ✅ Check summary quality ✅ Monitor token usage ❌ Don't assume it works

Troubleshooting

Memory Not Working

Symptoms:

• Agent doesn't remember previous turns

• Follow-up questions fail

• No context continuity

Diagnosis:

// Check memory configuration
console.log(agent.memory.enabled)  // Should be true
console.log(agent.memory.type)     // Session or user

// Check session ID
console.log(request.sessionId)     // Should be consistent

Solutions:

1. Verify memory is enabled

2. Use same sessionId for follow-ups

3. Check TTL hasn't expired

4. Verify maxTurns not exceeded

Token Limit Exceeded

Symptoms:

• Errors about context length

• Truncated responses

• Missing information

Solutions:

// Reduce memory size
{
  "maxTurns": 5,          // Was 20
  "maxTokens": 1000,      // Was 4000
  "summarizeAfter": 5     // Was 10
}

Poor Summarization

Symptoms:

• Lost important context

• Agent forgets key details

• Irrelevant responses

Solutions:

// Keep more recent turns
{
  "keepRecentTurns": 7,   // Was 3
  "summarizeAfter": 12    // Was 8
}

// Use better summarization prompt
{
  "summaryPrompt": "Summarize key entities,
  user intent, and important details..."
}

Advanced Topics

Custom Memory Backends

// Implement custom storage
class CustomMemoryBackend {
  async store(sessionId, memory) {
    // Store in Redis, Database, etc.
  }
  
  async retrieve(sessionId) {
    // Retrieve from custom storage
  }
  
  async clear(sessionId) {
    // Clear memory
  }
}

Memory Analytics

// Track memory usage
{
  "analytics": {
    "trackMemoryUsage": true,
    "trackTokenUsage": true,
    "trackSummarizationQuality": true
  }
}

Metrics to Monitor:

• Average turns per session

• Token usage per turn

• Summary quality scores

• Memory hit rate

Cross-Device Sync

{
  "memory": {
    "persistAcrossSessions": true,
    "syncAcrossDevices": true,
    "userId": "user-123",         // Identify user
    "conflictResolution": "latest" // How to handle conflicts
  }
}

Next Steps

• Prompt Engineering - Craft memory-aware prompts

• Agent Configuration - Configure memory settings

• RAG Strategies - How memory enhances retrieval

• Privacy Best Practices - Secure memory handling