Skip to main content

Database Architecture

Claude-Mem uses SQLite 3 with the bun:sqlite native module for persistent storage and FTS5 for full-text search.

Database Location

Path: ~/.claude-mem/claude-mem.db The database uses SQLite’s WAL (Write-Ahead Logging) mode for concurrent reads/writes.

Database Implementation

Primary Implementation: bun:sqlite (native SQLite module)
  • Used by: SessionStore and SessionSearch
  • Format: Synchronous API with better performance
  • Note: Database.ts (using bun:sqlite) is legacy code

Core Tables

1. sdk_sessions

Tracks active and completed sessions.
Indexes:
  • idx_sdk_sessions_claude_session on claude_session_id
  • idx_sdk_sessions_project on project
  • idx_sdk_sessions_status on status
  • idx_sdk_sessions_created_at on created_at_epoch DESC

2. observations

Individual tool executions with hierarchical structure.
Observation Types:
  • decision - Architectural or design decisions
  • bugfix - Bug fixes and corrections
  • feature - New features or capabilities
  • refactor - Code refactoring and cleanup
  • discovery - Learnings about the codebase
  • change - General changes and modifications
Indexes:
  • idx_observations_session on session_id
  • idx_observations_sdk_session on sdk_session_id
  • idx_observations_project on project
  • idx_observations_tool_name on tool_name
  • idx_observations_created_at on created_at_epoch DESC
  • idx_observations_type on type

3. session_summaries

AI-generated session summaries (multiple per session).
Indexes:
  • idx_session_summaries_sdk_session on sdk_session_id
  • idx_session_summaries_project on project
  • idx_session_summaries_created_at on created_at_epoch DESC

4. user_prompts

Raw user prompts with FTS5 search (as of v4.2.0).
Indexes:
  • idx_user_prompts_sdk_session on sdk_session_id
  • idx_user_prompts_project on project
  • idx_user_prompts_created_at on created_at_epoch DESC

5. tool_uses

Durable backup index for raw tool I/O (schema v51). Written from the same ingest choke point that feeds the observation generator, so it captures the same calls without adding a second capture path.
Why it exists: pending_messages is the generation queue — rows are claimed, summarized, and deleted, so raw tool payloads were unrecoverable once an observation existed. tool_uses is the durable side index that survives the queue, and it is what the get_tool_uses MCP tool (progressive-disclosure layer 4) reads. Not a transcript store: the JSONL transcripts on disk and the transcript watcher remain the spine for session history. This table is a by-reference index for tool bodies, nothing more. Idempotency: UNIQUE(content_session_id, tool_use_id) means a replayed PostToolUse (hook retry, transcript re-scan) updates the row instead of duplicating it. observation_id is linked after generation and is never overwritten once set. Payload cap: tool_input / tool_response are truncated at 64 KB with a …[truncated: N bytes] marker. content_hash is computed over the original, pre-truncation payload. Cost columns: none, deliberately. or_generation_id and or_session_id are nullable join keys back to an OpenRouter spend line; dollars live on that line, never here. Indexes: project, memory_session_id, content_session_id, session_db_id, tool_name, created_at_epoch, observation_id, or_generation_id

Legacy Tables

  • sessions: Legacy session tracking (v3.x)
  • memories: Legacy compressed memory chunks (v3.x)
  • overviews: Legacy session summaries (v3.x)
SQLite FTS5 (Full-Text Search) virtual tables enable fast full-text search across observations and session summaries. User prompts are searched by substring (LIKE) and have no FTS index.

FTS5 Virtual Tables

observations_fts

session_summaries_fts

Automatic Synchronization

FTS5 tables stay in sync via triggers. The update triggers fire only when an indexed column is written, so bookkeeping updates (sync revision, project merges, content hashes) never touch the index. Each index update appends a delete marker plus a re-insert of the row’s text, so an unscoped update trigger would grow the index on every such write:

FTS5 Query Syntax

FTS5 supports rich query syntax:
  • Simple: "error handling"
  • AND: "error" AND "handling"
  • OR: "bug" OR "fix"
  • NOT: "bug" NOT "feature"
  • Phrase: "'exact phrase'"
  • Column: title:"authentication"

Security

As of v4.2.3, all FTS5 queries are properly escaped to prevent SQL injection:
  • Double quotes are escaped: query.replace(/"/g, '""')
  • Comprehensive test suite with 332 injection attack tests

Database Classes

SessionStore

CRUD operations for sessions, observations, summaries, and user prompts. Location: src/services/sqlite/SessionStore.ts Methods:
  • createSession()
  • getSession()
  • updateSession()
  • createObservation()
  • getObservations()
  • createSummary()
  • getSummaries()
  • createUserPrompt()

SessionSearch

FTS5 full-text search with 8 specialized search methods. Location: src/services/sqlite/SessionSearch.ts Methods:
  • searchObservations() - Full-text search across observations
  • searchSessions() - Full-text search across summaries
  • searchUserPrompts() - Full-text search across user prompts
  • findByConcept() - Find by concept tags
  • findByFile() - Find by file references
  • findByType() - Find by observation type
  • getRecentContext() - Get recent session context
  • advancedSearch() - Combined filters

Migrations

Database schema is managed via migrations in src/services/sqlite/migrations.ts. Migration History:
  • Migration 001: Initial schema (sessions, memories, overviews, diagnostics, transcript_events)
  • Migration 002: Hierarchical memory fields (title, subtitle, facts, concepts, files_touched)
  • Migration 003: SDK sessions and observations
  • Migration 004: Session summaries
  • Migration 005: Multi-prompt sessions (prompt_counter, prompt_number)
  • Migration 006: FTS5 virtual tables and triggers
  • Migration 007-010: Various improvements and user prompts table
  • Migration 051: tool_uses durable raw tool I/O index

Performance Considerations

  • Indexes: All foreign keys and frequently queried columns are indexed
  • FTS5: Full-text search is significantly faster than LIKE queries
  • Triggers: Automatic synchronization has minimal overhead
  • Connection Pooling: bun:sqlite reuses connections efficiently
  • Synchronous API: bun:sqlite uses synchronous API for better performance

Troubleshooting

See Troubleshooting - Database Issues for common problems and solutions.