> ## Documentation Index
> Fetch the complete documentation index at: https://docs.claude-mem.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Worker Service

> HTTP API and Bun process management

# Worker Service

The worker service is a long-running HTTP API built with Express.js and managed natively by Bun. It processes observations through the Claude Agent SDK separately from hook execution to prevent timeout issues.

## Overview

* **Technology**: Express.js HTTP server
* **Runtime**: Bun (auto-installed if missing)
* **Process Manager**: Native Bun process management via ProcessManager
* **Port**: Per-user default `37700 + (uid % 100)` (override with `CLAUDE_MEM_WORKER_PORT`). The active port is stored in `~/.claude-mem/settings.json` and reported by `GET /api/health`.
* **Location**: `src/services/worker-service.ts`
* **Built Output**: `plugin/scripts/worker-service.cjs`
* **Model**: Configurable via `CLAUDE_MEM_MODEL` (default: `claude-haiku-4-5-20251001`)

## REST API Endpoints

The worker service exposes 22 HTTP endpoints organized into six categories:

### Viewer & Health Endpoints

#### 1. Viewer UI

```
GET /
```

**Purpose**: Serves the web-based viewer UI (v5.1.0+)

**Response**: HTML page with embedded React application

**Features**:

* Real-time memory stream visualization
* Infinite scroll pagination
* Project filtering
* SSE-based live updates
* Theme toggle (light/dark mode) as of v5.1.2

#### 2. Health Check

```
GET /health
```

**Purpose**: Worker health status check

**Response**:

```json theme={null}
{
  "status": "ok",
  "uptime": 12345,
  "port": 37742
}
```

The `port` value is the actual worker port for the current user — per-user default `37700 + (uid % 100)`, or whatever `CLAUDE_MEM_WORKER_PORT` is set to. The example above is illustrative; your value will differ.

#### 3. Server-Sent Events Stream

```
GET /stream
```

**Purpose**: Real-time updates for viewer UI

**Response**: SSE stream with events:

* `observation-created`: New observation added
* `session-summary-created`: New summary generated
* `user-prompt-created`: New prompt recorded

**Event Format**:

```
event: observation-created
data: {"id": 123, "title": "...", ...}
```

### Data Retrieval Endpoints

#### 4. Get Prompts

```
GET /api/prompts?project=my-project&limit=20&offset=0
```

**Purpose**: Retrieve paginated user prompts

**Query Parameters**:

* `project` (optional): Filter by project name
* `limit` (default: 20): Number of results
* `offset` (default: 0): Pagination offset

**Response**:

```json theme={null}
{
  "prompts": [{
    "id": 1,
    "session_id": "abc123",
    "prompt": "User's prompt text",
    "prompt_number": 1,
    "created_at": "2025-11-06T10:30:00Z"
  }],
  "total": 150,
  "hasMore": true
}
```

#### 5. Get Observations

```
GET /api/observations?project=my-project&limit=20&offset=0
```

**Purpose**: Retrieve paginated observations

**Query Parameters**:

* `project` (optional): Filter by project name
* `limit` (default: 20): Number of results
* `offset` (default: 0): Pagination offset

**Response**:

```json theme={null}
{
  "observations": [{
    "id": 123,
    "title": "Fix authentication bug",
    "type": "bugfix",
    "narrative": "...",
    "created_at": "2025-11-06T10:30:00Z"
  }],
  "total": 500,
  "hasMore": true
}
```

#### 6. Get Summaries

```
GET /api/summaries?project=my-project&limit=20&offset=0
```

**Purpose**: Retrieve paginated session summaries

**Query Parameters**:

* `project` (optional): Filter by project name
* `limit` (default: 20): Number of results
* `offset` (default: 0): Pagination offset

**Response**:

```json theme={null}
{
  "summaries": [{
    "id": 456,
    "session_id": "abc123",
    "request": "User's original request",
    "completed": "Work finished",
    "created_at": "2025-11-06T10:30:00Z"
  }],
  "total": 100,
  "hasMore": true
}
```

#### 7. Get Observation by ID

```
GET /api/observation/:id
```

**Purpose**: Retrieve a single observation by its ID

**Path Parameters**:

* `id` (required): Observation ID

**Response**:

```json theme={null}
{
  "id": 123,
  "sdk_session_id": "abc123",
  "project": "my-project",
  "type": "bugfix",
  "title": "Fix authentication bug",
  "narrative": "...",
  "created_at": "2025-11-06T10:30:00Z",
  "created_at_epoch": 1730886600000
}
```

**Error Response** (404):

```json theme={null}
{
  "error": "Observation #123 not found"
}
```

#### 8. Get Observations by IDs (Batch)

```
POST /api/observations/batch
```

**Purpose**: Retrieve multiple observations by their IDs in a single request

**Request Body**:

```json theme={null}
{
  "ids": [123, 456, 789],
  "orderBy": "date_desc",
  "limit": 10,
  "project": "my-project"
}
```

**Body Parameters**:

* `ids` (required): Array of observation IDs
* `orderBy` (optional): Sort order - `date_desc` or `date_asc` (default: `date_desc`)
* `limit` (optional): Maximum number of results to return
* `project` (optional): Filter by project name

**Response**:

```json theme={null}
[
  {
    "id": 789,
    "sdk_session_id": "abc123",
    "project": "my-project",
    "type": "feature",
    "title": "Add new feature",
    "narrative": "...",
    "created_at": "2025-11-06T12:00:00Z",
    "created_at_epoch": 1730891400000
  },
  {
    "id": 456,
    "sdk_session_id": "abc124",
    "project": "my-project",
    "type": "bugfix",
    "title": "Fix authentication bug",
    "narrative": "...",
    "created_at": "2025-11-06T10:30:00Z",
    "created_at_epoch": 1730886600000
  }
]
```

**Error Responses**:

* `400 Bad Request`: `{"error": "ids must be an array of numbers"}`
* `400 Bad Request`: `{"error": "All ids must be integers"}`

**Use Case**: This endpoint is used by the `get_observations` MCP tool to efficiently retrieve multiple observations in a single request, avoiding the overhead of multiple individual requests.

#### 9. Get Session by ID

```
GET /api/session/:id
```

**Purpose**: Retrieve a single session by its ID

**Path Parameters**:

* `id` (required): Session ID

**Response**:

```json theme={null}
{
  "id": 456,
  "sdk_session_id": "abc123",
  "project": "my-project",
  "request": "User's original request",
  "completed": "Work finished",
  "created_at": "2025-11-06T10:30:00Z"
}
```

**Error Response** (404):

```json theme={null}
{
  "error": "Session #456 not found"
}
```

#### 10. Get Prompt by ID

```
GET /api/prompt/:id
```

**Purpose**: Retrieve a single user prompt by its ID

**Path Parameters**:

* `id` (required): Prompt ID

**Response**:

```json theme={null}
{
  "id": 1,
  "session_id": "abc123",
  "prompt": "User's prompt text",
  "prompt_number": 1,
  "created_at": "2025-11-06T10:30:00Z"
}
```

**Error Response** (404):

```json theme={null}
{
  "error": "Prompt #1 not found"
}
```

#### 12. Get Stats

```
GET /api/stats
```

**Purpose**: Get database statistics by project

**Response**:

```json theme={null}
{
  "byProject": {
    "my-project": {
      "observations": 245,
      "summaries": 12,
      "prompts": 48
    },
    "other-project": {
      "observations": 156,
      "summaries": 8,
      "prompts": 32
    }
  },
  "total": {
    "observations": 401,
    "summaries": 20,
    "prompts": 80,
    "sessions": 20
  }
}
```

#### 13. Get Projects

```
GET /api/projects
```

**Purpose**: Get list of distinct projects from observations

**Response**:

```json theme={null}
{
  "projects": ["my-project", "other-project", "test-project"]
}
```

### Settings Endpoints

#### 14. Get Settings

```
GET /api/settings
```

**Purpose**: Retrieve user settings

**Response**:

```json theme={null}
{
  "sidebarOpen": true,
  "selectedProject": "my-project",
  "theme": "dark"
}
```

#### 15. Save Settings

```
POST /api/settings
```

**Purpose**: Persist user settings

**Request Body**:

```json theme={null}
{
  "sidebarOpen": false,
  "selectedProject": "other-project",
  "theme": "light"
}
```

**Response**:

```json theme={null}
{
  "success": true
}
```

### Queue Management Endpoints

#### 16. Get Pending Queue Status

```
GET /api/pending-queue
```

**Purpose**: View current processing queue status and identify stuck messages

**Response**:

```json theme={null}
{
  "queue": {
    "messages": [
      {
        "id": 123,
        "session_db_id": 45,
        "claude_session_id": "abc123",
        "message_type": "observation",
        "status": "pending",
        "retry_count": 0,
        "created_at_epoch": 1730886600000,
        "started_processing_at_epoch": null,
        "completed_at_epoch": null
      }
    ],
    "totalPending": 5,
    "totalProcessing": 2,
    "totalFailed": 0,
    "stuckCount": 1
  },
  "recentlyProcessed": [
    {
      "id": 122,
      "session_db_id": 44,
      "status": "processed",
      "completed_at_epoch": 1730886500000
    }
  ],
  "sessionsWithPendingWork": [44, 45, 46]
}
```

**Status Definitions**:

* `pending`: Message queued, not yet processed
* `processing`: Message currently being processed by SDK agent
* `processed`: Message completed successfully
* `failed`: Message failed after max retry attempts (3 by default)

**Stuck Detection**: Messages in `processing` status for >5 minutes are considered stuck and included in `stuckCount`

**Use Case**: Check queue health after worker crashes or restarts to identify unprocessed observations

#### 17. Trigger Manual Recovery

```
POST /api/pending-queue/process
```

**Purpose**: Manually trigger processing of pending queues (replaces automatic recovery in v5.x+)

**Request Body**:

```json theme={null}
{
  "sessionLimit": 10
}
```

**Body Parameters**:

* `sessionLimit` (optional): Maximum number of sessions to process (default: 10, max: 100)

**Response**:

```json theme={null}
{
  "success": true,
  "totalPendingSessions": 15,
  "sessionsStarted": 10,
  "sessionsSkipped": 2,
  "startedSessionIds": [44, 45, 46, 47, 48, 49, 50, 51, 52, 53]
}
```

**Response Fields**:

* `totalPendingSessions`: Total sessions with pending messages in database
* `sessionsStarted`: Number of sessions we started processing this request
* `sessionsSkipped`: Sessions already actively processing (not restarted)
* `startedSessionIds`: Database IDs of sessions started

**Behavior**:

* Processes up to `sessionLimit` sessions with pending work
* Skips sessions already actively processing (prevents duplicate agents)
* Starts non-blocking SDK agents for each session
* Returns immediately with status (processing continues in background)

**Use Case**: Manually recover stuck observations after worker crashes, or when automatic recovery was disabled

**Recovery Strategy Note**: As of v5.x, automatic recovery on worker startup is disabled by default. Users must manually trigger recovery using this endpoint or the CLI tool (`bun scripts/check-pending-queue.ts`) to maintain explicit control over reprocessing.

### Session Management Endpoints

#### 19. Initialize Session

```
POST /sessions/:sessionDbId/init
```

**Request Body**:

```json theme={null}
{
  "sdk_session_id": "abc-123",
  "project": "my-project"
}
```

**Response**:

```json theme={null}
{
  "success": true,
  "session_id": "abc-123"
}
```

#### 20. Add Observation

```
POST /sessions/:sessionDbId/observations
```

**Request Body**:

```json theme={null}
{
  "tool_name": "Read",
  "tool_input": {...},
  "tool_result": "...",
  "correlation_id": "xyz-789"
}
```

**Response**:

```json theme={null}
{
  "success": true,
  "observation_id": 123
}
```

#### 21. Generate Summary

```
POST /sessions/:sessionDbId/summarize
```

**Request Body**:

```json theme={null}
{
  "trigger": "stop"
}
```

**Response**:

```json theme={null}
{
  "success": true,
  "summary_id": 456
}
```

#### 22. Session Status

```
GET /sessions/:sessionDbId/status
```

**Response**:

```json theme={null}
{
  "session_id": "abc-123",
  "status": "active",
  "observation_count": 42,
  "summary_count": 1
}
```

#### 23. Delete Session

```
DELETE /sessions/:sessionDbId
```

**Response**:

```json theme={null}
{
  "success": true
}
```

**Note**: As of v4.1.0, the cleanup hook no longer calls this endpoint. Sessions are marked complete instead of deleted to allow graceful worker shutdown.

## Bun Process Management

### Overview

The worker is managed by the native `ProcessManager` class which handles:

* Process spawning with Bun runtime
* PID file tracking at `~/.claude-mem/worker.pid`
* Health checks with automatic retry
* Graceful shutdown with SIGTERM/SIGKILL fallback

### Commands

```bash theme={null}
# Start worker (auto-starts on first session)
npm run worker:start

# Stop worker
npm run worker:stop

# Restart worker
npm run worker:restart

# View logs
npm run worker:logs

# Check status
npm run worker:status
```

### Auto-Start Behavior

The worker service auto-starts when the SessionStart hook fires. Manual start is optional.

### Bun Requirement

Bun is required to run the worker service. If Bun is not installed, `npx claude-mem install` (and `npx claude-mem repair`) installs it globally during setup, with a visible clack spinner:

* **Windows**: `powershell -c "irm bun.sh/install.ps1 | iex"`
* **macOS/Linux**: `curl -fsSL https://bun.sh/install | bash`

You can also install manually via:

* `winget install Oven-sh.Bun` (Windows)
* `brew install oven-sh/bun/bun` (macOS)

## Claude Agent SDK Integration

The worker service routes observations to the Claude Agent SDK for AI-powered processing:

### Processing Flow

1. **Observation Queue**: Observations accumulate in memory
2. **SDK Processing**: Observations sent to Claude via Agent SDK
3. **XML Parsing**: Responses parsed for structured data
4. **Database Storage**: Processed observations stored in SQLite

### SDK Components

* **Prompts** (`src/sdk/prompts.ts`): Builds XML-structured prompts
* **Parser** (`src/sdk/parser.ts`): Parses Claude's XML responses
* **Worker** (`src/sdk/worker.ts`): Main SDK agent loop

### Model Configuration

Set the Claude model used for compression via environment variable or `~/.claude-mem/settings.json`:

```bash theme={null}
export CLAUDE_MEM_MODEL=claude-haiku-4-5-20251001
```

Allowed values:

* `claude-haiku-4-5-20251001` - default, fast and cheap (best for compression)
* `claude-sonnet-4-6` - balanced quality and cost
* `claude-opus-4-7` - highest quality, most expensive

## Port Allocation

The worker uses a per-user default port so different OS users on the same machine never collide:

* **Default**: `37700 + (uid % 100)` (set in `src/shared/SettingsDefaultsManager.ts`)
* **Override**: Set `CLAUDE_MEM_WORKER_PORT` (env or `~/.claude-mem/settings.json`)
* **Discovery**: `GET /api/health` returns the active port; `~/.claude-mem/settings.json` stores the configured value

If the chosen port is occupied, the worker fails to start — pin a different port via `CLAUDE_MEM_WORKER_PORT` and restart.

## Data Storage

The worker service stores data in the user data directory:

```
~/.claude-mem/
├── claude-mem.db           # SQLite database (bun:sqlite)
├── worker.pid              # PID file for process tracking
├── settings.json           # User settings
└── logs/
    └── worker-YYYY-MM-DD.log  # Daily rotating logs
```

## Error Handling

The worker implements graceful degradation:

* **Database Errors**: Logged but don't crash the service
* **SDK Errors**: Retried with exponential backoff
* **Network Errors**: Logged and skipped
* **Invalid Input**: Validated and rejected with error response

## Performance

* **Async Processing**: Observations processed asynchronously
* **In-Memory Queue**: Fast observation accumulation
* **Batch Processing**: Multiple observations processed together
* **Connection Pooling**: SQLite connections reused

## Troubleshooting

See [Troubleshooting - Worker Issues](../troubleshooting.md#worker-service-issues) for common problems and solutions.