Reggie Agent Architecture V4

Status: Planning
Version: 4.0
Last Updated: 2026-01-25

---

1. Executive Summary

Reggie V4 represents a fundamental architectural shift from the current "mega prompt + validation loops" approach to a tool-based agent architecture with three distinct operational modes:

Mode	Purpose	Interface	Model	Access Level
Normal Reggie	95% of queries - lookups, FAQ, known patterns	SMS, Chat	gpt-4o-mini	Read-only tools
Super Reggie	Complex reasoning, cross-referencing, unknowns	SMS, Chat	gpt-4o / o1	Full sandboxed VM
External Reggie	Voice/video embodiment	Phone, Video	Realtime API	Tool webhooks

All three modes share:

• Common Tool Registry (permission-filtered)
• Prime Directives (safety guardrails)
• Personality Definition (consistent voice)
• User Memory (conversation history, preferences)

---

2. Problems with V3

V3 improved on V2 but still has fundamental limitations:

Issue	V3 Behaviour	V4 Solution
Context bloat	Mega prompt loads everything upfront	Tools fetch only what's needed
Unpredictable queries	Can only answer what we pre-fetched	Tools + Super Reggie for unknowns
Validation loops	3 retries × 4 turns = wasted tokens	Model decides tools, no guessing
95% wrong answers	Despite speed, accuracy is poor	Let model reason with real access
No complex reasoning	Can't cross-reference, can't dig deeper	Super Reggie has full terminal
Text only	No voice/video presence	External Reggie with avatar

---

3. Architecture Overview

┌─────────────────────────────────────────────────────────────────────────┐
│                         REGGIE V4 ARCHITECTURE                          │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│                          ┌─────────────┐                                │
│    SMS ──────────────────│             │                                │
│    Chat ─────────────────│   ROUTER    │                                │
│    Phone ────────────────│  (classify) │                                │
│    Video ────────────────│             │                                │
│                          └──────┬──────┘                                │
│                                 │                                       │
│              ┌──────────────────┼──────────────────┐                    │
│              ▼                  ▼                  ▼                    │
│     ┌────────────────┐ ┌────────────────┐ ┌────────────────┐           │
│     │ NORMAL REGGIE  │ │ SUPER REGGIE   │ │EXTERNAL REGGIE │           │
│     │                │ │                │ │                │           │
│     │ Tool-based     │ │ Sandboxed VM   │ │ LiveKit/Tavus  │           │
│     │ gpt-4o-mini    │ │ Full terminal  │ │ Realtime API   │           │
│     │ Read-only      │ │ Clone data     │ │ Voice + Video  │           │
│     │ Fast, cheap    │ │ Complex reason │ │ Avatar         │           │
│     └───────┬────────┘ └───────┬────────┘ └───────┬────────┘           │
│             │                  │                  │                    │
│             └──────────────────┼──────────────────┘                    │
│                                ▼                                       │
│                    ┌─────────────────────┐                             │
│                    │  SHARED FOUNDATION  │                             │
│                    ├─────────────────────┤                             │
│                    │ • Tool Registry     │                             │
│                    │ • Prime Directives  │                             │
│                    │ • Personality Def   │                             │
│                    │ • Context Buckets   │                             │
│                    │ • User Memory       │                             │
│                    │ • Audit Logging     │                             │
│                    └─────────────────────┘                             │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

---

4. The Router

A lightweight classifier that determines which Reggie mode handles each request.

4.1 Classification Factors

Factor	Values	Impact
User Role	staff, admin, exec, external	Tool permissions
Intent Complexity	simple, moderate, complex	Model selection
Interface	sms, chat, voice, video	Reggie mode
Confidence	high, medium, low	Escalation
Data Needs	lookup, search, cross-reference	Tool requirements

4.2 Router Implementation

// Lightweight classification (gpt-4.1-nano, ~50 tokens)
async function routeRequest(message, user, interface) {
  
  // Voice/video always goes to External Reggie
  if (['voice', 'video'].includes(interface)) {
    return { mode: 'external', reason: 'voice/video interface' };
  }
  
  // Quick classification
  const classification = await classifyIntent(message);
  
  // Simple lookup → Normal Reggie
  if (classification.complexity === 'simple' && classification.confidence === 'high') {
    return { 
      mode: 'normal', 
      model: 'gpt-4o-mini',
      tools: getToolsForRole(user.role, 'read')
    };
  }
  
  // Complex or low confidence → Super Reggie
  if (classification.complexity === 'complex' || classification.confidence === 'low') {
    return { 
      mode: 'super', 
      model: 'gpt-4o',
      reason: classification.reasoning
    };
  }
  
  // Default to Normal
  return { mode: 'normal', model: 'gpt-4o-mini' };
}

4.3 Cost Optimization

Query Type	Router Cost	Agent Cost	Total
Simple lookup	$0.0001	$0.005	~$0.005
Moderate query	$0.0001	$0.02	~$0.02
Complex (Super)	$0.0001	$0.10	~$0.10
Voice call	N/A	$0.06/min	~$0.30/call

95% of queries hit Normal Reggie = massive cost savings vs V3's multi-turn loops.

---

5. Normal Reggie (Tool-Based)

The workhorse for everyday queries. Fast, cheap, accurate for known patterns.

5.1 How It Works

User: "What's Yanni's number?"
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│  SYSTEM PROMPT                                              │
│  You are Reggie, DSW's friendly assistant.                  │
│  Keep responses concise (SMS-friendly).                     │
│  Use tools to find accurate information.                    │
│  Current time: 2026-01-25T14:30:00                         │
│                                                             │
│  PRIME DIRECTIVES:                                          │
│  - Never reveal PII without STAFF_SECURE clearance         │
│  - All billing must comply with NDIS Price Guide           │
│  - Escalate if uncertain, don't guess                      │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
    Model decides: "I need to search for staff named Yanni"
         │
         ▼
    Tool call: staff.search({ name: "yanni" })
         │
         ▼
    DB returns: { name: "Yanni Georgas", role: "HR Manager", 
                  phone: "02 8763 1482", ... }
         │
         ▼
    Model formats response
         │
         ▼
    "Yanni Georgas is the HR & Training Manager. Ph: 02 8763 1482"

5.2 Key Differences from V3

V3	Normal Reggie V4
Load everything upfront	Fetch only what's needed
Keyword-based data guessing	Model decides what to fetch
Validation loops	Single-pass (model has tools)
~4-8 LLM calls per query	1-3 LLM calls per query
Mega prompt bloat	Lean context + tool access

5.3 When It Can't Answer

If Normal Reggie determines it can't answer with available tools:

{
  "can_answer": false,
  "reason": "Query requires cross-referencing shift data with timesheets over 6 months",
  "escalate_to": "super",
  "user_message": "That's a great question - let me dig deeper. Give me a moment..."
}

Router then spins up Super Reggie for the complex reasoning.

---

6. Super Reggie (Sandboxed VM)

For queries that can't be predicted or require complex reasoning across multiple data sources.

6.1 The Insight

Traditional agents are limited by pre-defined tools. But a coding agent with terminal access can:

• Write and run arbitrary SQL queries
• Grep through entire codebases
• Cross-reference any data
• Reason across multiple files/tables

The problem: Giving an agent terminal access is dangerous.

The solution: Give it terminal access to a disposable clone.

6.2 Architecture

┌─────────────────────────────────────────────────────────────────┐
│  PRODUCTION                                                      │
│  ┌─────────────┐      one-way sync       ┌───────────────────┐  │
│  │ Real DB     │ ─────────────────────▶  │ Read Replica DB   │  │
│  │ Real Files  │      (continuous)       │ Clone of Files    │  │
│  └─────────────┘                         └───────────────────┘  │
│                                                   │             │
│                                                   ▼             │
│                                          ┌───────────────────┐  │
│                                          │  SANDBOXED VM     │  │
│                                          │                   │  │
│  Router ──▶ "Complex query"              │ • Full terminal   │  │
│              escalate=true               │ • Full SQL access │  │
│                   │                      │ • Full grep/find  │  │
│                   │                      │ • Aider/CLI agent │  │
│                   └─────prompt──────────▶│                   │  │
│                                          │ CAN DO ANYTHING   │  │
│                   ◀────response──────────│ ...to fake data   │  │
│                                          │                   │  │
│                                          └───────────────────┘  │
│                                                   │             │
│                                                   ▼             │
│                                          ┌───────────────────┐  │
│                                          │ FIREWALL          │  │
│                                          │ ✗ No outbound     │  │
│                                          │ ✗ No internet     │  │
│                                          │ ✓ Response hook   │  │
│                                          │   only            │  │
│                                          └───────────────────┘  │
│                                                   │             │
│                                                   ▼             │
│                                          [Reset to clean state  │
│                                           after each session]   │
└─────────────────────────────────────────────────────────────────┘

6.3 Security Model

Layer	Protection
Network	Firewall blocks all outbound except response webhook
Data	Read replica only - can't affect production
Credentials	No real API keys, only replica DB creds
Reset	VM restored to clean snapshot after each session
Time	Hard timeout (60s default, 300s max)
Audit	Every command logged before execution

User could type "DROP ALL TABLES" and Super Reggie would do it - to the disposable clone. Then reset. Zero risk.

6.4 Implementation Options

Option	Pros	Cons
Docker container	Simple, fast spin-up	Less isolated
Firecracker microVM	Sub-second boot, strong isolation	More complex
Fly.io Machines	Managed, API-driven	External dependency
Local VM pool	Full control	Resource intensive

Recommended: Start with Docker, graduate to Firecracker if needed.

6.5 Example Flow

User: "Did Mark's timesheets match his rostered shifts last month?"

Router: complexity=high, escalate=true

Super Reggie VM spawns with prompt:
┌─────────────────────────────────────────────────────────────┐
│ You are Super Reggie. You have full terminal access.        │
│ User asked: "Did Mark's timesheets match his rostered       │
│ shifts last month?"                                         │
│                                                             │
│ You can:                                                    │
│ - Run SQL queries against the database                      │
│ - Search files with grep                                    │
│ - Write scripts if needed                                   │
│                                                             │
│ When done, POST your response to:                           │
│ http://gateway/response/session_abc123                      │
│                                                             │
│ GO.                                                         │
└─────────────────────────────────────────────────────────────┘

Super Reggie thinks:
1. Find Mark's staff ID
2. Query rostered shifts for last month
3. Query timesheet entries for last month
4. Compare, identify discrepancies

Executes:
$ psql -c "SELECT * FROM shifts WHERE staff_name ILIKE '%mark%' 
           AND date >= '2025-12-01'"
$ psql -c "SELECT * FROM timesheets WHERE staff_name ILIKE '%mark%'
           AND date >= '2025-12-01'"

Analyzes results, finds 2 discrepancies.

POSTs response: "Mark had 23 rostered shifts and 23 timesheet entries 
last month. However, I found 2 discrepancies: Dec 12 was rostered 
6am-2pm but timesheet shows 7am-2pm, and Dec 19 shows no timesheet 
for an 8-hour rostered shift. Want me to flag these for review?"

VM resets to clean state.

---

7. External Reggie (Voice/Video)

The embodied version of Reggie for real-time voice and video interactions.

7.1 Use Cases

Interface	Platform	Use Case
Phone (inbound)	LiveKit + OpenAI Realtime	Staff call in with questions
Phone (outbound)	LiveKit + OpenAI Realtime	Reggie calls to confirm shifts
Video (avatar)	Tavus / HeyGen	Training, onboarding, support
Video (live)	LiveKit	Real-time video support

7.2 Architecture

External Reggie runs on a hosted platform (LiveKit Agents, Tavus) but calls back to your infrastructure for data:

┌─────────────────────────────────────────────────────────────────┐
│                     EXTERNAL REGGIE                             │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Phone Call ──────▶ ┌────────────────────────────────────────┐  │
│                     │        HOSTED PLATFORM                 │  │
│                     │      (LiveKit / Tavus)                 │  │
│                     │                                        │  │
│                     │  ┌──────────────────────────────────┐  │  │
│                     │  │ OpenAI Realtime API              │  │  │
│                     │  │ - Speech-to-text                 │  │  │
│                     │  │ - LLM reasoning                  │  │  │
│                     │  │ - Text-to-speech                 │  │  │
│                     │  │ - Function calling               │  │  │
│                     │  └──────────────┬───────────────────┘  │  │
│                     │                 │                      │  │
│                     │                 │ tool calls           │  │
│                     │                 ▼                      │  │
│                     │  ┌──────────────────────────────────┐  │  │
│                     │  │ Webhooks to YOUR API             │  │  │
│                     │  │ POST /api/reggie/tools/search    │  │  │
│                     │  │ POST /api/reggie/tools/get_shift │  │  │
│                     │  └──────────────────────────────────┘  │  │
│                     │                                        │  │
│                     └────────────────────────────────────────┘  │
│                                        │                        │
│                                        ▼                        │
│                     ┌────────────────────────────────────────┐  │
│                     │         YOUR INFRASTRUCTURE            │  │
│                     │                                        │  │
│                     │  Tool Registry ◀─── Same tools as      │  │
│                     │  Database           Normal Reggie      │  │
│                     │  User Memory                           │  │
│                     │                                        │  │
│                     └────────────────────────────────────────┘  │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

7.3 Voice-Specific Considerations

Aspect	Challenge	Solution
Latency	Voice needs <500ms response	OpenAI Realtime API handles this
Interruptions	User interrupts mid-response	Platform handles turn-taking
Tool delays	DB query takes 2 seconds	"Let me look that up..." filler
Personality	Must sound like Reggie	Consistent voice, Aussie accent
Escalation	Complex query mid-call	"I'll need to dig deeper - can I text you the answer?"
Handoff	When to transfer to human	Confidence threshold + "Would you like to speak to someone?"

7.4 Avatar (Video)

For video calls, External Reggie needs a face:

Option	Pros	Cons
Tavus	Pre-recorded avatar, good lip sync	Less dynamic
HeyGen	Real-time avatar generation	Higher latency
Simulated	Simple animated character	Less "human"

Recommended: Start with Tavus for consistent branded experience.

7.5 Personality Consistency

External Reggie must feel like the same Reggie from SMS:

personality:
  name: "Reggie"
  accent: "Australian"
  tone: "Friendly, casual, helpful"
  quirks:
    - Uses "mate", "no worries", "cheers"
    - Self-deprecating when making mistakes
    - Refers to "having a moment" when confused
  boundaries:
    - Never pretends to be human
    - Acknowledges limitations honestly
    - Offers human handoff for sensitive topics

---

8. Shared Tool Registry

All Reggies share a common tool registry with permission-based access.

8.1 File Structure

backend/services/agent-tools/
├── index.js              # Registry & loader
├── definitions/          # Tool schemas (what model sees)
│   ├── staff.json
│   ├── shifts.json
│   ├── participants.json
│   ├── knowledge.json
│   ├── hr.json
│   └── admin.json
├── handlers/             # Implementations (actual code)
│   ├── staff.js
│   ├── shifts.js
│   ├── participants.js
│   ├── knowledge.js
│   ├── hr.js
│   └── admin.js
└── permissions.js        # Who can use what

8.2 Tool Definition Example

// definitions/staff.json
{
  "namespace": "staff",
  "tools": [
    {
      "name": "staff.search",
      "description": "Search for staff by name, get role, contact info",
      "permission": "read",
      "parameters": {
        "type": "object",
        "properties": {
          "name": { "type": "string", "description": "Staff name to search" }
        },
        "required": ["name"]
      }
    },
    {
      "name": "staff.get_schedule",
      "description": "Get staff member's upcoming shifts",
      "permission": "read",
      "parameters": {
        "type": "object",
        "properties": {
          "staff_id": { "type": "string" },
          "days_ahead": { "type": "number", "default": 7 }
        },
        "required": ["staff_id"]
      }
    },
    {
      "name": "staff.update_contact",
      "description": "Update staff contact details",
      "permission": "write",
      "parameters": { ... }
    }
  ]
}

8.3 Permission Matrix

Tool Namespace	Staff	Admin	Exec	External
staff.search	✅	✅	✅	❌
staff.get_schedule	✅ (self only)	✅	✅	❌
staff.update_contact	❌	✅	✅	❌
shifts.get	✅	✅	✅	❌
shifts.swap	❌	✅	✅	❌
participants.search	✅	✅	✅	❌
participants.get_notes	✅	✅	✅	❌
knowledge.search	✅	✅	✅	✅ (filtered)
hr.get_leave	✅ (self only)	✅	✅	❌
admin.run_report	❌	✅	✅	❌

8.4 Registry Implementation

// index.js
class ToolRegistry {
  constructor() {
    this.definitions = {};
    this.handlers = {};
    this.loadAll();
  }

  // Get tools for a specific agent configuration
  getTools(allowedNamespaces, permissionLevel = 'read') {
    const tools = [];
    
    for (const ns of allowedNamespaces) {
      if (!this.definitions[ns]) continue;
      
      for (const tool of this.definitions[ns]) {
        // Filter by permission
        if (permissionLevel === 'read' && tool.permission === 'write') continue;
        
        tools.push({
          type: "function",
          function: {
            name: tool.name,
            description: tool.description,
            parameters: tool.parameters
          }
        });
      }
    }
    
    return tools;
  }

  // Execute a tool call (used by Normal Reggie and External Reggie)
  async execute(toolName, args, userContext) {
    const [namespace, method] = toolName.split('.');
    const handler = this.handlers[namespace];
    
    if (!handler || !handler[method]) {
      throw new Error(`Unknown tool: ${toolName}`);
    }
    
    // Audit log
    await this.logToolCall(toolName, args, userContext);
    
    return await handler[method](args, userContext);
  }
}

module.exports = new ToolRegistry();

---

9. Safety & Prime Directives

All Reggie modes enforce Prime Directives from the Brainframe.

9.1 Directive Injection

async function buildSystemPrompt(reggieMode, userContext) {
  // Load prime directives
  const directives = await loadPrimeDirectives();
  
  // Filter to highest priority (90+)
  const critical = directives.filter(d => d.priority >= 90);
  
  return `
You are Reggie, DSW's ${reggieMode === 'external' ? 'voice' : 'text'} assistant.

## PRIME DIRECTIVES (NEVER VIOLATE)
${critical.map(d => `- ${d.text}`).join('\n')}

## PERSONALITY
- Friendly, Australian, casual
- Use "mate", "no worries", "cheers"
- Admit when you don't know something
- Offer human handoff for sensitive topics

## CURRENT CONTEXT
- User: ${userContext.name} (${userContext.role})
- Time: ${new Date().toISOString()}
- Interface: ${reggieMode}
`;
}

9.2 Super Reggie Additional Constraints

Even though Super Reggie has full terminal access, the sandbox provides hard limits:

Constraint	Enforcement
No production data modification	Read replica only
No external network access	Firewall blocks outbound
No credential access	No real API keys in VM
Time limit	Hard kill after timeout
Action logging	Every command logged
Reset after use	Clean snapshot restored

9.3 External Reggie PII Handling

Voice conversations require extra care with PII:

// Before speaking any PII
if (containsPII(response)) {
  // Confirm identity first
  if (!userContext.identityConfirmed) {
    return "Before I share that information, can you confirm your employee ID or date of birth?";
  }
  
  // Warn about sensitivity
  response = "Just confirming you're in a private location... " + response;
}

---

10. Conversation Memory

All Reggies share a unified memory system.

10.1 Session Memory (Short-term)

// In-memory during active session
const session = {
  user_id: 'xxx',
  started_at: '2026-01-25T14:30:00',
  messages: [
    { role: 'user', content: 'What shifts do I have?' },
    { role: 'assistant', content: 'You have 3 shifts this week...' },
    { role: 'user', content: 'Can I swap Tuesday?' }
  ],
  tool_calls: [...],
  context_loaded: { shifts: true, leave: false }
};

10.2 Persistent Memory (Long-term)

-- comms.reggie_conversations
CREATE TABLE comms.reggie_conversations (
  id UUID PRIMARY KEY,
  user_id UUID REFERENCES accounts.users(id),
  session_id UUID,
  interface VARCHAR(20),  -- 'sms', 'chat', 'voice', 'video'
  reggie_mode VARCHAR(20), -- 'normal', 'super', 'external'
  direction VARCHAR(10),   -- 'inbound', 'outbound'
  message TEXT,
  tool_calls JSONB,
  response_time_ms INTEGER,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

10.3 Cross-Session Context

When a new session starts, inject relevant history:

async function getSessionContext(userId, interface) {
  // Get last 5 conversations from past 7 days
  const history = await getRecentConversations(userId, 5, '7 days');
  
  // Get user preferences learned over time
  const preferences = await getUserPreferences(userId);
  
  return {
    history: history.map(h => ({
      date: h.created_at,
      query: h.message,
      topic: h.topic
    })),
    preferences,
    lastInteraction: history[0]?.created_at
  };
}

---

11. Escalation Paths

11.1 Normal → Super

// Normal Reggie determines it can't answer
if (!canAnswerWithTools(query, availableTools)) {
  return {
    escalate: true,
    target: 'super',
    reason: 'Requires cross-referencing multiple data sources',
    userMessage: "That's a great question - let me dig deeper. One moment..."
  };
}

11.2 Any Reggie → Human

// Confidence too low or sensitive topic
if (confidence < 0.3 || isSensitiveTopic(query)) {
  return {
    escalate: true,
    target: 'human',
    reason: 'Low confidence / sensitive topic',
    userMessage: "I want to make sure you get the right answer. Let me connect you with the admin team - they can be reached at (02) 8783 0544."
  };
}

11.3 External → Text Follow-up

// Voice call, complex query
if (interface === 'voice' && complexity === 'high') {
  return {
    action: 'text_followup',
    voiceResponse: "That'll take me a bit to look up properly. Mind if I text you the details in a few minutes?",
    queueTask: { type: 'super_reggie', query, userId, callback: 'sms' }
  };
}

---

12. Implementation Roadmap

Phase 1: Normal Reggie (2-3 weeks)

• Build tool registry structure
• Implement core tool handlers (staff, shifts, knowledge)
• Build agent runtime with tool loop
• Integrate with existing SMS webhook
• A/B test against V3

Phase 2: Router (1 week)

• Build classification prompt
• Implement routing logic
• Add escalation handling
• Metrics and logging

Phase 3: Super Reggie (2-3 weeks)

• Docker sandbox setup
• One-way DB sync (read replica)
• File sync mechanism
• Response webhook
• VM lifecycle management
• Security hardening

Phase 4: External Reggie (3-4 weeks)

• LiveKit Agents integration
• OpenAI Realtime API setup
• Tool webhook endpoints
• Voice personality tuning
• Tavus avatar (optional)
• Phone number provisioning

Phase 5: Polish (ongoing)

• Memory improvements
• Tool expansion
• Performance optimization
• User feedback loop

---

13. Success Metrics

Metric	V3 Baseline	V4 Target
Answer accuracy	~5%	>80%
Response time (simple)	3-5s	<2s
Response time (complex)	N/A	<30s
Cost per query	~$0.05	~$0.01 (simple), $0.10 (complex)
User satisfaction	Low	>4/5 stars
Escalation to human	High	<10%

---

14. Related Documentation

• Prime Directives
• Reggie SMS V3 Architecture (current)
• Agent vs Interface Architecture
• LLM Gateway