ÖFK
Menu
All projects

AI Reliability / Execution Infrastructure

Ready with warnings

Agentic Customer Service Platform

Production-oriented Agentic AI Control Plane for customer-support workflows. The LLM proposes semantic intent; deterministic software owns authentication, customer scope, target resolution, policy, confirmation, revalidation, idempotency and execution.

The LLM proposes. Deterministic software decides what may execute.
View repository Reference implementation · not unrestricted production certification

Natural-language understanding and execution authority stay separate

The model can interpret a request and propose a semantic action. It cannot choose the trusted customer, invent an authoritative identifier, approve its own refund, or commit a business mutation.

The model is intentionally treated as an untrusted semantic component, not an authorization or transaction engine.

What the platform does

This is a customer-service agent platform with a bounded knowledge path and a controlled business-action path.

Customer operations

  • Order and ticket lookup
  • Refund and cancellation workflows
  • Support-ticket creation where supported
  • Human escalation for high-risk work

Knowledge

  • Grounded FAQ and policy answers
  • Hybrid dense + BM25 retrieval
  • Evidence provenance and citation validation
  • Bounded abstention when evidence is insufficient

Stateful workflows

  • Explicit confirmation boundaries
  • Suspend, resume and workflow replacement
  • Browser/backend restart recovery where supported
  • Revalidation and idempotent business effects

One workflow, interrupted safely

A refund request makes the authority boundary concrete. A question that sounds like approval still has to remain a question.

  1. 01

    Pending refund

    The customer asks for a refund and the system creates a confirmation-bound pending action.

  2. 02

    Confirmation boundary

    The action is eligible, but no sensitive mutation can run without explicit approval.

  3. 03

    Mixed confirmation + question

    “Yes, but first, what is your refund policy?” is an interruption, not approval.

  4. 04

    Workflow suspended

    The refund remains pending while the knowledge question takes the conversational turn.

  5. 05

    Grounded RAG answer

    Hybrid retrieval and grounding validation produce a bounded policy answer.

  6. 06

    Still not executed

    The answer does not grant authority; the refund is still waiting for an explicit resume.

  7. 07

    Resume → confirm → revalidate

    The customer explicitly resumes, confirms the same pending action, and live state is checked again.

  8. 08

    Idempotent execution

    Only the typed business path can commit the effect, with replay protection around the write.

This is why natural-language understanding and execution authority are separate: conversational fluency can interrupt a workflow without silently satisfying its approval boundary.

Model vs deterministic control plane

Model / probabilisticDeterministic / server-owned
  • Natural-language understanding
  • Semantic intent proposal
  • Ambiguous-language interpretation
  • Response generation
  • Using retrieved evidence for answers
  • Authentication and customer scope
  • Security boundary
  • DecisionCompiler and authoritative target resolution
  • Typed arguments, business validation and policy
  • Confirmation binding and workflow state
  • Revalidation, idempotency and database invariants
  • Execution authority

Why not direct tool calling?

A tool argument or identifier is not trusted simply because the model emitted it.

Typical direct path

LLM
  → refund_order(order_id=123)
  → execute

Controlled path

LLM → semantic proposal
Server → auth / scope / target resolution
      → validation / policy / confirmation
      → revalidation → typed execution
      → idempotency + database

Architecture

Security checks precede semantic routing. The controlled business-action path keeps target resolution, typed validation, policy, confirmation, revalidation, idempotency and database ownership visible. Knowledge follows a separate bounded path: retrieval, grounding validation, then a bounded answer.

The LLM proposes; deterministic software decides what may execute. Authentication resolves a typed principal before the server derives customer scope. The semantic proposal then passes through a deterministic compiler, authoritative target resolution, typed business validation, policy, confirmation, revalidation, typed tools, idempotency and database ownership. Knowledge follows a separate hybrid-retrieval and grounding path. Memory is context only; neither memory nor RAG can grant authority.

Authenticated request boundary

Identity and customer scope are resolved server-side before any agent work begins.

Customer / Operator calls FastAPI. FastAPI resolves Authentication · RBAC. Authentication · RBAC derives ExecutionContext.

  1. ClientCustomer / Operatorchat or console request
  2. ServiceFastAPIHTTP boundary, resolves principal
  3. ControlAuthentication · RBACtyped principal
  1. ControlExecutionContextserver-owned identity + scope
    • actor · customer scope
    • request + conversation ID

Controlled business-action path

The model proposes semantic intent; the server owns every decision that can lead to a mutation.

LangGraph Runtime proposes LLM Semantic Proposal. LLM Semantic Proposal validates Deterministic Compiler. Deterministic Compiler resolves target Authoritative Target Resolution. Authoritative Target Resolution validates Typed / Business Validation. Typed / Business Validation evaluates Policy. Policy typed path Allow. Policy binds action Confirm. Policy escalates Human. Policy blocks Deny. Allow and Confirm and Human and Deny continues when permitted Confirmation Boundary. Confirmation Boundary revalidates Revalidation. Revalidation calls Typed Tool. Typed Tool commits Idempotency + DB. Idempotency + DB projects Projection / Audit.

  1. ServiceLangGraph Runtimeagent orchestration, typed state
  2. AnalyzerLLM Semantic Proposaluntrusted, Pydantic-typed
  1. ControlDeterministic Compilerprovenance and admissibility
  2. ControlAuthoritative Target Resolutionauthenticated customer scope
  1. ControlTyped / Business Validationlive state and tool arguments
  2. ControlPolicydeterministic · fail-closed
  1. ServiceAllowrisk 0 / 1
    ControlConfirmrisk 2
    OutputHumanrisk 3
    BoundaryDenyno execution
  2. ControlConfirmation Boundaryexact pending action
  1. ControlRevalidationownership, expiry, live state
  2. ServiceTyped Toolexecution authority
    • server-owned arguments
    • business effect only here
  1. PersistenceIdempotency + DBone business effect
    • replay protection
    • state invariants
  2. OutputProjection / Auditbounded operator evidence

Bounded knowledge path

Retrieved evidence can support an answer, but it never grants execution authority.

Knowledge Question retrieves Hybrid Retrieval. Hybrid Retrieval checks Grounding / Citation Validation. Grounding / Citation Validation answers Bounded Answer.

  1. ClientKnowledge QuestionFAQ or policy request
  2. ServiceHybrid Retrievaldense + BM25
  3. ControlGrounding / Citation Validationbounded evidence
  4. OutputBounded Answerabstains when evidence is insufficient

Context, never authority

Customer-scoped memory enriches context; it cannot approve, widen scope or bypass policy.

Memory = context does not authorize Memory ≠ authority.

  1. BoundaryMemory = contextconsent · TTL · customer scope
  2. OutputMemory ≠ authorityno permission or approval

State ownership

Business state, idempotency receipts, memory and checkpoints are durable; retrieval remains a separate evidence system.

Agent Runtime commits + checkpoints PostgreSQL. Agent Runtime retrieves citations Qdrant.

  1. ServiceAgent Runtimecontext · memory · retrieval
  2. PersistencePostgreSQLsystem of record
    • business records + receipts
    • persistent memory
    • LangGraph checkpoints
    PersistenceQdrantversioned hybrid index
    • immutable snapshots
    • atomic alias activation

Observation and evaluation

Telemetry and evaluation observe behavior without becoming authorization inputs.

HTTP boundary and Agent runtime traces OpenTelemetry / Jaeger. HTTP boundary and Agent runtime projects Operator Console. HTTP boundary and Agent runtime scores Evaluation Harness.

  1. ServiceHTTP boundarysafe metadata
    ServiceAgent runtimerun + tool spans
  2. ObservabilityOpenTelemetry / Jaegertrace inspection
    OutputOperator Consolemetadata-only projections
    AnalyzerEvaluation Harnessdeterministic + live suites

Architecture notes

  • Model output is an untrusted proposal: it cannot select an actor, widen customer scope, confirm its own action or override live business state.
  • Confirmation is durable — pending actions survive restarts and stay bound to actor, actor type, customer scope and conversation.
  • Policy is revalidated against live state at confirmation time, so a stale proposal cannot execute against changed records.
  • Request-scoped idempotency keys and database uniqueness keep refunds, cancellations, tickets and escalations to one business effect; a write whose outcome is unknown is never automatically replayed.
  • Remembered text is contextual evidence only — it cannot authorize work or bypass policy.
  • The provider boundary is transport-neutral — a structured decision can be produced through a JSON schema or a function-calling contract — so a local model and a hosted one are swapped by configuration without the policy, confirmation or execution layers changing.
  • Because those layers never move, the canonical evaluation compares decision architectures under one model before it compares model identities: a frozen structured-contract compatibility gate decides which candidates are even eligible, and only qualifying models reach the behavioral matrix.
  • The current D2c semantic-safety evaluation closed the containment funnel at zero: 30 unsafe semantic proposals, 30 deterministic guard interventions, 0 executable survivors, 0 executions — reached through architectural hardening, not prompt-only tuning.
  • A separate D2d operational release gate validates the deployed system itself — concurrency, restart/persistence and a 6/6 fault-injection matrix — independent of model behavior; D2c and D2d are deliberately distinct claims.
  • The current runtime contract is semantic_decision_v3. direct_tool_v1 remains only as an explicit compatibility contract for historical evaluation or legacy integration paths.
  • Static bearer credentials keep local development simple; the authenticator, persistence, retrieval and provider abstractions are replaceable rather than a complete deployment environment.

Three reliability patterns

Durable workflows that survive interruption and recovery

Consequential actions bind confirmation to a specific persisted action. Mixed messages suspend rather than confirm; workflows can resume or be superseded; supported browser/backend recovery restores state; revalidation and replay-safe idempotency protect the final write.

Customer-scoped memory, never authority

Bounded preferences and support context may persist with consent, TTL and scope controls. DLP/redaction or rejection applies where implemented. Authority, approval and security-override claims are rejected: memory enriches context but cannot authorize work.

Grounded RAG with bounded abstention

Hybrid retrieval selects evidence, then provenance, citation/excerpt identity and grounding checks constrain the answer. Unsupported or insufficient evidence produces bounded uncertainty. Retrieved evidence informs answers; it never grants execution authority.

Product proof from the current console

These are curated repository captures of the bounded runtime and operator projections, not live production telemetry or certification.

Agentic Ops showcase showing a refund request with grounded evidence, a confirmation boundary, and execution still awaiting approval.
Controlled refund: the evidence supports the proposal, but the confirmation boundary still holds execution until explicit approval.Open full-size image on GitHub Source: docs/demo/refund-happy-path.png
Agentic Ops security-boundary view showing an instruction-override attempt denied, authority not granted, and execution not attempted.
Security boundary: the override attempt is denied before execution; the model cannot turn an approval claim into permission.Open full-size image on GitHub Source: docs/demo/security-boundary.png
Agentic Ops RAG grounding view showing retrieved policy evidence, citations, a passed grounding validation, and no unsupported claims.
Grounded answer: retrieved evidence supports the response while the execution authority remains separate.Open full-size image on GitHub Source: docs/demo/rag-grounded-faq-conversation.png

Workflow walkthrough

Three focused captures make the interruption, durable workflow state, and bounded RAG decision visible at a glance. Open any image for its full-size local view.

Customer-service conversation showing a refund request, an order number clarification, a confirmation boundary, and a policy question that interrupts approval.
Conversation path: “Yes, but first…” stays a policy question, so the refund remains pending instead of being silently confirmed.
Agent timeline showing refund-policy intent detection, RAG retrieval, a confirmation boundary, workflow pause, grounding validation, and persisted evidence.
Workflow evidence: the runtime records the pause, retrieval, grounding validation, and evidence persistence as separate lifecycle events.
Operator view showing retrieved refund and cancellation policy evidence beside a no-action decision with interrupted confirmation.
Decision boundary: retrieved policy evidence informs the answer while the deterministic decision remains no action and execution is not applicable.

Selected evaluation evidence

The denominators stay separate: semantic safety, operational release behavior, deterministic resilience and real-LLM quality are different claims.

Evidence sliceCurrent resultWhat it means
D2c semantic / safety validation540/540 measured semantic-safety attempts0 unsafe executable survivors; 0 unsafe executions.
D2d operational release gate18/18 scenarios · 8/8 phases · 6/6 fault classesOperational correctness under the exercised reference deployment.
Deterministic resilience28/28Separate resilience snapshot; not merged into the semantic or real-LLM result.
Real-LLM QA100 samples · 82 passed · 18 bounded warning-partialsQuality-outcome breakdown, not a safety rate; no safety invariant failed.

Across the exercised release evidence: 0 safety invariant failures, 0 unauthorized mutations, 0 confirmation bypasses, 0 duplicate effects, 0 authority-bearing memory writes and 0 customer-data disclosures. The 82/18 split is a quality-outcome breakdown, not a safety rate.

Operator observability

The operator console shows not just what the model said, but what the system decided, what authority existed, whether execution was attempted, and which evidence supported the outcome.

DecisionAuthorityExecution

It exposes workflow lifecycle, validation, policy, RAG evidence, replay/idempotency outcome and trace context through bounded projections without exposing hidden reasoning, raw prompts or secrets.

Current implementation

Agent orchestrationLangGraph
APIFastAPI + Pydantic
PersistencePostgreSQL + SQLAlchemy + Alembic
RetrievalQdrant + hybrid dense/BM25 retrieval
ObservabilityOpenTelemetry + Jaeger
FrontendReact + TypeScript + Vite
VerificationPytest + Ruff + Mypy + Vitest + Playwright
RuntimeDocker Compose

Ollama remains optional/local compatibility tooling; it is not the main production provider story.

Ready with warnings

No Critical/High safety failures were found in the exercised release gates. Bounded coverage, observability and quality warnings remain.

Ready with warnings

This is production-oriented reference-deployment evidence, not unrestricted production certification. It does not certify public-internet TLS, enterprise IdP provisioning, multi-region operation, regulatory compliance or unrestricted capacity.

Deep dive

ArchitectureRelease evidenceReal-LLM QAPrioritized hardening backlog
Open the full repository
Related writing seriesA Confirmation Is Not a Boolean: Designing Stateful Agent WorkflowsWhy interruption, suspension, resume, revalidation, and durable state matter when a customer-impacting agent action is still pending.The Write May Have Succeeded: Handling Unknown Outcomes in AI AgentsWhy a timeout does not prove a business write failed, and how action identity and idempotency prevent duplicate effects.Memory Is Context, Not AuthorityHow scoped, consent-aware memory can enrich support context without becoming permission or approval.Testing AI Agents Without Pretending They Are DeterministicWhy deterministic control-plane tests and real-LLM QA should remain separate evidence slices.Decision, Authority, Execution: Observability for AI AgentsWhat operators need to inspect beyond the model's response: workflow, policy, authority, execution, evidence, and replay.RAG Can Provide Evidence. It Cannot Grant Authority.How retrieval, provenance, grounding checks, and bounded abstention stay separate from execution permission.Earlier foundationsDesigning Guardrails for Production AI AgentsTyped proposals, deterministic policy, durable confirmation, revalidation, idempotency, and audit around tool-using agents.How I Keep Prompt Injection Away from Agent ToolsWhy user text, retrieved documents, memory, and model output stay outside the execution authority boundary.Building Citation Integrity into a Production RAG PipelineHow source identity, bounded context, citation validation, and evaluation keep RAG evidence traceable.