Three tiers of inference, one cryptographic receipt chain. Tier 0 captures (glasses, sensors). Tier 1 routes on-device (phone). Tier 2 escalates to the sovereign substrate (Howdify-owned MoE, never a third-party API). Every event in every tier signs a receipt that commits to its parent. Tampering at any tier breaks the chain at the next signed reference. This page is the v3 Lab POC scope.
The market is converging on a hybrid inference pattern: a small model on the device handles the hot path, a frontier model in the cloud handles complex queries. Every major vendor ships some version of this. Apple Intelligence, Google AICore + Gemini Nano, Microsoft Phi-Silica plus Azure OpenAI. The on-device model handles privacy-sensitive and latency-sensitive work locally; the cloud handles depth.
Every shipping version of this pattern has the same problem: the cloud half is someone else's runtime. Apple's cloud is Apple. Google's is Google. Microsoft's is Microsoft. Even the open-source stacks default to running inference in a vendor account where the tenant has no way to prove their data did not pass through systems they don't control.
HALO replaces the cloud half with a sovereign substrate. Same hybrid pattern. Same hot path on the device. Same depth on escalation. But the substrate is a Howdify-owned MoE running on client-controlled infrastructure, and every event from the sensor capture through the inference is cryptographically receipted in a chain that survives audit.
Howdify Core continues to use Bedrock for workloads where vendor-managed inference is acceptable. HALO is the new sovereign tier for clients who require the substrate fully under their control. Different product, different positioning, no conflict between them.
The distinguishing architectural characteristic of HALO is what happens at every tier boundary. An HMAC-signed receipt is written for every event: sensor captures, routing decisions, expert loads, token emissions, response returns. Each receipt commits to its parent receipt's hash via a stored pointer, so the chain forms a verifiable graph from the moment a sensor wakes up through the final token rendered back on the display.
Any byte modified at any tier breaks verification at the next downstream check, with the failed tier and event identified deterministically by the audit tool. The chain is not an add-on or a logging layer. It is how the system observes itself, and it is what makes after-the-fact reproduction of any decision feasible.
This becomes the operational substrate for regulated-industry use cases (compliance, field service, executive briefing) where reproducing the decision-making chain is the deliverable, not a feature. See §03 for tenant-mapped use cases.
Bootstrap is over link-encrypted transport at session start. Per-tenant and per-session subkeys are derived from the existing Howdify HMAC root via HKDF (full hierarchy in §07). All receipts land in the existing Howdify receipt ledger with new receipt_kind values for HALO-specific events.
HALO is not a single-use product. It is the substrate primitive that makes sovereign tier-escalated AI possible across verticals Howdify already serves, plus regulated verticals Howdify wants to enter.
Operator wearing glasses double-taps to query inventory or aging POs. Tier 0 captures the SKU label or item geometry via the IMU-triggered optical sensor. Tier 1 routes simple lookups locally; complex queries escalate to the sovereign substrate. Receipts prove what item context was processed and what the engine returned without exposing raw visuals to an external cloud.
Employee asks "how many of this in stock," "what's the lab COA on this batch," "is this customer in our loyalty tier." Hot-path responses on-device. Escalations to substrate for personalized product recommendations or compliance lookups. Cannabis-regulated verticals like Cosmic Reserve gain a sovereignty claim regulators will recognize.
Every action taken with HALO produces a receipt chain that can be replayed and verified. For SOC 2, ISO 42001, HIPAA, and pharma GxP, this is the difference between "AI was used in a workflow" (today) and "every AI-influenced action is cryptographically provenanced from sensor capture through inference" (HALO). The receipts ARE the audit artifact.
Field technician captures descriptive audio plus an image frame of the target machinery. Tier 1 matches obvious diagnostic profiles locally. Faulty configurations or specialized asset histories escalate to the substrate. The receipt chain becomes an immutable liability and insurance artifact proving exactly what the technician saw and what the model diagnosed.
Executive in a meeting receives discreet briefing data on incoming people, topics, or financial figures via Tier 0 display. Tier 1 stays local for sensitive material; only de-identified queries escalate. The sovereign chain proves no third party heard, saw, or processed the conversation. This is the CEO-grade use case that justifies premium pricing for HALO over commodity hybrid stacks.
Tier 0 captures audio. Tier 1 transcribes locally with Gemini Nano. Tier 2 substrate only sees what the user explicitly escalates for analysis. Default-private architecture, opt-in cloud, receipt chain proves boundary compliance. Applies to legal, medical, journalist, and therapist workflows where ambient audio capture is useful but cloud transcription is unacceptable.
UC-01 through UC-03 map directly onto Howdify's existing tenant base (WLG manufacturing, Pure Shaka retail, Cosmic Reserve regulated retail). UC-04 through UC-06 are net-new verticals where the audit-chain primitive opens doors that commodity hybrid stacks cannot.
The architecture has two views. The production view is the future state with real hardware and real cellular transport. The Lab POC view is what ships in 2 weeks, with Python emulators replacing the physical Tier 0 and Tier 1 devices.
┌─────────────────────────────────────────────────────────────────────┐ │ HOWDIFY LAB · account 348117807165 · us-east-2 │ │ │ │ ┌──────────────────────┐ │ │ │ tier0_emulator.py │ Python sensor emulator │ │ │ • Reads wav/img file │ • Splits to 240B chunks │ │ │ • Emits sensor evts │ • HMAC-signs every chunk per §6 │ │ └──────────┬───────────┘ │ │ │ local WebSocket (mimics BLE transport) │ │ ▼ │ │ ┌──────────────────────┐ │ │ │ tier1_emulator.py │ Python orchestrator emulator │ │ │ • Receives + verifies│ • Intent classifier (Bedrock Haiku stub)│ │ │ • Reassembles payload│ • Routes to substrate or stays local │ │ │ • Forwards receipts │ • Streams tokens back to Tier 0 console │ │ └──────────┬───────────┘ │ │ │ in-VPC WebSocket (mimics cellular transport) │ │ ▼ │ │ ┌──────────────────────┐ │ │ │ halo-lab-substrate/ │ FastAPI + KTransformers + Qwen3 │ │ │ on r7iz.4xlarge │ • HMAC verifies Tier 0 + Tier 1 chain │ │ │ (stopped when idle) │ • Signs every expert load + token │ │ └──────────┬───────────┘ │ │ │ │ │ ▼ │ │ ┌──────────────────────┐ │ │ │ howdify-receipts DDB │ Existing table, new receipt_kinds: │ │ │ │ halo_sensor_capture, halo_tier_routing, │ │ │ │ halo_expert_load, halo_token_emit, │ │ │ │ halo_tier_response │ │ └──────────────────────┘ │ │ │ │ ┌──────────────────────┐ │ │ │ halo-receipts/ │ Verification CLI + audit dashboard │ │ │ cli + dashboard │ • Replays any session_id │ │ │ │ • Visualizes chain integrity │ │ └──────────────────────┘ │ └─────────────────────────────────────────────────────────────────────┘
No BLE. No cellular. No AICore. No physical glasses. Just the protocols, the substrate, and the audit chain, all inside Lab.
┌─────────────────────────────────────────────────────────────────────┐
│ TIER 0 · Halo open-source glasses │
│ Alif B1 NPU firmware. VAD-triggered audio (dual mic, beamforming). │
│ Gesture-triggered 640x480 optical capture for local VLM inference. │
│ HMAC-signed BLE chunks. Halo Display + stereo speaker response. │
└────────────────────────────┬────────────────────────────────────────┘
│ BLE GATT
▼
┌─────────────────────────────────────────────────────────────────────┐
│ TIER 1 · Android orchestrator │
│ Kotlin app, Firebase AI Logic SDK, Gemini Nano via AICore │
│ Custom escalation router, BLE GATT client, receipt forwarder │
└────────────────────────────┬────────────────────────────────────────┘
│ Cellular 5G / LTE
▼
┌─────────────────────────────────────────────────────────────────────┐
│ TIER 2 · Sovereign substrate │
│ AWS us-east-2 DEDICATED tier VPC │
│ KTransformers + SGLang + Qwen3-Next-80B-A3B on r7iz/r8i │
│ Per-tenant context retrieval, receipt verification + signing │
└─────────────────────────────────────────────────────────────────────┘
The wire format and receipt schema do NOT change between POC and production. Only the transport changes (WebSocket → BLE GATT for Tier 0, in-VPC WebSocket → cellular WebSocket for Tier 1). When the open-source glasses arrive, the firmware swaps in for tier0_emulator.py and everything else continues to work.
The Tier 0 device is the Halo open-source glasses (Sensor Edge column in the pipeline diagram above). The POC ships with a Python emulator standing in for the device, but the wire protocol and receipt schema below are designed so the firmware integration when the glasses arrive is mechanical, not architectural. Each labeled component on the glasses maps to a specific message type or receipt field.
The interaction model is multimodal: voice and gesture in, audio and peripheral visual display out. Notably, the frame includes a low-power 640x480 (VGA) optical sensor tied directly to the Alif B1 processor. It is architected strictly for local AI inference, with no native user photo storage or video recording pathway, making it highly defensible for sensitive compliance environments.
| Halo component | Function in HALO architecture | Wire protocol mapping |
|---|---|---|
| Optical Sensor | Low-power 640x480 (VGA) visual frame capture on gesture trigger. Feeds local VLM inference on the Alif B1 or gets tokenized for upstream validation. No native photo or video storage on the device. | 0x11 IMAGE_CHUNK (Tier 0 → Tier 1, packetized into 240B frames) |
| Gesture Sensor | Operator input trigger (tap, double-tap, swipe). Starts a session, confirms an action, or dismisses a response. | 0x12 GESTURE_EVENT (Tier 0 → Tier 1, 1B gesture code + timestamp) |
| Mic × 2 | VAD-triggered audio capture (operator voice query, ambient context). Stereo input enables beamforming for noisy environments. | 0x10 AUDIO_CHUNK (Tier 0 → Tier 1, audio payload in 240B chunks) |
| AI Processor (Alif B1) | On-device signal pre-processing, VAD detection, local VLM inference for visual context, HMAC signing of every outgoing frame per §07 | signs every AUDIO_CHUNK, IMAGE_CHUNK, GESTURE_EVENT, and SENSOR_END frame |
| Halo Display | Lens-projected visual response, token-by-token rendering | 0x70 DISPLAY_RENDER (Tier 1 → Tier 0 downlink, visual) |
| Speaker × 2 | Audio response output (synthesized voice or audio cue). Stereo allows spatialized feedback. | 0x71 AUDIO_RENDER (Tier 1 → Tier 0 downlink, audio) |
| Battery | Power budget monitor, auto-throttle at <20% or thermal threshold | emits 0x50 BACKPRESSURE when capacity-constrained |
| PCB | Main board carrying the AI Processor, sensors, BLE radio, and audio path | physical substrate, not protocol-visible |
| Lens | Display optics with see-through prescription compatibility. The Halo Display projects onto the lens. | display surface, not protocol-visible |
The HMAC signing happens on the AI Processor before any bytes leave the glasses. This is the cryptographic root of the Tier 0 chain. Every chunk emitted over BLE carries an HMAC signature derived from the per-session key (full HKDF chain in §07).
Transport-agnostic by design. Same frame format whether the underlying carrier is WebSocket, BLE GATT, or cellular.
Frame format (240B max payload, fits BLE MTU when hardware lands): ┌─────┬─────┬──────────┬─────────────┬─────────────┐ │ ver │ typ │ session │ payload │ hmac-sha256 │ │ 1B │ 1B │ 4B + 4B │ ≤ 198B │ 32B │ └─────┴─────┴──────────┴─────────────┴─────────────┘ Message types (0x1X = Tier 0 sensory inputs, 0x3X+ = orchestration): 0x10 AUDIO_CHUNK Tier 0 → Tier 1, audio bytes from mic 0x11 IMAGE_CHUNK Tier 0 → Tier 1, sequential VGA frame bytes 0x12 GESTURE_EVENT Tier 0 → Tier 1, tap / double-tap / swipe 0x20 SENSOR_END Tier 0 → Tier 1, capture complete (audio or image) 0x30 TIER_ROUTING Tier 1 internal, classification decision 0x40 ACK Tier 1 → Tier 0, chunk N received 0x50 BACKPRESSURE Tier 1 → Tier 0, pause sensor stream 0x60 ESCALATE_REQUEST Tier 1 → Tier 2, with full receipt bundle 0x61 ESCALATE_TOKEN Tier 2 → Tier 1, streamed token 0x62 ESCALATE_END Tier 2 → Tier 1, response complete 0x70 DISPLAY_RENDER Tier 1 → Tier 0, token for Halo Display 0x71 AUDIO_RENDER Tier 1 → Tier 0, synthesized audio for speakers 0x80 VERIFICATION_FAILED Tier 1 → Tier 0, parent hash or signature validation failed 0x81 SESSION_RESET Tier 1 → Tier 0, command to purge keys and force re-bootstrap
| Tier hop | POC transport (Lab) | Production transport |
|---|---|---|
| Tier 0 → Tier 1 | local WebSocket (ws://) | BLE GATT, 240B chunks |
| Tier 1 → Tier 2 | in-VPC WebSocket (wss://) | Cellular 5G/LTE WebSocket |
| Tier 2 → Tier 1 | in-VPC WebSocket reply | Cellular streamed tokens |
| Tier 1 → Tier 0 | WebSocket reply | BLE GATT writes |
Every event in every tier produces a signed receipt with this shape:
{
"receipt_id": "uuid-v4",
"session_id": "uuid-v4",
"session_epoch": uint32,
"tier": 0 | 1 | 2,
"tenant_id": "string",
"receipt_kind": "halo_sensor_capture | halo_tier_routing |
halo_expert_load | halo_token_emit |
halo_tier_response",
"event_payload_hash": "sha256-hex",
"parent_receipt_id": "uuid-v4 | null",
"parent_receipt_hash": "sha256-hex | null",
"timestamp_utc": "iso8601",
"hmac_signature": "hex"
}
HMAC computed over canonical JSON of the receipt minus the hmac_signature field. The canonical JSON is also stored alongside as hmac_canonical in DynamoDB so verification does not depend on JSON formatting reconstruction (existing Howdify pattern from the receipt-writer bug fix in April 2026).
All HALO keys derive from the existing howdify/receipt-hmac-secret SSM parameter. Standing Rule 16 (single source of truth for receipt secrets) is preserved. No parallel key infrastructure.
# Root (existing, shared across Howdify) ROOT_SECRET = ssm://howdify/receipt-hmac-secret # Per-tenant key (rotation: monthly, auto-derived) TENANT_KEY[tenant_id, epoch] = HKDF( ikm = ROOT_SECRET, salt = "halo-tenant-v1", info = tenant_id || epoch_month, L = 32 ) # Per-session key (ephemeral, session-scoped) SESSION_KEY[session_id, tenant_id, epoch] = HKDF( ikm = TENANT_KEY[tenant_id, epoch], salt = "halo-session-v1", info = session_id || session_epoch, L = 32 ) # Per-tier session subkey (one per tier within a session) TIER_KEY[tier, session_id, tenant_id, epoch] = HKDF( ikm = SESSION_KEY[...], salt = "halo-tier-v1", info = "tier-" || tier_number, L = 32 )
session lifecycle: T0 halo_sensor_capture parent: null (root of session) T0 halo_sensor_capture parent: previous T0 capture ... T1 halo_tier_routing parent: most recent T0 capture T2 halo_expert_load parent: triggering T1 routing receipt T2 halo_expert_load parent: previous T2 expert_load T2 halo_token_emit parent: preceding T2 expert_load(s) T2 halo_tier_response parent: final T2 token_emit T1 halo_tier_response parent: T2 tier_response
Any tampering at any tier breaks the chain at the next signed parent reference. The halo-verify CLI prints the exact tier, receipt_id, and field where the chain failed.
Detection alone is not enough. A single dropped packet or corrupted bit, malicious or accidental, must not leave the glasses hanging or permanently desynced from the parent ledger. The protocol carries an active mitigation pathway: when Tier 1 receives a frame sequence and the parent hash validation fails, Tier 1 drops the execution pipeline immediately and sends a 0x80 VERIFICATION_FAILED frame back to Tier 0 instead of writing a corrupt receipt row.
Upon receiving 0x80, Tier 0 must:
0x81 SESSION_RESET sequence to regenerate session keys via the HKDF runtime pipeline.The Lab POC proves this loop end-to-end with a deliberate tamper injection (see WP-D in §08). The emulator path:
REAL-TIME MITIGATION FLOW (tamper injection test):
1. Tamper trigger
────────────────────────
Operator clicks "Inject Tamper Byte" in the audit dashboard.
tier0_emulator.py mutates one byte mid-stream inside a sequence
of 0x11 IMAGE_CHUNK payload frames.
2. Tier 1 verification fails
────────────────────────
tier1_emulator.py computes SHA-256 of the payload and compares
against the sequence ledger. Hash mismatch. Bedrock Haiku local
routing halts. NO corrupt row is written to howdify-receipts.
3. Fault alert downlink
────────────────────────
tier1_emulator.py discards the packet block and emits
0x80 VERIFICATION_FAILED back to Tier 0 over the local WebSocket.
4. Tier 0 state purge + visual alert
────────────────────────
tier0_emulator.py catches 0x80. Instantly purges TIER_KEY[0]
from local memory. Renders "SECURITY ALERT - CHAIN BROKEN"
on the console UI (will be a lens-overlay on real hardware).
Refuses to transmit any subsequent sensory chunks until a
clean 0x81 SESSION_RESET handshake completes.
5. Clean recovery
────────────────────────
Tier 1 sends 0x81 SESSION_RESET. Tier 0 re-bootstraps via
HKDF, derives fresh TIER_KEY[0], renders "READY TO RE-LINK",
and resumes capture. The audit chain shows a deterministic
break + re-bootstrap event, not silent data loss.
This is the difference between an audit chain that logs tampering and one that actively defends against it. Sovereign Tech Post 3 shows the dashboard mid-tamper: the chain breaks visibly, the glasses go dark, and the system recovers cleanly without operator intervention.
halo-receipts/ shared Python library implementing the schema + HKDF derivationhalo-receipts/cli/ verification tool (halo-verify <session_id>)halo-receipts/dashboard/ HTML audit visualizationhalo-receipts/spec.md the protocol spec, locked at end of POC week 2/v1/halo/stream on a Lab-private subnethowdify-receipts table with new receipt_kind valuestier0_emulator.py reads wav or VGA jpeg as sensor input, chunks at 240B, HMAC-signs per §07, emits over local WebSockettier1_emulator.py receives + verifies + reassembles, runs Bedrock Haiku as intent classifier stub, routes locally OR escalates to Tier 20x80 VERIFICATION_FAILED exception listener inside tier0_emulator.py. On receipt, halt packet generation, flush the local mock memory key storage, and render the "CHAIN ERROR" alert. Implement 0x81 SESSION_RESET handler to re-bootstrap cleanly.howdify-receipts, and successfully forces Tier 0 into automated recovery state via the 0x80 → 0x81 mitigation loop.halo-verify <session_id>)| Resource | Provisioning | State |
|---|---|---|
| AWS account | 348117807165 | existing |
| Region | us-east-2 | existing |
| VPC | Howdify Lab VPC (to be confirmed) | operator to specify or stand up new |
| Substrate compute | r7iz.4xlarge baseline (or r8i.8xlarge per Spike) | on-demand, auto-stop |
| Storage | 4TB gp3 EBS for model weights | persistent |
| Receipt store | howdify-receipts DynamoDB | existing, new receipt_kinds added |
| HMAC root | /howdify/receipt-hmac-secret SSM SecureString | existing |
| Tier 1 intent classifier | Bedrock Haiku global.anthropic.claude-haiku-4-5 | existing |
| Audit dashboard hosting | S3 + CloudFront E3UQDOQ06ROVSA | existing |
Two gates must clear before WP-A through WP-D start.
halo-lab-vpcPRE-KICKOFF (must complete before Day 1): ├─ Gate A: Benchmark spike on r7iz └─ Gate B: Lab VPC choice confirmed Week 1: Day 1 - Benchmark spike on r7iz (or r8i if needed) Day 2-3 - WP-A receipt library + HKDF key derivation Day 4-5 - WP-B substrate FastAPI + KTransformers + Qwen3 Day 5 - WP-A spec locked Week 2: Day 6-8 - WP-C tier0 + tier1 emulators, end-to-end loop Day 9 - WP-D audit dashboard + verify CLI Day 10 - Tamper test + bug fixes Day 11+ - Sovereign Tech Post 3 drafted using audit dashboard as visual
| Resource | Unit cost | POC monthly estimate |
|---|---|---|
| r7iz.4xlarge (Lab substrate, ~50hr/wk active) | $1.35/hr | ~$268/mo |
| r8i.8xlarge fallback (if benchmark requires) | $2.10/hr | ~$420/mo |
| 4TB gp3 EBS | $0.08/GB-mo | $328/mo |
| Bedrock Haiku (Tier 1 classifier) | $0.80/1M tokens | ~$5/mo at POC scale |
| DynamoDB receipts | PAY_PER_REQUEST | marginal |
| POC baseline | ~$350/mo | |
| POC worst case (r8i + 24/7) | ~$1,500/mo |
Order of magnitude cheaper than the v2 production-scoped budget. Matches Lab experimentation budget shape.
When the Halo open-source glasses arrive:
halo-receipts/spec.md (locked at end of week 2)tier1_emulator.py (now talking to real glasses instead of a fake Tier 0) for the first integration testtier1_emulator.py to a real Android Kotlin app, or hold it as a paired Linux/Mac client if the glasses ship with their own pairing patternBecause the wire format and receipt schema are locked at POC time, hardware integration is mechanical, not architectural.
None of these block the POC. None of these block Sovereign Tech Post 3.
| Version | Scope | Trigger to pivot |
|---|---|---|
| v1 | 4-week production push, full physical-hardware build with Brilliant Labs Frame + Pixel 9 Pro + cellular + DEDICATED tier substrate + WLG demo video | Solo capacity review showed timeline was 6-8 weeks realistic, not 4. |
| v2 | Same 6 WPs as v1 but with hard pre-kickoff gates (Firebase AI Logic spike + KTransformers spike), HKDF key hierarchy spec, Standing Rule 8 carve-out, warm/cold TTFT split, solo execution acknowledged | WLG demo dropped, physical hardware deferred until Halo open-source glasses arrive. Lab POC becomes the right scope. |
| v3 | 2-week Lab POC. Python emulators replace Tier 0 + Tier 1 hardware. Substrate, receipt chain, protocol spec all unchanged. WP count drops 6 → 4. | Current. |
Receipt chain first. Hardware second. Demos third.