Technical spike reports and feasibility assessments on local-first data
architectures, multi-agent systems, and policy-as-code.
I conduct hands-on research spikes evaluating the practical viability, developer
ergonomics, and operational trade-offs of emerging tools and frameworks before
recommending them for production systems.
Active & Completed Spike Reports
Every spike closes with a verdict, not a summary. Read the table for where
things landed; the sections below carry the objective and the evidence.
Spike
Area
Verdict
Local-first analytical lakehouses
Data engineering
Adopt immediately — for dev, ingestion & analytics
Deterministic multi-agent choreography
Applied AI
Adopt with guardrails — cyclic graphs require budget limits
Policy-as-code guardrails for IaC
Cloud platform
Mandatory — on every multi-account landing zone
Self-healing observability loops
Platform operations
Evaluating — strong for assistance, not yet unattended
Local-first LLM serving & routing
AI infrastructure
Adopt with guardrails — local for utility tasks, APIs as fallback
Agent evaluation & LLM-as-a-judge
AI evaluation
Mandatory — gates every agentic workflow (ADR-004)
Local-First Analytical Lakehouses
Objective
Assess the viability of zero-infrastructure, low-cost analytical prototyping using DuckDB, Polars, and the Apache Iceberg v3 REST catalog.
Findings
Prototyping data transformations locally with DuckDB and Polars reduces compute costs during development by up to ~90% in internal benchmarks compared to cloud SQL engines. Apache Iceberg v3 — production-ready since Iceberg 1.11, with deletion vectors, row lineage, and the variant type — integrates through the REST catalog (Apache Polaris, Lakekeeper) and exports to enterprise cloud lakehouses (BigQuery, Snowflake) with zero code changes. DuckLake 1.0 was evaluated as the lighter, SQL-catalog alternative for single-node work; the Iceberg REST catalog is retained for cross-engine portability.
When not to use
As a replacement for a governed cloud warehouse serving many concurrent teams — the local-first path covers development, ingestion and lightweight analytics, while distributed-scale workloads and org-wide governance stay on the cloud engines it exports to.
Verdict
Adopt immediately for development environments, local ingestion pipelines, and lightweight analytical platforms.
Deterministic Multi-Agent Choreography
Objective
Evaluate the trade-offs of cyclic, stateful multi-agent systems built with LangGraph, Pydantic AI, remote MCP servers, and A2A delegation against linear chains.
Findings
Cyclic multi-agent graphs dramatically improve success rates on complex reasoning tasks through self-correction loops. Integrating remote Model Context Protocol (MCP) servers over Streamable HTTP decouples tool execution from agent runtimes; the 2026-07 specification (stateless core, OAuth-aligned authorization, formal extensions) and A2A v1.0 under the Linux Foundation split the concerns cleanly — MCP for tools and context, A2A for delegation between agents on different runtimes. Pydantic AI schemas and OpenTelemetry tracing prevent runaway loops.
When not to use
When a linear chain or a single structured call already solves the task — cyclic graphs add latency, token cost and failure surface, so they must earn their place on genuinely non-linear problems.
Verdict
Adopt with guardrails. Implement comprehensive runtime observability (Langfuse/OpenTelemetry) and token billing limits.
Policy-as-Code Guardrails for IaC
Objective
Test OPA Rego and Conftest to automatically enforce cloud security policies within ephemeral CI/CD pipelines.
Findings
Running Conftest validation on Terraform or OpenTofu plans before execution prevents security misconfigurations (like public S3 buckets or open GKE clusters) before they are provisioned, reducing auditing overhead and shifting security far left. Kyverno covers the admission side on Kubernetes, and OpenTofu's native state encryption closes a class of plan-time secret exposure that Terraform still leaves to the backend.
When not to use
Plan-time validation gates what Terraform proposes, not what changes out-of-band in a console — it complements rather than replaces runtime drift detection, and skipping it is defensible only in throwaway single-account sandboxes.
Verdict
Mandatory adoption for all multi-account landing zones and enterprise infrastructures.
Self-Healing Observability Loops
Objective
Build a prototype using lightweight, local open-weight models (Hermes 4, Qwen3.5, gpt-oss-20b) to parse pipeline logs, diagnose failures, and auto-propose PRs.
Findings
Local open-weight models — hybrid-reasoning Hermes 4, Qwen3.5, and gpt-oss-20b on a single GPU — excel at identifying standard formatting, query, and timeout issues. Running these loops on-premises preserves data privacy and limits public API exposure while automating repetitive debugging tasks.
When not to use
Unattended remediation in production — the loops are assistants, so auto-generated fixes merge only through human review, and incidents whose root cause spans systems outside the logs the model sees still need an engineer first.
Verdict
Evaluating. Excellent for developer assistance and dev-only environments, but human-in-the-loop review is mandatory before merging auto-PRs.
Local-First LLM Serving & Inference Routing
Objective
Evaluate local-first serving and routing of open-weight models (SGLang, vLLM, TensorRT-LLM, llama.cpp / Ollama) against enterprise LLM APIs — disaggregated prefill/decode over NVIDIA Dynamo and NIXL, KV prefix reuse tiered with LMCache, EAGLE-3 speculative decoding, and cost-aware routing.
Findings
Local serving with prompt caching and prefix reuse makes high-frequency, well-scoped tasks (classification, extraction, log parsing) drastically cheaper than metered APIs — the local-routing pattern validated in production client work. Disaggregation has moved from research into production stacks (Dynamo 1.0, with NIXL integrated into vLLM, SGLang, and TensorRT-LLM), and the open-weight ladder (gpt-oss-120b/20b, Qwen3.5, Gemma 3, DeepSeek V4) now covers every utility tier on a single GPU. Open-weight quality still trails frontier APIs on hard reasoning, so the router treats enterprise endpoints as the escalation path, and continuous TTFT/ITL and cost telemetry (OpenTelemetry GenAI conventions, Helicone) is a precondition, not an add-on.
When not to use
Frontier-grade reasoning where open-weight models still lag, and teams without GPU capacity or the bandwidth to operate a serving stack — start API-first and add local serving when volume justifies it.
Verdict
Adopt with guardrails. Route high-frequency utility workloads locally; keep enterprise APIs as the fallback behind circuit breakers, token budgets and cost telemetry. Continued as inference-fabric in Active Labs.
Agent Evaluation & LLM-as-a-Judge
Objective
Test whether LLM-as-a-judge pipelines (DeepEval, Promptfoo, Braintrust, Phoenix, LangSmith) can gate agentic workflows the way tests gate code — regression detection, golden datasets, synthetic test generation and consensus scoring.
Findings
A single judge model is noisy and biased toward its own phrasing; multi-evaluator consensus with pairwise ranking stabilizes scores enough to catch real regressions. Golden datasets plus synthetic generation keep coverage growing with the system, and wiring evaluation into CI turns prompt and graph changes into reviewable diffs instead of judgment calls. Public agent benchmarks (τ²-bench, Terminal-Bench, SWE-bench Verified) anchor the internal suites, so a regression can be told apart from a moving target.
When not to use
As the only gate for high-stakes correctness — consensus reduces judge bias but does not eliminate correlated failure, so deterministic assertions and human review stay in the loop for anything user-facing.
Verdict
Mandatory before any agentic workflow ships (ADR-004). Completed and folded into the engineering baseline as eval-fabric in Active Labs.