Tech Radar

Local analytics, Iceberg v3 and DuckLake

Iceberg v3 introduces format capabilities including deletion vectors, row lineage and additional types. A format revision is not a promise that every engine can read and write every feature. DuckLake reached v1.0 in April 2026 with SQL-backed metadata and support for shared access through a PostgreSQL catalog; calling it only a single-node catalog misses that distinction. Iceberg specification, DuckLake v1.0 announcement. DuckLabs announced on 2026-08-26 that it will become an AWS subsidiary; DuckDB and DuckLake stay MIT-licensed under the DuckDB Foundation, and a DuckDB 2.0 alpha followed on 2026-09-02 with a stable release planned for late October. DuckLabs and AWS, DuckDB 2.0 alpha.

My position: Start local development with DuckDB/Polars and Parquet. Choose Iceberg when its engine ecosystem is the requirement; trial DuckLake when its catalog model simplifies the intended deployment.

Next test: Run the same insert/delete/schema-change workload through every intended engine, then restore from backup. Record decimal and timestamp semantics, snapshot behavior, concurrent commits and permissions. Reject a migration that requires silent data coercion or an unsupported writer.

dbt v2 and Fusion

The June 2026 licensing update describes two distributions sharing a Rust engine: the Apache-2.0 dbt-oss runtime and dbt Labs' dbt build, which adds static analysis, LSP features and dbt lint under a product license. The old equation “Fusion engine equals a proprietary core” is no longer accurate. dbt v2.0 reached general availability on 2026-09-14 (2.0.8 by 2026-10-14) with five GA adapters; Spark and ClickHouse are still in beta, and v1 remains fully supported as 1.12.x. Sharing an engine does not guarantee compatibility between distributions. dbt licensing FAQ.

My position: Trial against an existing project before changing its production runner. Compare adapter support, custom materializations, Python models, macros, artifact consumers and CI behavior. Keep the previous runtime available until output equivalence and rollback are demonstrated.

Streaming lakehouses

Fluss documents tiering into lake storage and a Flink-based service connecting fresh data with historical Iceberg snapshots. Supported integrations and constraints differ between the released and development documentation. Fluss lakehouse storage, Iceberg integration.

My position: Evaluate against the simpler CDC → durable log → lakehouse path. Require event-time correctness, backpressure, duplicate handling, schema-change recovery and a measured freshness target. A new stateful tier must earn its storage and operational cost.

Analytical serving and orchestration

ClickHouse 26.8 introduces pipelined SQL, a way to express transformations in stages. This is a dialect extension, not a guarantee of faster queries. Kafka/Flink process events; Airflow coordinates tasks: they do not solve the same workload.

My position: Trial Kafka → Flink → ClickHouse when fresh queries justify a serving engine. Keep batch over Parquet/DuckDB as the baseline. Measure late events, duplicates, lag, replay and rebuilding the destination, including on-call effort. The medallion pattern separates received, validated and business-ready data; it does not require three physical copies when they add no value.

Inference: optimize useful work

Disaggregated prefill/decode, KV-aware routing, prefix reuse and speculative decoding address different bottlenecks. Separating workers can isolate long prefill from decode, but adds transfer and capacity coordination; speculative decoding depends on draft acceptance and compatible execution. Dynamo performance tuning.

My position: Start with one well-instrumented server or an API baseline. Evaluate one change at a time. Model selection comes from a versioned task set, license requirements and hardware constraints, rather than a permanently “best” model list.

Next test: Sweep arrival rate and concurrency using cold and warm caches. Report quality, TTFT, inter-token latency, p95/p99, rejected requests, queueing and cost per successful task. Check per-tenant cache isolation. A faster median with worse tail latency or quality does not pass the same acceptance gate.

MCP 2026-07-28 and A2A 1.0

MCP’s July release moves to a stateless protocol core, changes discovery and request handling, adds routing headers and hardens authorization. Application state still needs an explicit home. A2A 1.0 defines cross-agent communication; it serves a different boundary from tool access. MCP release notes, A2A 1.0 announcement.

My position: Pin a tested client/server pair and migrate deliberately. Do not carry an old transport-session design into a new protocol unchanged. Add A2A when ownership spans runtimes, and keep permissions explicit per tool.

Next test: Expired or wrong-audience credentials, identity mix-ups, tenant crossing, cancellation, duplicate requests and a restart around an approved side effect. A protocol or output schema alone cannot make model reasoning deterministic or prevent an execution loop.

Memory and assisted remediation

Persistent memory is useful only if it improves retrieval and remains governable. A retained wrong fact can repeatedly contaminate later tasks. For remediation, the first useful output is a bounded, reproducible diagnosis and a reviewable patch.

My position: Begin with explicit session context and lexical/vector retrieval. Trial graph memory when relationship queries justify extraction cost. Keep provenance, validity dates, tenant filters and deletion propagation. Test prompt injection through documents, tool results and memory. OWASP LLM application risks.

Next test: Compare with a memory-free baseline on held-out tasks, delete a source and verify it cannot return, then replay an incident in an isolated environment. Production remediation requires explicit permissions, validation and review before writes.

Evaluation: measure outcomes, calibrate judges

Agent evaluations need both task-level outcomes and operational failure tests. Code assertions, model rubrics and human review serve different purposes; model judges need calibration. The evaluation harness itself can introduce shared state or grading errors. Agent evaluation methodology.

My position: Use deterministic checks for permissions, schemas, data integrity and known outcomes. Use model judges for subjective dimensions with a documented rubric and human reference labels. Report failed trials and variance; track capability improvements separately from regression protection.

Next test: Hold out data by source and time, replay representative failures, and compare judges against human labels. Public benchmarks can supply task ideas; a leaderboard score does not establish fitness for a private workflow.

Platforms: schedule, route and recover

Kubernetes DRA allocates devices through claims and drivers; an inference gateway chooses endpoints using serving-specific information. These are separate control layers, with their own compatibility requirements. Kubernetes DRA, Gateway API Inference Extension.

My position: Trial when several teams share a serving fleet. Start from documented driver and gateway support, workload identity, admission controls, image provenance and a restore procedure. A single service may need much less.

Next test: Driver failure, drained GPU nodes, lost capacity, model rollback, noisy neighbors and denied admission. Policy checks on a Terraform/OpenTofu plan complement runtime drift detection; neither observes the whole system. See the production gates.

Observability and cost: version the contract

GenAI semantic conventions now have a dedicated OpenTelemetry repository. Pin the conventions and instrumentation together, and review stability at the individual signal level. FOCUS 1.4 defines cost-and-usage data and contract commitment relationships; it does not automatically map a bill to a successful agent task. GenAI conventions, FOCUS 1.4.

My position: Correlate request identity, model/route revision, tokens, latency, outcome and allocated cost. Redact prompts and tool payloads by default. Control metric cardinality, sampling, retention and telemetry expense.

Next test: Reconcile a representative bill with traces, including retries, cache hits, idle GPUs and egress. Enforce request budgets in execution code, and flag unattributed cost explicitly. Continue the experiments in Research Labs.