Technical glossary
Look up a concept, follow its connections, or try it in a local tool. Each definition includes an example, a limitation and a primary source.
A2A (Agent2Agent)
AI
An open protocol through which agents built on different frameworks discover each other's capabilities and exchange tasks and messages.
In practice. A planning agent reads a data agent's Agent Card and delegates a report as a tracked task.
Watch for. Interoperability does not establish trust: authentication, authorization and task limits remain the deployer's responsibility.
AI agent
AI
A system in which a model chooses tools or steps to pursue a task within configured boundaries.
In practice. Inspect an incident, request a permitted log lookup and propose the next step.
Watch for. Model output is not authority: software must enforce permissions and approval.
AIOps
Platform
Use AI to assist operations, such as alert correlation, incident investigation and remediation proposals.
In practice. Connect a latency alert with traces and a recent deployment for an operator to review.
Watch for. Correlation is not proof of cause; remediation needs scoped authority and validation.
Apache Airflow
Data
Coordinate workflows defined as code, including dependencies, schedules, retries and task state.
In practice. Run extraction, validation and reporting in dependency order and inspect failed tasks.
Watch for. It is an orchestrator, not a per-event streaming engine.
Apache Kafka
Data
A distributed event-streaming platform with partitioned logs that producers write and consumers can replay.
In practice. Publish order events so inventory and analytics can consume them independently.
Watch for. Ordering is per partition, not global; retention and delivery semantics need deliberate configuration.
Backfill
Data
Recomputing or loading data for a past period, for example after fixing a transformation or adding a column.
In practice. After correcting a currency bug, rerun the daily revenue model for the last 90 partitions.
Watch for. A backfill must be idempotent and bounded: overwrite whole partitions and limit its load on shared engines.
Backoff
Platform
Delay successive retries to reduce pressure on a failing dependency; jitter spreads synchronized clients.
In practice. Try again after progressively longer bounded delays, with a total retry budget.
Watch for. More retries can amplify an outage. Retry only eligible failures and make side effects safe.
Batch processing
Data
Process a bounded collection of data as a job, rather than treating it as an endless flow.
In practice. Calculate yesterday's sales from a closed set of orders.
Watch for. Scheduling frequency and job duration determine how long readers wait for new results.
Change data capture
Data
Turn database changes into events that downstream consumers can process incrementally.
In practice. Propagate inserts, updates and deletes from an operational database to an analytical system.
Watch for. Snapshots, replay, deletes and schema changes require explicit consumer behavior.
Circuit breaker
Platform
Temporarily stop calls after enough failures, then admit limited probes to check recovery.
In practice. Open a circuit during a persistent dependency outage and test it after a cooldown.
Watch for. It does not repair the dependency and can delay recovery if thresholds are poorly chosen.
ClickHouse
Data
A column-oriented SQL database designed for analytical queries over large volumes of data.
In practice. Aggregate event histories to power an operational dashboard.
Watch for. An analytical serving layer needs its own ingestion, retention and recovery design; it does not replace every transactional database.
Compaction
Data
Rewriting many small data files into fewer files near a target size, without changing the table's rows.
In practice. Five-minute streaming commits leave 11 MiB files; a nightly rewrite merges each hourly partition into one 512 MiB file.
Watch for. Compaction commits a new snapshot; the small files stay on storage until older snapshots expire.
Data lineage
Data
Recorded relationships between datasets, jobs and runs that explain where data came from and how it changed.
In practice. Trace a dashboard field back to the job and source dataset that produced it.
Watch for. Lineage coverage can be incomplete; a dependency graph does not prove data correctness.
DataOps
Data
Collaborative practices that improve delivery and operation of analytical data through automation, testing and feedback.
In practice. Validate a pipeline change with representative data before releasing it and observe quality afterwards.
Watch for. An orchestrator alone cannot replace ownership, quality criteria or communication with consumers.
dbt
Data
Build, test and document data transformations from versioned models and their dependencies.
In practice. Turn raw order tables into tested revenue models with traceable dependencies.
Watch for. dbt does not replace ingestion; check the v1 or v2 line, the open or product-licensed build and the adapter separately.
Dead-letter queue
Platform
A separate destination for messages that could not be processed within the configured attempt limit.
In practice. Quarantine an invalid event after repeated failures and inspect it before replay.
Watch for. Moving a message to a DLQ is not successful processing; retention and replay need an owner.
DevOps
Platform
Shared practices and responsibility across software delivery and operation, supported by automation and feedback.
In practice. The team that ships a service also uses incident feedback to improve its delivery process.
Watch for. Renaming a team or installing a CI server does not establish shared ownership.
DevSecOps
Platform
Integrate security practices and shared responsibility throughout software development, delivery and operation.
In practice. Review threat assumptions, verify dependencies and test authorization before releasing a change.
Watch for. A scanner passing does not establish secure design; exceptions, remediation and incident learning still need owners.
DuckDB
Data
An analytical SQL engine that can run inside an application and query files such as Parquet.
In practice. Explore a local dataset without deploying a separate database server.
Watch for. Local performance does not establish distributed or multi-user service capacity; preview features have separate limits.
Durable execution
Platform
Workflow execution whose progress is recorded, so it resumes after crashes, deployments or long waits without repeating completed steps.
In practice. An approval workflow waits three days for a person and, after a restart, continues at the next step.
Watch for. Steps with external side effects still need idempotency, and code changes must stay compatible with running histories.
Embedding
AI
A fixed-length vector that a model assigns to text, images or other data, so that similar meaning ends up close together.
In practice. Two support tickets about a failed payment get nearby vectors even when they share no words.
Watch for. Vectors from different models, versions or dimensions are not comparable; changing the model means re-embedding the corpus.
Read the source · Google · Machine Learning Crash Course: embeddings
Error budget
Platform
The unreliability an SLO allows over its window: one minus the target, applied to eligible requests or measured time.
In practice. A 99.9% target over 10 million monthly requests allows 10,000 failed requests.
Watch for. An exhausted budget is a signal to prioritize reliability work, not an automatic penalty; partial windows need care.
Evaluation
AI
A defined set of tasks and scoring rules used to inspect system behavior, including failures.
In practice. Check task completion and unauthorized actions separately on a held-out test set.
Watch for. A high aggregate score can hide dangerous failures; model judges also need calibration.
FinOps
Platform
Shared practices connecting technology spending, ownership and business value through engineering, finance and product collaboration.
In practice. Attribute a service's cost, review demand and compare cost per successful task.
Watch for. Reducing a bill without preserving needed quality, reliability or value can be a false saving.
GitOps
Platform
Continuous reconciliation of running systems against a versioned description of their intended state.
In practice. A controller restores a deployment after an unapproved configuration change.
Watch for. A pipeline that runs scripts from Git is not necessarily a reconciliation loop.
Hybrid attention
AI
A model design that interleaves full-attention layers with linear-attention layers whose recurrent state does not grow with context.
In practice. Qwen3.8 keeps a KV cache in one layer of every four; the other three hold a fixed state per request.
Watch for. The saving applies to the cache, not the weights; long-context recall and runtime support vary by model and engine.
Idempotency
Platform
Repeating the same operation has the same intended effect as performing it once.
In practice. Reuse an operation identifier when retrying an order creation after a timeout.
Watch for. The key, scope and retention window matter; a timeout does not prove the first attempt failed.
Infrastructure as code
Cloud
Infrastructure definitions stored as reviewable, versioned configuration that automation can apply.
In practice. Review a network change and its plan before applying it.
Watch for. Version control does not prevent drift or make a destructive plan safe.
KV cache
AI
Stored attention keys and values from prior tokens, reused during autoregressive decoding.
In practice. Reuse the prompt cache while producing each subsequent token.
Watch for. Memory grows with sequence length and concurrency; hybrid models cache only their full-attention layers, and layout depends on the runtime.
Lakehouse
Data
An analytical architecture combining lake-style storage with table management and warehouse-style capabilities.
In practice. Keep analytical tables in object storage and query them through a tested catalog and engine.
Watch for. Open files alone do not establish transactions, permissions or interoperability.
Landing zone
Cloud
A prepared multi-account or multi-project cloud environment with baseline identity, networking, logging and security guardrails for new workloads.
In practice. A new project inherits central logging, a shared network and organization policies when it is created.
Watch for. It is a starting baseline, not a finished security posture; its guardrails need owners, exceptions and change control.
LLMOps
AI
Lifecycle practices for releasing, evaluating and operating applications built around language models.
In practice. Version prompts and retrieval data, evaluate a release and observe its latency, cost and failures.
Watch for. A model benchmark does not validate the application's tools, permissions or real user tasks.
MCP
AI
A protocol through which AI applications discover and interact with tools, resources and prompts exposed by servers.
In practice. Expose a read-only catalog tool through a server with explicit access controls.
Watch for. Protocol compatibility does not grant permission or make tool output trustworthy.
Medallion architecture
Data
Organize data by refinement: bronze retains raw input, silver validates it and gold serves a use case.
In practice. Preserve incoming orders, validate identifiers, then publish a revenue summary.
Watch for. Layer names are a design convention, not proof of data quality or a requirement to copy every dataset three times.
Mixture of experts (MoE)
AI
A model layer that routes each token to a few expert sub-networks, so only part of the parameters computes each token.
In practice. Qwen3.8-Flash-Next holds 180B parameters but activates about 6B per token.
Watch for. Every expert stays in memory: the active share lowers compute and bandwidth per token, not resident weight memory.
MLOps
AI
Practices for delivering and operating ML systems across code, data, training, deployment and monitoring.
In practice. Version a model with its training data references, evaluation results and rollback artifact.
Watch for. Automating deployment does not ensure model quality or detect every change in production data.
Multi-agent systems
AI
Several agents cooperate with distinct responsibilities, context or permissions in one workflow.
In practice. A coordinator requests a report from a data agent and a reviewer checks its sources.
Watch for. Additional agents increase coordination, latency and failure paths; they do not guarantee better answers.
Object Lock (WORM)
Cloud
A storage setting that makes an object version immutable for a retention period: write once, read many.
In practice. Backups are written with a 30-day lock, so neither ransomware nor an operator mistake can delete them.
Watch for. A compliance-mode lock cannot be shortened, even by administrators; test retention on a non-production bucket first.
Object storage
Cloud
Storage that keeps whole objects with metadata in flat buckets, addressed by key through an HTTP API.
In practice. Keep Parquet files and table metadata in an S3-compatible bucket that several engines share.
Watch for. S3 compatibility varies by feature: conditional writes, versioning, object lock and listing behavior must be tested.
Observability
Platform
The ability to understand a system from the outside, through the traces, metrics and logs it emits, including problems nobody anticipated.
In practice. Follow a slow request's trace from the API to the query that caused the delay.
Watch for. More telemetry is not more understanding; cardinality, sampling and retention decide cost and which questions stay answerable.
OLAP
Data
Online analytical processing: queries that scan and aggregate many rows over few columns, usually served by columnar storage and vectorized execution.
In practice. Aggregate a year of events by region to refresh a dashboard.
Watch for. Fast scans do not make an engine good at frequent point updates, transactions or high-concurrency writes.
Open table format
Data
A specification that turns data files in object storage into tables with snapshots, schema and partition metadata that several engines can share.
In practice. Spark writes an Iceberg table that Trino and DuckDB query through the same catalog.
Watch for. Engines support different spec versions and features; sharing a format still requires a shared catalog and tested writers.
OpenTelemetry
Platform
An open standard, with SDKs and a collector, for producing traces, metrics and logs that share context and naming conventions, exported over OTLP.
In practice. A request's trace ID follows it from the API through the queue to the worker, so the one slow step shows up end to end.
Watch for. It standardizes how signals are collected, not where they are stored or alerted on; sampling and label cardinality still decide the cost.
Prefill and decode
AI
The two phases of LLM inference: prefill processes the prompt in parallel and is compute-bound; decode generates one token at a time and is memory-bound.
In practice. Serve long prompts on prefill workers and hand their KV cache to decode workers over a fast interconnect.
Watch for. Disaggregation adds KV transfer and scheduling cost; it pays off only for some prompt lengths, output lengths and latency targets.
Private cloud
Cloud
Cloud resources dedicated to one organization, whether operated on its premises or by a provider.
In practice. An internal platform offers self-service environments to several business units.
Watch for. Virtual machines alone do not establish a cloud operating model.
Quantization
AI
Represent selected model values with lower precision to reduce storage and potentially computation.
In practice. Evaluate 4-bit weights against the original model using a held-out workload.
Watch for. Smaller weights do not guarantee faster inference or unchanged quality; hardware support matters, and published checkpoints often keep sensitive layers in BF16.
Query federation
Data
Querying several data sources from one SQL engine through connectors, without first copying the data into that engine's storage.
In practice. Trino joins an Iceberg table with a PostgreSQL reference table in one query.
Watch for. Speed depends on what each connector can push down; federation still loads the sources and does not reconcile their consistency.
RAG
AI
Generation conditioned on information retrieved from an external collection, rather than model parameters alone.
In practice. Retrieve relevant policy passages and provide them as context for an answer.
Watch for. Retrieval can miss evidence; supplied context does not guarantee a supported answer.
Schema evolution
Data
Changing a table or message schema while stating which existing readers and writers keep working.
In practice. Adding an optional column keeps current readers working; dropping a required one breaks them.
Watch for. Compatibility in one direction does not imply the other; a rename is a removal plus an addition unless the format tracks column IDs.
Site reliability engineering (SRE)
Platform
Apply software engineering to operating services with explicit reliability goals and limited manual toil.
In practice. Use an error budget to decide whether to ship a change or address reliability first.
Watch for. Reliability targets reflect user needs and cost; a dashboard alone is not an SRE practice.
SLO
Platform
A target for a measured service indicator, over a defined population and time window.
In practice. At least 99.9% of valid requests succeed over a rolling 30-day window.
Watch for. A target is not a measurement or a contractual guarantee; exclusions must be explicit.
Streaming
Data
Process events as a continuing flow, with explicit rules for time, state and late arrivals.
In practice. Update available inventory as order events arrive.
Watch for. Low latency still requires replay, backpressure and duplicate handling; it does not mean instant or lossless.
Table catalog
Data
The service that maps a table name to its current metadata and makes each commit atomic, so every engine sees the same table state.
In practice. Spark writes analytics.orders and Trino reads it through the same REST catalog, so the new snapshot is visible to both.
Watch for. A shared bucket without a shared catalog is not a shared table: engines must speak the same catalog protocol and honor its permissions.
Read the source · Apache Iceberg · REST catalog specification
Table snapshot and time travel
Data
An immutable version of a table's state after a commit; a query can read an older snapshot by its ID or timestamp.
In practice. Compare today's revenue with the table as it was before last night's backfill.
Watch for. Old snapshots keep their files on storage until they expire, so retention is a cost and compliance decision.
Tail latency (p95/p99)
Platform
The response time that only the slowest few percent of requests exceed, such as the 99th percentile.
In practice. An API averages 80 ms, but its p99 is 1.2 s: one request in a hundred waits more than a second.
Watch for. Percentiles cannot be averaged across servers or time windows; aggregate the underlying histograms instead.
Vector search
AI
Retrieval that ranks items by the similarity of their embeddings, numeric vectors a model produces from text, images or other data.
In practice. Find policy passages close in meaning to a question even when they share no keywords.
Watch for. Approximate indexes trade recall for speed; exact identifiers, filters and freshness often need hybrid keyword search.
VPC
Cloud
A logically isolated virtual network in a cloud provider, with controlled addressing and connectivity.
In practice. Separate application subnets from a database subnet and define their routes.
Watch for. A VPC is a network boundary, not proof of complete workload isolation.
Watermark
Data
A stream-processing signal of event-time progress, used to decide when time-based operations can advance.
In practice. Close a window after its watermark passes the window end, with a policy for late events.
Watch for. A watermark is not a guarantee that no older event will ever arrive.
Zero trust
Cloud
A security model that authorizes each request from verified identity, device and context instead of trusting network location.
In practice. A service reaches a database only with a short-lived workload identity, even inside the same VPC.
Watch for. Zero trust is an architecture adopted incrementally, not a product; legacy paths often keep implicit trust.