Every field builds its own vocabulary, and data is no exception. The annoying part is that half these words get thrown around in meetings like everyone already agrees on what they mean, and usually nobody does. This is the glossary I wish someone had handed me early on: not textbook definitions, but what these terms actually mean when you run into them at work, with an example or a code snippet wherever one actually helps.

Here's the rough shape of the journey most data takes, worth keeping in the back of your mind as you read the rest of this:

flowchart LR
  A[Source Systems] --> B[Ingestion]
  B --> C[Data Lake / Warehouse]
  C --> D[Transform]
  D --> E[Served: dashboards, reports, apps]

The Basics

Data is just recorded facts: numbers, text, measurements, before anyone's made sense of them. A single one of those facts is technically a datum, the singular form nobody actually says out loud. "Data" has won as both singular and plural in everyday speech, and that battle isn't worth fighting.

Information is data that's been organized enough to answer a question. "47" is data. "Average order value this month is $47" is information. The distinction matters more than it sounds: a huge pile of raw data with no structure or context isn't useful to anyone yet, and a good chunk of data engineering is the work of turning one into the other.

Metadata is data about data: a file's size, when a table was last updated, who owns a dataset, what a column is supposed to contain. It sounds like a minor detail, but metadata is what makes data findable and trustworthy at any real scale. A data catalog (below) is basically a big, searchable pile of metadata.

Where Data Lives

Database: a system built to store and query data reliably, usually structured into tables with rows and columns. When people say "database" without qualifying it, they usually mean a relational database like Postgres or MySQL, built for transactional work: lots of small, fast reads and writes.

OLTP vs OLAP: OLTP (Online Transaction Processing) describes systems built for exactly that: fast, small transactions, like an app checking or updating one row. OLAP (Online Analytical Processing) describes systems built for the opposite: fewer, much bigger queries scanning huge amounts of data for analysis. A production app's database is OLTP. A data warehouse is OLAP. Using one for the other's job usually ends badly.

Data Warehouse: a database built specifically for analytics rather than day-to-day transactions. Data gets pulled in from source systems, structured, and stored in a way optimized for big aggregate queries (Snowflake, BigQuery and Redshift are all data warehouses). The tradeoff is that warehouses are usually built around a schema decided in advance.

Data Lake: a storage system that holds data in its raw, often unstructured or semi-structured form, files, logs, JSON blobs, without forcing a schema up front. The idea is to keep everything cheaply and figure out the structure later, once you actually know what you need it for. The tradeoff: without real discipline, a data lake turns into what people call a "data swamp," a pile of files nobody trusts or understands anymore.

Data Lakehouse: an attempt to get the best of both. Store data cheaply like a lake, but add the structure, versioning and query performance of a warehouse on top. Table formats like Apache Iceberg and Delta Lake exist specifically to make this possible.

Data Mart: a smaller, more focused slice of a data warehouse, usually built for one team. If the warehouse holds the whole company's data, a mart might just be "everything the marketing team needs," pre-joined and simplified for them.

Table Formats You'll Hear About

These matter more than people expect, because the format data is stored in determines how fast you can query it, whether you can update it safely, and whether ten people modifying the same table will step on each other.

Parquet: a columnar file format, meaning it stores data column by column instead of row by row. That sounds like a small detail, but it's a big deal for analytics: if you only need three columns out of fifty, a columnar format lets you skip reading the other forty-seven. Most modern lakes and warehouses use Parquet underneath.

Avro: another common format, row-based instead of columnar, which makes it a better fit for write-heavy or streaming use cases than for big analytical scans.

ORC: similar goal to Parquet, columnar and optimized for analytics, more common in the Hadoop/Hive world.

Apache Iceberg: a table format that sits on top of files like Parquet and adds things a plain folder of files can't do on its own: safe concurrent writes, schema evolution without rewriting everything, and time travel, querying the table as it looked at a specific point in the past.

-- Query a table as it existed at a specific snapshot
SELECT * FROM orders
FOR VERSION AS OF 8317998783848532421;

-- Or as of a specific timestamp
SELECT * FROM orders
FOR TIMESTAMP AS OF TIMESTAMP '2026-08-01 00:00:00';

That's the kind of thing that used to need a full backup system, and Iceberg just gives it to you as a query.

Delta Lake: Databricks' answer to the same problem. Similar goals to Iceberg: ACID transactions, schema enforcement, time travel, all on top of Parquet files.

Apache Hudi: a third option in the same category, historically stronger for upsert-heavy, incremental workloads.

You don't need to memorize the differences between these three. What matters is knowing they all exist to solve the same problem: making a folder of files behave like a real, safe, queryable table.

Getting Data From A to B

ETL (Extract, Transform, Load): pull data out of a source system, clean and reshape it, then load the finished version into its destination. The transformation happens before the data lands anywhere permanent.

ELT (Extract, Load, Transform): load the raw data in first, then transform it inside the destination, usually a warehouse, using its own compute. This got popular once warehouses became powerful and cheap enough to do the heavy lifting themselves, and it means the raw data is always sitting there if you ever need to reprocess it differently.

Reverse ETL: the opposite direction of the usual flow. Instead of pulling data into the warehouse, it pushes already-cleaned, already-modeled data back out into the tools people actually work in, syncing a customer's calculated lifetime value back into the CRM, for example, so a sales rep sees it without needing to query the warehouse themselves.

Data Pipeline: the general term for any automated flow that moves data from one place to another, whether that's a simple ETL job or a much more complex system with validation, branching logic and monitoring built in.

Data Ingestion: specifically the "getting data in" part of a pipeline, pulling from APIs, databases, files or event streams, whatever the source happens to be.

Data Migration: a one-time (or at least rare) move of data from one system to another, retiring a legacy database in favor of a new platform, for example. Different in character from a regular pipeline because it's usually a single big push rather than an ongoing flow, and rollback if something's wrong is a lot harder.

Batch Processing: handling data in chunks on a schedule, every hour, every night. Simple, predictable, and fine for most reporting use cases.

Stream Processing: handling data continuously, as it arrives, event by event, instead of waiting for a batch window. Worth the added complexity when the lag from batch processing is an actual problem: fraud detection, live dashboards.

CDC (Change Data Capture): a way of tracking exactly what changed in a source database, row by row, insert, update or delete, instead of re-pulling the whole table every time. This is usually what makes near-real-time pipelines feasible without hammering the source system.

// A simplified CDC event, showing what actually changed
{
  "operation": "UPDATE",
  "table": "customers",
  "before": { "id": 42, "email": "old@example.com" },
  "after":  { "id": 42, "email": "new@example.com" },
  "timestamp": "2026-08-29T10:15:03Z"
}

Structure and Modeling

Structured data: fits neatly into rows and columns with a defined schema, think a spreadsheet or a SQL table.

Semi-structured data: has some organization but no fixed schema. JSON and XML are the classic examples: the fields have names, but different records can carry different fields.

{"user_id": 101, "event": "login", "device": "mobile"}
{"user_id": 102, "event": "purchase", "item": "shoes", "amount": 59.99}

Same general shape, different fields per record. A rigid table would fight you here; semi-structured formats don't.

Unstructured data: no predictable structure at all: images, audio, free-text documents, PDFs.

Schema: the definition of what a dataset is supposed to look like: field names, types, which fields are required. A schema is basically a contract, whether or not anyone's written it down as one.

Star Schema / Snowflake Schema: two common ways of organizing a data warehouse. A star schema has one central fact table (the actual measurements, like individual sales) surrounded by dimension tables (context, like customer, product, date). A snowflake schema is the same idea, but the dimension tables are broken down further into their own related tables, more normalized, more joins, less duplication.

Normalization / Denormalization: normalizing means splitting data into separate related tables to avoid storing the same information twice. Denormalizing means deliberately duplicating some of that data back together so queries run faster and simpler, at the cost of extra storage and the risk that the duplicated copies drift out of sync.

Partitioning: physically splitting a large table into smaller chunks based on a column, usually a date, so queries only scan the relevant chunk instead of the whole table.

CREATE TABLE events (
  event_id BIGINT,
  event_date DATE,
  payload STRING
)
PARTITIONED BY (event_date);

A query filtered to WHERE event_date = '2026-08-29' only touches that day's partition, not the table's entire history.

Partition Pruning: a query engine's ability to skip reading partitions that can't possibly match the query's filters. This is the actual performance payoff of partitioning done right, not the partitioning itself.

Sharding: a related idea, but for splitting data across separate database instances entirely (usually by something like customer ID) rather than splitting files within one system, done to scale storage and throughput horizontally.

Data Quality and Trust

Data Quality: whether a dataset is accurate, complete, consistent, timely and fit for the decision someone's about to make with it. I've gone deeper on this here.

Data Validation: the actual checks that enforce quality: null checks, range checks, format checks, uniqueness checks, run automatically rather than trusted on faith.

Data Profiling: examining a dataset to understand what's actually in it before building anything on top of it: value distributions, null rates, duplicate rates, unexpected formats. Profiling is how you find out a "phone number" column has values like "N/A" and "TBD" mixed in with real numbers.

Data Reconciliation: comparing two datasets that are supposed to agree, a source and a target, or two systems that both claim to know "the" customer count, and finding where they don't.

Data Lineage: the traceable path a piece of data took to get where it is: which source it came from, what transformations touched it along the way. Lineage is what lets you answer "why does this number look wrong" without guessing.

Master Data Management (MDM): the discipline of keeping one authoritative version of core business entities (customers, products, vendors) instead of letting every system maintain its own slightly different copy. This is the formal name for the problem I wrote about here.

Golden Record: the single trusted, deduplicated version of an entity that MDM work is supposed to produce, the answer to "which of these five customer records is the real one."

Data Contract: an explicit, agreed-upon definition of what a dataset should look like and how it's allowed to change, so a producer can't silently rename a field and break every consumer downstream without warning.

# A simple data contract for an events table
table: user_events
owner: growth-team
fields:
  - name: user_id
    type: integer
    required: true
  - name: event_type
    type: string
    required: true
    allowed_values: [login, signup, purchase, logout]
  - name: occurred_at
    type: timestamp
    required: true

Governance and Safety

Data Governance: the standards, ownership and controls that keep data quality durable over time, not a one-time cleanup. More on how this relates to quality specifically here.

Data Catalog: a searchable inventory of what datasets exist, what's in them, who owns them, and how trustworthy they are. Basically a phone book for an organization's data.

PII (Personally Identifiable Information): any data that can identify a specific person: name, email, phone number, government ID. Handling PII correctly usually isn't optional, it's a legal requirement.

Data Masking / Anonymization: replacing or obscuring sensitive values so data can still be used for testing or analysis without exposing the real underlying information.

Data Retention: how long data is kept before it's deleted or archived, driven by both practical cost and legal requirements.

Architecture Patterns You'll Hear About

Data Mesh: an organizational approach where individual teams own and publish their own data as a product, instead of one central team owning all data for the whole company. Solves a scaling problem, creates a governance problem, trades one set of headaches for a different one.

Data Product: treating a dataset as an actual product, with an owner, a defined interface (a schema or contract), and quality guarantees, rather than an accidental byproduct of whatever system happened to generate it. The idea underneath data mesh.

Data Fabric: an architecture focused on connecting and providing unified access across many different, often disconnected data sources, without necessarily moving all of it into one central place first.

Data Observability: monitoring the health of the data itself (freshness, volume, schema, distribution) the same way you'd monitor server uptime, catching problems before a business user does. I wrote about the specific failure mode this is meant to catch here.

A Few More Words That Come Up Constantly

Idempotent: running an operation multiple times produces the same result as running it once. This matters enormously in pipelines, because retries happen, and a pipeline step that isn't idempotent can silently double-count data the second time it runs.

# Not idempotent — running this twice doubles the total
balance += payment_amount

# Idempotent — running this twice has no extra effect
balances[payment_id] = payment_amount  # keyed by a unique payment ID

Immutable: data that, once written, is never changed, only ever replaced by a new version. A lot of modern data architecture leans toward immutability because it makes debugging and time travel dramatically simpler: you can always see exactly what existed at any point.

Schema Drift: when the structure of incoming data changes without warning, a new column appears, a field's type changes, and nothing was told about it in advance. One of the most common causes of a pipeline breaking silently.

Data Freshness / Latency: how current the data actually is by the time someone looks at it. A dashboard labeled "real-time" but built on data that's four hours stale isn't real-time, whatever the label says.

SLA (Service Level Agreement): a formal commitment about how fast or how reliable something needs to be, "this table will be updated by 6am daily," a documented version of "this needs to actually work, and here's how we'll know if it doesn't."

Orchestration: coordinating when and in what order a set of pipeline tasks run, handling dependencies, retries and failures. Tools like Airflow exist specifically for this.

DAG (Directed Acyclic Graph): the structure most orchestration tools use to represent a pipeline: a set of tasks with dependencies between them, and no loops, meaning nothing can depend on itself, directly or indirectly.

Compaction: merging many small files into fewer, larger ones. Small files pile up naturally in streaming and incremental systems, and too many of them slows every query down; compaction cleans that up periodically.

Why Any of This Matters

None of these words are interesting on their own. What actually matters is that a lot of expensive mistakes come from two people using the same word to mean different things: one person's "data lake" is another person's "unstructured mess," one person's "real-time" is another's "updated every few hours." Half of getting data right is technical. The other half is making sure everyone's actually talking about the same thing when they use these words.