What Apache Pinot is designed to do

Apache Pinot is an open-source distributed OLAP database originally developed at LinkedIn. It supports analytical queries over data ingested from streaming and batch sources and is commonly used for user-facing or operational analytics where freshness, latency, and concurrency matter.

Those terms are not universal performance guarantees. A design must specify the dataset, ingestion rate, freshness target, query shapes, result correctness, concurrency, availability, hardware, and cost envelope.

Read the active series in this order

  1. Advanced Apache Pinot: Optimizing Performance and Querying
    Start with the workload, schema, segment, index, and query-plan decisions that determine whether Pinot is a good fit.
  2. Designing Pinot with Apache Iceberg
    Separate Pinot's hot serving layer from an authoritative historical Iceberg table, reconcile the two, and define a non-overlapping query boundary.
  3. Custom Aggregations, Transformations, and Real-Time Enrichment
    Decide which transformations belong before ingestion, during Pinot ingestion, or at query time, and validate their correctness and cost.
  4. Apache Druid vs. Apache Pinot
    Use a measured workload and operational requirements—not a feature checklist—to evaluate the two systems.

Earlier introductory Datanizant posts remain historical archive material. Do not link them as current tutorials until their configurations, dates, and routes are migrated and revalidated.

A useful mental model

  • Real-time table: consumes a supported stream and makes completed or consuming segments queryable.
  • Offline table: loads prebuilt segments from batch data.
  • Hybrid table: combines real-time and offline tables under one logical name with a time boundary.
  • Broker: receives queries and routes work to servers.
  • Server: hosts segments and executes the routed query work.
  • Controller and ZooKeeper/Helix: coordinate table, segment, and cluster state.

Evidence-backed production example: Uber

Uber's engineering team reported in December 2024 that it had used Pinot for more than six years across many use cases. The article describes workloads ranging from real-time cases with more than one million writes per second, about 100 QPS, and latency below 500 ms to low-latency analytics over offline data. A later Uber article describes hundreds of critical use cases and the evolution of its query architecture.

These are valuable production observations, but they are measurements from Uber's data, infrastructure, query mix, and architecture. They should be treated as evidence that Pinot can support demanding workloads—not as expected results for a new cluster.

How to decide whether Pinot fits

  1. Model the workload: representative records, cardinality, retention, late data, mutations, query shapes, and tenant isolation.
  2. Set measurable objectives: source-to-queryable freshness, p50/p95/p99 latency, QPS/concurrency, availability, correctness, and cost.
  3. Build representative segments: test schema, partitioning, sorted columns, indexes, star-tree or other accelerators only where the workload benefits.
  4. Test failure behavior: broker/server loss, stream interruption, segment recovery, rebalance, rollout, and overload.
  5. Plan operations: ZooKeeper quorum, deep store, backups, monitoring, security, capacity, and upgrades.
  6. Compare alternatives: run the same acceptance workload against any shortlisted system.

Claims this series should not make without evidence

  • That a sample Helm installation is production-ready without environment-specific hardening.
  • That Pinot alone performs fraud detection, recommendations, forecasting, or business decision-making.
  • That a named company achieved a metric or outcome unless the company's own technical publication documents it.
  • That Iceberg is a native Pinot archive sink or that Pinot deep store is an Iceberg table.
  • That “sub-second” is guaranteed independently of query, schema, scale, concurrency, and infrastructure.

Primary sources