Apache Pinot performance tuning starts with a measured query workload. An index is not a universal speed switch: its value depends on filters, grouping columns, aggregations, time ranges, concurrency, freshness, and latency objectives. Validate configuration against the exact Pinot release you deploy.

Define the workload and row grain

Record normalized queries, owners, traffic share, peak concurrency, time interval, expected result size, latency percentiles, errors, documents scanned, freshness, and availability requirements. Define one row grain and enforce producer contracts for identifiers, dimensions, metrics, timestamps, nulls, late events, and duplicates.

Changing a schema or table name requires dual-write or backfill, validation, cutover, and rollback. It is not a tuning shortcut.

Configure streaming ingestion for the deployed version

Current Pinot configuration uses ingestionConfig.streamIngestionConfig.streamConfigMaps. Choose a Kafka consumer factory and decoder supported by the release. Broker addresses, authentication, TLS, starting offsets, flush thresholds, segment size, and replication are environment decisions; do not copy sample values or embed secrets.

Replication balances availability, capacity, and cost. A single replica is not an “optimized” default. Test consumer lag, partition changes, restarts, replay, duplicates, and recovery.

Select indexes from predicates

  • Use inverted indexes for measured selective equality or IN filters.
  • Consider range indexes for bounded numeric predicates.
  • Consider timestamp indexes when recurring time-granularity queries justify generated columns and storage.
  • Use text, JSON, geospatial, or other indexes only for matching operations and documented constraints.
  • Test Star-Tree for stable aggregation and group-by families whose dimensions and functions can be enumerated.

Use Star-Tree only when queries match

Star-Tree stores pre-aggregated documents and can reduce work for compatible queries. It adds segment size and build cost; high-cardinality dimensions can expand it. A query’s filters, group-by columns, and aggregate function-column pairs must be covered. An index configured for SUM__likes does not necessarily serve an expression such as SUM(likes + shares).

Star-Tree is incompatible with Pinot upsert tables. Do not claim “instant” results without dataset, versions, hardware, concurrency, percentile, and before/after measurements.

Verify plans and segment state

Run the exact production SQL on the selected query engine and use EXPLAIN PLAN FOR and execution statistics to investigate index use. Time functions and query options can vary by version; explain output is diagnostic, not a stable application API.

  1. Benchmark a fixed representative snapshot.
  2. Apply one index change to an isolated or canary table.
  3. Reload or rebuild applicable existing segments; configuration changes alone do not retrofit them.
  4. Track asynchronous operation and segment health.
  5. Repeat the workload at expected concurrency and compare scans, latency, errors, CPU, memory, segment size, and ingestion cost.
  6. Promote only when acceptance criteria pass and rollback is tested.

Configure monitoring end to end

Starting empty Prometheus and Grafana containers does not monitor Pinot. Export component JMX metrics, configure authenticated scrape targets and retention, connect a datasource and reviewed dashboards, persist state, and secure access. Alert on symptoms tied to service objectives: broker latency and partial results; server ingestion lag, consumption health, query exceptions, heap and direct memory; controller segment availability, replicas, errors, and task failures.

Related implementation context is available in the Pinot series summary, advanced Pinot patterns, and production Pinot guide.

Originally published November 30, 2023; technically reviewed and substantially updated September 4, 2026.