GROUP BY is capped at 10,000,000 groups: undocumented, no configuration parameter, enforced only after a full scan (3.4.2.5)

[alexey-milovidov]

Version

• taosd version: 3.4.2.5.community (TDengine TSDB-OSS tarball)
• git: c15925333c9fe385902b153879b812c26bc612f7, build: Linux-arm64 2026-08-16 16:59:49
• Single dnode, stock taos.cfg, Ubuntu 24.04 aarch64

What happens

A query whose aggregation produces more than 10,000,000 groups is rejected with
Too many groups/time window in query [0x8000070A]
(TSDB_CODE_QRY_TOO_MANY_TIMEWINDOW). Ten million exactly is accepted, twelve
million is not, and I could not find either the limit or a configuration
parameter for it in the documentation — SHOW DNODE 1 VARIABLES has nothing
that changes it, and maxNumOfDistinctRes, which the 3.0 docs described for a
related cap, no longer exists.

Three things about it that were not obvious and cost us time to establish:

1. The ceiling is on the merged result, not on per-vnode partials. A
   supertable spread over 32 vgroups still refuses 15M distinct keys. And
   9.5M distinct keys assembled from 19M partial groups — each key present in
   two subtables, so the partials sum to well over the ceiling — is accepted.
   So sharding the data more finely does not help.
2. It is enforced during execution, not at planning time. A query that is
   going to be rejected scans first. On 100M rows we measured rejections after
   120 s, 221 s, 238 s, 390 s, 537 s and 546 s. Nothing is returned for that
   work.
3. Which limit you hit is not predictable. Some over-ceiling groupings run
   out of memory before reaching it and return Out of Memory [0x80000102] or
   Query memory exhausted [0x8000073A] instead.

Reproduction

CREATE DATABASE captest;
CREATE TABLE captest.t (ts TIMESTAMP, v BIGINT);

# n = 10000000 -> accepted;  n = 12000000 -> rejected
n = 12000000
with open('/tmp/ct.csv', 'w') as f:
    for i in range(n):
        f.write('%d,%d\n' % (1700000000000 + i, 1000000000000 + i * 7))

taos -d captest -s "INSERT INTO t FILE '/tmp/ct.csv'"

# n = 10,000,000
taos -d captest -s "SELECT count(*) FROM (SELECT v FROM t GROUP BY v)"
       count(*)        |
              10000000 |
Query OK, 1 row(s) in set (46.876693s)

# n = 12,000,000
taos -d captest -s "SELECT count(*) FROM (SELECT v FROM t GROUP BY v)"
DB error: Too many groups/time window in query [0x8000070A] (9.503482s)

The merged-vs-partial behaviour, with a supertable over 32 vgroups:

CREATE DATABASE captest2 VGROUPS 32;
CREATE STABLE captest2.s (ts TIMESTAMP, v BIGINT) TAGS (g INT);
-- 15,000,000 rows, v distinct, tbname = i % 96  -> rejected
-- 19,000,000 rows, v = i % 9500000, tbname = 'a' || (i / 9500000) % 32
--     i.e. 9.5M distinct keys, each in 2 subtables  -> accepted, 77.7s

Question / request

Is 10,000,000 an intended hard limit? If so it would help a great deal to have
it in the docs next to GROUP BY, and to have it enforced at planning time so
a doomed query fails in milliseconds instead of after ten minutes of scanning.
If it is meant to be tunable, we could not find the knob.

Context

We ran into this adding TDengine to
ClickBench, a 100M-row single-table
analytical benchmark. Six of its 43 queries group by high-cardinality keys
(UserID has 17.6M distinct values, URL 18.3M, (WatchID, ClientIP) ~100M)
and are refused by this ceiling; we record them as errors. That is a fair
outcome for a time-series engine on a workload it is not aimed at, and we say
so in the entry — but the ten-minute delay before the refusal, and the limit
not being documented, are the parts worth fixing.

[alexey-milovidov]

For context, the ClickBench entry this came out of is now open as ClickHouse/ClickBench#1704. It documents the workaround used for this and links back here, so anyone reading the benchmark entry can see the status of the underlying issue.

The other reports from the same exercise: #35449, #35450, #35451, #35452, #35453.