A proposal to remove pgrust from ClickBench says the database entry uses machinery tailored to the benchmark's 43 queries and reports hot-run results produced from cached outcomes, practices the author argues conflict with the benchmark's rules.
The claims appear in pull request 2165 in the ClickHouse ClickBench repository. The request targets the pgrust entry introduced through an earlier submission and recently updated to a version labelled 0.4-preview. At the event date, the removal was a proposal in the repository rather than a final independent judgment about the software.
According to the pull request, ClickBench requires query-result caches and caches near the end of the execution pipeline to be disabled. It also says the entry was marked as not tuned even though the implementation contained paths written for the exact ClickBench workload.
The author says the published 0.4-preview binary and publicly available 0.3 source were examined. The newer source was not public, while the release binary reportedly identified itself as pgrust 0.3-beta and contained paths naming individual ClickBench queries. The public source was said to include the benchmark SQL and a file describing hand-lowered physical plans for all 43 queries.
Several examples are offered to support the complaint. Some queries were described as being answered from metadata without a data scan. For queries 20, 21 and 23, the pull request says plan notes described a hot repetition as a fold over cached verdicts that touched no columns. The author also reported that pgrust's explain output identified specialized execution families for tested queries.
Local testing described in the pull request produced a much slower first execution for one substring query after a restart, followed by responses measured in milliseconds. The author interpreted the quick repeated result as replay of the earlier verdict rather than general-purpose query execution. Tests with modified benchmark queries also reportedly produced refusals for a portion of near-benchmark and perturbed statements.
The complaint does not allege that the 43 benchmark answers are broadly incorrect. The author compared results with ClickHouse local over the same Parquet data and said 30 matched exactly, with the remaining differences attributed to valid ordering ties in limited-result queries. The objection is instead about what the timing numbers measure and whether the entry follows ClickBench's configuration and tuning rules.
Because the evidence and interpretation were submitted by the author of the removal request, the claims should be read as allegations within an open-source review process. The unavailable 0.4 source also limits independent inspection from the material cited in the request. Maintainers can use the pull-request discussion and published artifacts to decide whether the entry should be removed, relabelled or tested under revised conditions.
