Clickhouse在进行大数据量同步时,感觉很爽,插入速度非常快,但是,在使用过程中却出现了几次“Too many parts”异常报错,搞得很痛苦,这里记录一下解决过程。
我这边采用的是Flink程序,实时将数据写入ClickHouse,在执行一段时间后,会提示“Too many parts”异常,如下异常如下:
ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 252, host: 10.10.4.1, port: 8114; Code: 252, e.displayText() = DB::Exception: Too many partitions for single INSERT block (more than 100). The limit is controlled by 'max_partitions_per_insert_block' setting. Large number of partitions is a common misconception. It will lead to severe negative performance impact, including slow server startup, slow INSERT queries and slow SELECT queries. Recommended total number of partitions for a table is under 1000..10000. Please note, that partitioning