druid调参

原创

用户8761231

发布于 2022-02-22 00:11:10

4830

发布于 2022-02-22 00:11:10

如下是推荐参数： `broker 可以不用很大`

broker-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms32g
-Xmx32g
-XX:MaxDirectMemorySize=20g
-Djute.maxbuffer=41943040

broker-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.processing.numMergeBuffers 12
druid.processing.numThreads 48
druid.server.http.numThreads 85

common

common.runtime.properties
------------------------------------------------------------------------------------------------------------
druid.indexer.logs.directory  hdfs://HDFSxxxx/druid/indexing-logs
druid.storage.storageDirectory hdfs://HDFSxxxxx/druid/segments

coordinator

coordinator-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms12g
-Xmx12g
-Djute.maxbuffer=41943040

historical分冷热节点不同节点可以参考评论中的配置 historical冷节点

historical-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms100g
-Xmx100g
-XX:MaxDirectMemorySize=100g
-Djute.maxbuffer=41943040

historical-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.processing.numMergeBuffers 12
druid.processing.numThreads 48
druid.segmentCache.locations [{"path":"/data1/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data2/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data3/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data4/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data5/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data6/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data7/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data8/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data9/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data10/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data11/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data12/druid/var/druid/segment-cache","maxSize":10000000000000}]
druid.server.http.numThreads 85
druid.server.maxSize 120000000000000
druid.server.tier cold

historical热节点

historical-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms100g
-Xmx100g
-XX:MaxDirectMemorySize=100g
-Djute.maxbuffer=41943040

historical-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.processing.numMergeBuffers 12
druid.processing.numThreads 48
druid.segmentCache.locations [{"path":"/data/druid/var/druid/segment-cache","maxSize":4000000000000}]
druid.server.http.numThreads 85
druid.server.maxSize 4000000000000
druid.server.priority 100

middleManager

middleManager-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms128m
-Xmx128m
-Djute.maxbuffer=41943040

middleManager-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.indexer.task.baseTaskDir /data/druid/var/druid/task
druid.server.http.numThreads 25
druid.worker.capacity 40

overlord

overlord-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms12g
-Xmx12g
-Djute.maxbuffer=41943040

router

router-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms1g
-Xmx4g
-XX:MaxDirectMemorySize=512m
-Djute.maxbuffer=41943040


router-runtime.properties 
------------------------------------------------------------------------------------------------------------
druid.plaintextPort 8080 （上机器修改）

hadoop 参数和xml

druid和hdfs分开 http://yun.isd.com/index.php/package/versions/?product=other&package=druid_hdfs_conf cd /usr/local/service/druid/conf/druid/cluster/_common; rm *site.xml; cp /usr/local/services/druid_hdfs_conf-1.0/conf/hdfs42717/* . cd /usr/local/service/druid/conf/druid/_common; rm *site.xml; cp /usr/local/services/druid_hdfs_conf-1.0/conf/hdfs42717/* .
调整hdfs参数 hadoop-env.sh DNHeapsize 20480 Heapsize 10240 NNHeapsize 30720