文章/答案/技术大牛

发布

社区首页 >专栏 >【重磅更新】Graylog6.1.6集群部署实践教程

【重磅更新】Graylog6.1.6集群部署实践教程

yuanfan2012

发布于 2025-02-18 13:15:01

49501

代码可运行

文章被收录于专栏：WalkingCloudWalkingCloud

运行总次数：1

代码可运行

Graylog6.1.6集群部署实践教程

GrayLog6.X版本多节点的集群逻辑架构如下

(图片点击放大查看)

GrayLog6.1.6集群+MongoDB6.0集群+Opensearch2.15集群

只不过三个不同服务均部署在同一个节点上

部署可以参考之前的文章Graylog4.2集群部署完整教程

一、基础环境准备

三台服务器：RockyLinux9.5的虚拟机

/data分区（LVM) 用于opensearch日志数据存储

内存大小建议为8GB以上

IP地址和主机名如下：

192.168.31.145 graylog01 graylog01.walkingcloud.com
192.168.31.146 graylog02 graylog02.walkingcloud.com
192.168.31.147 graylog03 graylog03.walkingcloud.com

均已关闭SELINUX

(图片点击放大查看)

三台服务器需要配置主机名本地host解析

echo "192.168.31.145 graylog01 graylog01.walkingcloud.com" >> /etc/hosts
echo "192.168.31.146 graylog02 graylog02.walkingcloud.com" >> /etc/hosts
echo "192.168.31.147 graylog03 graylog03.walkingcloud.com" >> /etc/hosts

(图片点击放大查看)

二、搭建MongoDB6.0集群

我这边已经打包好EL9下mongodb所需要的rpm包

mongodb-database-tools-100.11.0-1.x86_64.rpm
mongodb-mongosh-2.3.9.x86_64.rpm
mongodb-org-6.0.20-1.el9.x86_64.rpm
mongodb-org-database-6.0.20-1.el9.x86_64.rpm
mongodb-org-database-tools-extra-6.0.20-1.el9.x86_64.rpm
mongodb-org-mongos-6.0.20-1.el9.x86_64.rpm
mongodb-org-server-6.0.20-1.el9.x86_64.rpm
mongodb-org-tools-6.0.20-1.el9.x86_64.rpm

三台均按如下命令安装好mongodb

mkdir /opt/mongodb
tar -zxvf mongodb6.0.tar.gz -C /opt/mongodb/
cd /opt/mongodb/
 yum localinstall -y mongodb*.rpm

(图片点击放大查看)

graylog01主节点上先启动mongod服务

systemctl daemon-reload
systemctl enable mongod --now
systemctl start mongod
systemctl status mongod
firewall-cmd --add-port=27017/tcp --permanent --zone=public 
firewall-cmd --reload

(图片点击放大查看)

先在主节点上面创建账号，然后再修改配置文件

mongosh
use admin
db.createUser({user: "admin", pwd: "Admin@2025", roles: ["root"]})
db.auth("admin","Admin@2025")

(图片点击放大查看)

use graylog
db.createUser({
   user: "graylog", 
   pwd: "Graylog2025",
  "roles" : [{
      "role" : "dbOwner",
      "db" : "graylog"
    }, {
      "role" : "readWrite",
      "db" : "graylog"
    }]
})

(图片点击放大查看)

接下来主节点上生成keyfile并scp同步到graylog02和graylog03

openssl rand -base64 756 > /var/lib/mongo/access.keyfile
chown mongod:mongod /var/lib/mongo/access.keyfile
chmod 600 /var/lib/mongo/access.keyfile 
scp -rp /var/lib/mongo/access.keyfile root@graylog02:/var/lib/mongo/
scp -rp /var/lib/mongo/access.keyfile root@graylog03:/var/lib/mongo/

(图片点击放大查看)

然后继续修改mongod.conf 修改如下位置

net:
  port: 27017
  bindIp: 192.168.31.145

security:
  keyFile: /var/lib/mongo/access.keyfile

replication:
  replSetName: graylog-rs

(图片点击放大查看)

并将配置文件scp到graylog02和graylog03节点上

scp -rp /var/lib/mongo/access.keyfile root@graylog02:/var/lib/mongo/
scp -rp /var/lib/mongo/access.keyfile root@graylog03:/var/lib/mongo/

(图片点击放大查看)

接下来在graylog02节点上

sed -i "s/192.168.31.145/192.168.31.146/g" /etc/mongod.conf
cat  /etc/mongod.conf  | grep  bindIp
chown mongod:mongod /var/lib/mongo/access.keyfile
chmod 600 /var/lib/mongo/access.keyfile 
firewall-cmd --add-port=27017/tcp --permanent --zone=public 
firewall-cmd --reload

(图片点击放大查看)

同样graylog03节点也执行上面的命令

(图片点击放大查看)

然后graylog01，graylog02节点和graylog03上重启mongodb服务

systemctl enable mongod 
systemctl restart mongod 

systemctl status mongod

(图片点击放大查看)

检查一下三台的mongdb状态

接下来登录主节点进行初始化操作

mongosh -u admin mongodb://192.168.31.145:27017/

rs.initiate( {
   _id : "graylog-rs",
   members: [
      { _id: 0, host: "graylog01:27017" },
      { _id: 1, host: "graylog02:27017" },
      { _id: 2, host: "graylog03:27017" }
   ]
})

 rs.status()查看集群状态

(图片点击放大查看)

(图片点击放大查看)

可以看到graylog01为Primary，其它两个节点为secondary状态

这样mongdb集群就搭建完成了

可以用如下命令验证是否可以登录mongdb集群

mongosh mongodb://graylog:Graylog2025@graylog01:27017,graylog02:27017,graylog03:27017/graylog?replicaSet=graylog-rs

(图片点击放大查看)

三、接下来搭建opensearch集群

先在主节点上面安装opensearch2.15.0

env OPENSEARCH_INITIAL_ADMIN_PASSWORD=Opensearch_2025 rpm -ivh /root/opensearch-2.15.0-linux-x64.rpm

(图片点击放大查看)

单独的/data目录下创建目录用于存放opensearch数据

mkdir -p /data/opensearch/data
mkdir -p /data/opensearch/logs
chown -R opensearch /data/opensearch
sysctl -w vm.max_map_count=262144
echo 'vm.max_map_count=262144' >> /etc/sysctl.conf
cp /etc/opensearch/opensearch.yml /etc/opensearch/opensearch.yml_default

#修改opensearch相关配置文件

sed -i "s@#cluster.name: my-application@cluster.name: graylog@g" /etc/opensearch/opensearch.yml
sed -i "s@#node.name: node-1@node.name: graylog01@g" /etc/opensearch/opensearch.yml
sed -i "s#path.data: /var/lib/opensearch#path.data: /data/opensearch/data#g" /etc/opensearch/opensearch.yml
sed -i "s#path.logs: /var/log/opensearch#path.logs: /data/opensearch/logs#g" /etc/opensearch/opensearch.yml
sed -i "s@#network.host: 192.168.0.1@network.host: 0.0.0.0@g" /etc/opensearch/opensearch.yml
echo "action.auto_create_index: false" >> /etc/opensearch/opensearch.yml
echo "indices.query.bool.max_clause_count: 32768" >> /etc/opensearch/opensearch.yml
echo "node.roles: [cluster_manager, data, ingest]"  >> /etc/opensearch/opensearch.yml
echo 'discovery.seed_hosts: ["graylog01", "graylog02", "graylog03"]' >> /etc/opensearch/opensearch.yml
echo 'cluster.initial_cluster_manager_nodes: ["graylog01", "graylog02", "graylog03"] '>> /etc/opensearch/opensearch.yml

sed -i "s#plugins.security.ssl.http.enabled: true#plugins.security.ssl.http.enabled: false#g" /etc/opensearch/opensearch.yml

#启动opensearch服务

systemctl daemon-reload
systemctl enable opensearch.service
systemctl restart opensearch.service
firewall-cmd --add-port=9200/tcp --permanent --zone=public
firewall-cmd --add-port=9300/tcp --permanent --zone=public  
firewall-cmd --reload

(图片点击放大查看)

同样graylog02节点

mkdir -p /data/opensearch/data
mkdir -p /data/opensearch/logs
chown -R opensearch /data/opensearch
sysctl -w vm.max_map_count=262144
echo 'vm.max_map_count=262144' >> /etc/sysctl.conf
cp /etc/opensearch/opensearch.yml /etc/opensearch/opensearch.yml_default
#修改opensearch相关配置文件
sed -i "s@#cluster.name: my-application@cluster.name: graylog@g" /etc/opensearch/opensearch.yml
sed -i "s@#node.name: node-1@node.name: graylog02@g" /etc/opensearch/opensearch.yml
sed -i "s#path.data: /var/lib/opensearch#path.data: /data/opensearch/data#g" /etc/opensearch/opensearch.yml
sed -i "s#path.logs: /var/log/opensearch#path.logs: /data/opensearch/logs#g" /etc/opensearch/opensearch.yml
sed -i "s@#network.host: 192.168.0.1@network.host: 0.0.0.0@g" /etc/opensearch/opensearch.yml
echo "action.auto_create_index: false" >> /etc/opensearch/opensearch.yml
echo "indices.query.bool.max_clause_count: 32768" >> /etc/opensearch/opensearch.yml
echo "node.roles: [cluster_manager, data, ingest]"  >> /etc/opensearch/opensearch.yml
echo 'discovery.seed_hosts: ["graylog01", "graylog02", "graylog03"]' >> /etc/opensearch/opensearch.yml
echo 'cluster.initial_cluster_manager_nodes: ["graylog01", "graylog02", "graylog03"] '>> /etc/opensearch/opensearch.yml

sed -i "s#plugins.security.ssl.http.enabled: true#plugins.security.ssl.http.enabled: false#g" /etc/opensearch/opensearch.yml
#启动opensearch服务
systemctl daemon-reload
systemctl enable opensearch.service
systemctl restart opensearch.service


firewall-cmd --add-port=9200/tcp --permanent --zone=public
firewall-cmd --add-port=9300/tcp --permanent --zone=public  
firewall-cmd --reload

graylog03上

mkdir -p /data/opensearch/data
mkdir -p /data/opensearch/logs
chown -R opensearch /data/opensearch
sysctl -w vm.max_map_count=262144
echo 'vm.max_map_count=262144' >> /etc/sysctl.conf
cp /etc/opensearch/opensearch.yml /etc/opensearch/opensearch.yml_default
#修改opensearch相关配置文件
sed -i "s@#cluster.name: my-application@cluster.name: graylog@g" /etc/opensearch/opensearch.yml
sed -i "s@#node.name: node-1@node.name: graylog03@g" /etc/opensearch/opensearch.yml
sed -i "s#path.data: /var/lib/opensearch#path.data: /data/opensearch/data#g" /etc/opensearch/opensearch.yml
sed -i "s#path.logs: /var/log/opensearch#path.logs: /data/opensearch/logs#g" /etc/opensearch/opensearch.yml
sed -i "s@#network.host: 192.168.0.1@network.host: 0.0.0.0@g" /etc/opensearch/opensearch.yml
echo "action.auto_create_index: false" >> /etc/opensearch/opensearch.yml
echo "indices.query.bool.max_clause_count: 32768" >> /etc/opensearch/opensearch.yml
echo "node.roles: [cluster_manager, data, ingest]"  >> /etc/opensearch/opensearch.yml
echo 'discovery.seed_hosts: ["graylog01", "graylog02", "graylog03"]' >> /etc/opensearch/opensearch.yml
echo 'cluster.initial_cluster_manager_nodes: ["graylog01", "graylog02", "graylog03"] '>> /etc/opensearch/opensearch.yml

sed -i "s#plugins.security.ssl.http.enabled: true#plugins.security.ssl.http.enabled: false#g" /etc/opensearch/opensearch.yml
#启动opensearch服务
systemctl daemon-reload
systemctl enable opensearch.service
systemctl restart opensearch.service
firewall-cmd --add-port=9200/tcp --permanent --zone=public
firewall-cmd --add-port=9300/tcp --permanent --zone=public  
firewall-cmd --reload

(图片点击放大查看)

接下来验证opensearch集群是否部署成功

curl -s -XGET -u admin:Opensearch_2025 'http://192.168.31.147:9200/_cluster/health?pretty=true'

curl -s -XGET -u admin:Opensearch_2025 'http://192.168.31.147:9200/_cat/nodes?v'

(图片点击放大查看)

四、接下来部署Graylog6.1.6版本的集群

三台服务器都安装graylog

rpm -ivh graylog-server-6.1.6-1.x86_64.rpm 
cp /etc/graylog/server/server.conf /etc/graylog/server/server.conf_default

(图片点击放大查看)

修改graylog01的graylog配置文件vim server.conf

cat /etc/graylog/server/server.conf | grep -v "^#" | grep -v "^$"

最后修成后的server.conf配置如下

is_leader = true
node_id_file = /etc/graylog/server/node-id
password_secret = 0pAHJtPdZZUb5yHAvFbBezbWAlQwh9CbRX1rshJEVxM0kV7t0SpIgY5q9tLpVEwWLElhG3EtbvQ03mTm9i0HuvWKwlWgWiIJ
root_password_sha2 = 429d280c5ddad83d94770b077b22124231efc727d504b107883297304b3e2939
root_timezone = Asia/Shanghai
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 192.168.31.145:9000
http_publish_uri = http://192.168.31.145:9000/
http_external_uri =  http://192.168.31.145:9000/
stream_aware_field_types=false
disabled_retention_strategies = none,close
allow_leading_wildcard_searches = false
allow_highlighting = true
field_value_suggestion_mode = on
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://graylog:Graylog2025@graylog01:27017,graylog02:27017,graylog03:27017/graylog?replicaSet=graylog-rs
mongodb_max_connections = 1000
integrations_scripts_dir = /usr/share/graylog-server/scripts
elasticsearch_hosts = http://admin:Opensearch_2025@192.168.31.145:9200,http://admin:Opensearch_2025@192.168.31.146:9200,http://admin:Opensearch_2025@192.168.31.147:9200

(图片点击放大查看)

接下来可以将graylog01下graylog配置文件拷贝到节点graylog02和graylog03上面

 scp server.conf root@graylog02:/etc/graylog/server/
 scp server.conf root@graylog03:/etc/graylog/server/

(图片点击放大查看)

在节点graylog02和graylog03上修改如下四处即可

is_leader = false
http_bind_address = 192.168.31.147:9000
http_publish_uri = http://192.168.31.147:9000/
http_external_uri =  http://192.168.31.147:9000/

(图片点击放大查看)

(图片点击放大查看)

修改完成后三台同时重启服务

systemctl enable graylog-server.service
systemctl start graylog-server.service
firewall-cmd --permanent --zone=public --add-port=9000/tcp
firewall-cmd --reload

(图片点击放大查看)

五、验证Graylog集群是否搭建成功

(图片点击放大查看)

(图片点击放大查看)

可以看到集群节点状态，graylog01为主节点

Tips:

本文参考如下链接完成

https://opensearch.org/docs/latest/tuning-your-cluster/
https://go2docs.graylog.org/current/setting_up_graylog/multi-node_setup.html
https://www.mongodb.com/docs/manual/tutorial/deploy-replica-set/
https://opensearch.org/docs/2.15/tuning-your-cluster/
https://www.mongodb.com/zh-cn/docs/v7.0/tutorial/deploy-replica-set-with-keyfile-access-control/
https://go2docs.graylog.org/current/planning_your_deployment/planning_your_deployment.html
https://github.com/austinsenv/graylog-stack/blob/main/compose.yml
https://nickebo28.rssing.com/chan-55401497/article15.html
https://github.com/Graylog2/se-poc-docs
https://github.com/s0p4L1n3/Graylog-Cluster-Docker-Swarm