本文介绍如何在CentOS7.* 上搭建RabbitMQ Cluster及emqx桥接rabbitMQ集群的Haproxy相应配置 环境介绍
| 主机名 | IP |
| rabbitmq1 | 192.168.50.134 |
| rabbitmq2 | 192.168.50.135 |
| rabbitmq3 | 192.168.50.136 |
一、修改主机名
如果已经修改过主机名的话，就不再进⾏修改。但是切记rabbitmq搭建完成后期不要再进⾏修改，否则会出现问 题。
hostnamectl set-hostname rabbitmq1 hostnamectl set-hostname rabbitmq2 hostnamectl set-hostname rabbitmq3
每个节点添加hosts
192.168.50.134 rabbitmq1 192.168.50.135 rabbitmq2 192.168.50.136 rabbitmq3
最好是重启一下
reboot
二、安装源
三台分别都进⾏操作
分别是erlang和rabbitmq的源
erlang的源
[rabbitmq_erlang] name=rabbitmq_erlang baseurl=https://packagecloud.io/rabbitmq/erlang/el/7/$basearch repo_gpgcheck=1 gpgcheck=0 enabled=1 gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey sslverify=1 sslcacert=/etc/pki/tls/certs/ca-bundle.crt metadata_expire=300
rabbitmq-server的源
[rabbitmq-server] name=rabbitmq-server baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/7/$basearch repo_gpgcheck=1 gpgcheck=0 enabled=1 gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey sslverify=1 sslcacert=/etc/pki/tls/certs/ca-bundle.crt metadata_expire=300
三、安装
三台分别都进行操作
yum update -y；yum install socat logrotate -y;yum install --repo rabbitmq_erlang --repo rabbitmq_server erlang rabbitmq-server -y
分别启动rabbitmq-server服务
systemctl start rabbitmq-server
四、三台机器同步.erlang.cookie文件
这⾥我们以 rabbtmq1 节点作为集群管理节点， .erlang.cookie ⽂件在rabbitmq的安装⽬ 录 /var/lib/rabbitmq ⽬录下，这是⼀个隐藏⽂件，需要加上 -a 参数
[root@rabbitmq1 rabbitmq]# ls -al total 8 drwxr-xr-x 3 rabbitmq rabbitmq 42 Oct 30 21:47 . drwxr-xr-x. 27 root root 4096 Oct 30 21:20 .. -r-------- 1 rabbitmq rabbitmq 20 Oct 30 00:00 .erlang.cookie drwxr-x--- 4 rabbitmq rabbitmq 135 Oct 30 21:47 mnesia
这里我们把.erlang.cookie文件同步到其他机器上
[root@rabbitmq1 rabbitmq]# scp .erlang.cookie root@192.168.50.135:/var/lib/rabbitmq/ root@192.168.50.135's password: .erlang.cookie 100% 20 27.0KB/s 00:00 [root@rabbitmq1 rabbitmq]# scp .erlang.cookie root@192.168.50.136:/var/lib/rabbitmq/ root@192.168.50.136's password: .erlang.cookie 100% 20 36.7KB/s 00:00
同步完成之后我们再来查看⼏个节点的 .erlang.cookie ⽂件，发现都是⼀致的，这是我们想要的结果。
[root@rabbitmq1 rabbitmq]# cat .erlang.cookie SFWVLUCDUUVPIVRJDWTE[root@rabbitmq1 rabbitmq]#
五、重启服务
三个节点分别重启rabbitmq-server服务
systemctl restart rabbitmq-server
六、三个节点都打开rabbitmq监控插件
如果不打开的话，那么集群之间⽆法查看对⽅的数据，查看数据就是依靠web插件来实现的
[root@rabbitmq1 ~]# rabbitmq-plugins enable rabbitmq_management Enabling plugins on node rabbit@rabbitmq1: rabbitmq_management The following plugins have been configured: rabbitmq_management rabbitmq_management_agent rabbitmq_web_dispatch Applying plugin configuration to rabbit@rabbitmq1... The following plugins have been enabled: rabbitmq_management rabbitmq_management_agent rabbitmq_web_dispatch started 3 plugins.
其他另外两个节点都打开此功能（如有必要，需要重启服务，不过⼀般都不⽤重启rabbitmq）
七、在rabbitmq1上面添加用户
默认下，我们上⾯打开了监控插件功能，那么此时就能够看到15672端⼝了，我们可以使⽤IP:PORT的⽅式来进⾏ 访问，默认地账户密码都是guest，但是rabbitmq只允许本机通过localhost的⽅式进⾏通讯，因此我们再创建个其 他的⽤户实现远程访问。
rabbitmq1添加⽤户实现web访问
[root@rabbitmq1 ebin]# rabbitmqctl add_user test test123 Adding user "test" ... [root@rabbitmq1 ebin]# rabbitmqctl set_permissions -p / test ".*" ".*" ".*" Setting permissions for user "test" in vhost "/" ... [root@rabbitmq1 ebin]# rabbitmqctl set_user_tags admin administrator Setting tags for user "test" to [administrator] ...
这个时候再次访问并登录即可
八、将其他两个节点rabbitmq2、rabbitmq3加入集群中
这⾥需要强调的是，我们将rabbitmq1作为第⼀个集群节点，然后在rabbitm2和rabbitmq3节点分别操作申请加⼊ 集群的⽅式。
rabbitmq1节点查看当前集群
虽然只有一个节点，但也是集群，也是可以查看本身的集群信息的
[root@rabbitmq1 ebin]# rabbitmqctl cluster_status Cluster status of node rabbit@rabbitmq1 ... Basics Cluster name: rabbit@rabbitmq1 Disk Nodes rabbit@rabbitmq1 Running Nodes rabbit@rabbitmq1 Versions rabbit@rabbitmq1: RabbitMQ 3.8.9 on Erlang 21.3.8.18 Maintenance status Node: rabbit@rabbitmq1, status: not under maintenance Alarms (none) Network Partitions (none) Listeners Node: rabbit@rabbitmq1, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@rabbitmq1, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9- 1 and AMQP 1.0 Node: rabbit@rabbitmq1, interface: [::], port: 15672, protocol: http, purpose: HTTP API Feature flags Flag: drop_unroutable_metric, state: disabled Flag: empty_basic_get_metric, state: disabled Flag: implicit_default_bindings, state: enabled Flag: maintenance_mode_status, state: enabled Flag: quorum_queue, state: enabled Flag: virtual_host_metadata, state: enabled
我们需要注意的⼀⾏信息就是 Cluster name: rabbit@rabbitmq1 。要记住这个名字，⼀会我们要使⽤。
现在我们操作rabbitmq2节点
需要强调⼀下，集群中⾄少有⼀个节点是磁盘节点⽤于数据持久化，然后剩下的节点都设置为内存节点以提⾼性 能。
这⾥将 rabbitmq2 添加到集群当中，并成为内存节点，不加--ram 默认是磁盘节点
[root@rabbitmq2 ~]# rabbitmqctl stop_app Stopping rabbit application on node rabbit@rabbitmq2 ... [root@rabbitmq2 ~]# rabbitmqctl reset Resetting node rabbit@rabbitmq2 ... [root@rabbitmq2 ~]# rabbitmqctl join_cluster --ram rabbit@rabbitmq1 Clustering node rabbit@rabbitmq2 with rabbit@rabbitmq1 [root@rabbitmq2 ~]# rabbitmqctl start_app Starting node rabbit@rabbitmq2 ...
好了，现在rabbitmq2节点已经加⼊集群中了，怎么验证呢？我们来到rabbitmq1节点上⾯查看集群信息即可看到 rabbitmq2节点。
[root@rabbitmq1 ~]# rabbitmqctl cluster_status Cluster status of node rabbit@rabbitmq1 ... Basics Cluster name: rabbit@rabbitmq1 Disk Nodes rabbit@rabbitmq1 RAM Nodes rabbit@rabbitmq2 Running Nodes rabbit@rabbitmq1 rabbit@rabbitmq2 Versions rabbit@rabbitmq1: RabbitMQ 3.8.9 on Erlang 21.3.8.18 rabbit@rabbitmq2: RabbitMQ 3.8.9 on Erlang 21.3.8.18 Maintenance status Node: rabbit@rabbitmq1, status: not under maintenance Node: rabbit@rabbitmq2, status: not under maintenance Alarms (none) Network Partitions (none) Listeners Node: rabbit@rabbitmq1, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@rabbitmq1, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9- 1 and AMQP 1.0 Node: rabbit@rabbitmq1, interface: [::], port: 15672, protocol: http, purpose: HTTP API Node: rabbit@rabbitmq2, interface: [::], port: 15672, protocol: http, purpose: HTTP API Node: rabbit@rabbitmq2, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@rabbitmq2, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9- 1 and AMQP 1.0
现在操作rabbitmq3节点加入集群
[root@rabbitmq3 ~]# rabbitmqctl stop_app Stopping rabbit application on node rabbit@rabbitmq3 ... [root@rabbitmq3 ~]# rabbitmqctl reset Resetting node rabbit@rabbitmq3 ... [root@rabbitmq3 ~]# rabbitmqctl join_cluster --ram rabbit@rabbitmq1 Clustering node rabbit@rabbitmq3 with rabbit@rabbitmq1 [root@rabbitmq3 ~]# rabbitmqctl start_app Starting node rabbit@rabbitmq3 ...
现在也可以来到rabbitmq1节点查看到rabbitmq3的节点信息了。 当然也可以在web上⾯查看到三个节点的信息，如下所示：
九、设置集群为镜像模式
镜像队列机制就是将队列在 N 个节点之间设置主从关系，消息会在 N 个节点之间进⾏⾃动同步，且如果其中⼀个 节点不可⽤，并不会导致消息丢失或服务不可⽤的情况，提升 MQ 集群的整体⾼可⽤性。
这⾥在rabbitmq1节点操作（在任意节点操作都是可以的）
[root@rabbitmq1 ~]# rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all"}' Setting policy "ha-all" for pattern "^" to "{"ha-mode":"all"}" with priority "0" for vhost "/" ...十、验证当前集群
[root@rabbitmq1 ~]# rabbitmqctl cluster_status Cluster status of node rabbit@rabbitmq1 ... Basics Cluster name: rabbit@rabbitmq1 Disk Nodes rabbit@rabbitmq1 RAM Nodes rabbit@rabbitmq2 rabbit@rabbitmq3 Running Nodes rabbit@rabbitmq1 rabbit@rabbitmq2 rabbit@rabbitmq3 Versions rabbit@rabbitmq1: RabbitMQ 3.8.9 on Erlang 21.3.8.18 rabbit@rabbitmq2: RabbitMQ 3.8.9 on Erlang 21.3.8.18 rabbit@rabbitmq3: RabbitMQ 3.8.9 on Erlang 21.3.8.18 Maintenance status Node: rabbit@rabbitmq1, status: not under maintenance Node: rabbit@rabbitmq2, status: not under maintenance Node: rabbit@rabbitmq3, status: not under maintenance Alarms (none) Network Partitions (none) Listeners Node: rabbit@rabbitmq1, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@rabbitmq1, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9- 1 and AMQP 1.0 Node: rabbit@rabbitmq1, interface: [::], port: 15672, protocol: http, purpose: HTTP API Node: rabbit@rabbitmq2, interface: [::], port: 15672, protocol: http, purpose: HTTP API Node: rabbit@rabbitmq2, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@rabbitmq2, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9- 1 and AMQP 1.0 Node: rabbit@rabbitmq3, interface: [::], port: 15672, protocol: http, purpose: HTTP API Node: rabbit@rabbitmq3, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@rabbitmq3, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9- 1 and AMQP 1.
Haproxy配置
global log 127.0.0.1 local3 info #chroot /opt/app/haproxy #user haproxy #group haproxy daemon # DO NOT run in background, otherwise docker container will quit tune.ssl.default-dh-param 2048 maxconn 1024000 #nbproc 2 #cpu-map 1 0 #cpu-map 2 1 #cpu-map 3 2 #cpu-map 4 3 #cpu-map 5 4 #cpu-map 6 5 #cpu-map 7 6 #cpu-map 8 7 pidfile /usr/local/haproxy/haproxy.pid defaults log global mode tcp option tcplog maxconn 1024000 #option dontlognull timeout connect 30000 # timeout > mqtt's keepalive * 1.2 timeout client 600s timeout server 600s listen stats bind *:15672 mode http stats enable stats realm Haproxy\ Statistics stats uri / stats auth test:test123 listen rabbitmq bind *:5672 mode tcp balance roundrobin server rabbit1 192.168.50.134:5672 check inter 5s rise 2 fall 3 maxconn 34000 server rabbit2 192.168.50.135:5672 check inter 5s rise 2 fall 3 maxconn 34000 server rabbit3 192.168.50.136:5672 check inter 5s rise 2 fall 3 maxconn 34000