Q:

RabbitMQ Cannot connect the node in the cluster

I have 2 nodes and I wanna make cluster out of it. I have installed RabbitMq.
Now I am stopping the rabbit app in 2nd node and trying to join cluster like below command:
sudo rabbitmqctl join_cluster rabbit@rabbit1
But it throw below error
> DIAGNOSTICS
> ===========
> 
> attempted to contact: [rabbit@TELXRMQ01]
> 
> rabbit@rabbit1:   * unable to connect to epmd (port 4369) on rabbit1:
> nxdomain (non-existing domain)
> 
> 
> current node details:
> - node name: 'rabbitmq-cli-34@rabbit2'
> - home dir: /var/lib/rabbitmq
> - cookie hash: A85MNn8I1UhtrGozi+m/2g==

I am following this link : https://www.rabbitmq.com/clustering.html
The errang cookie are same in both the node and 4369 port is open in both the node.

A:

There are many ways to solved this problem.
Under RabbitMQ-3.6.9 and Erlang v8.3.3

Correct and Same the file of .erlang.cookie
chgrp rabbitmq /var/lib/rabbitmq/.erlang.cookie
chown rabbitmq /var/lib/rabbitmq/.erlang.cookie
chmod 400 /var/lib/rabbitmq/.erlang.cookie

Correct Hosts and Hostname

Hosts:
cat /etc/hosts

Result:
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

127.0.0.1     rabbitmq252
192.168.4.249 rabbitmq249
192.168.4.251 rabbitmq251
192.168.4.252 rabbitmq252

Hostname:
cat /etc/hostname

Result:
rabbitmq252

Remove Cluster Node
If the cluster already has the node, or status changed, or even some node disagree the other node. You can remove the node by the command:
rabbitmqctl forget_cluster_node rabbit@rabbitmq252

Clear Erlang Mnesia Database
Under the file path of /var/lib/rabbitmq/, there are three files, like erl_crash.dump  mnesia .erlang.cookie.
rm -rf /var/lib/rabbitmq/*

Kill the process of rabbitmq
ps aux|grep rabbitmq
kill -9 remaining_process

Reinstall RabbitMQ and Erlang

Anyway, if you encounter the problem, you should:

Read the status of service
systemctl status rabbitmq-server.service -l
# or
service rabbitmq-server status
# or
journalctl -ex

Read the logs of service 
You can find the path under the status of service.
cat /var/log/rabbitmq/rabbit@rabbitmq252.log
# or
cat /var/log/rabbitmq/rabbit@rabbitmq251-sasl.log

Read the status of ports
netstat -ntlp

