Erlang/OTP Forums

Author Message

<  RabbitMQ mailing list  ~  RabbitMQ on EC2

Guest
Posted: Fri Mar 14, 2008 2:11 am Reply with quote
Guest
I'm trying to create a cluster of RabbitMQ nodes on EC2 and I've been running
into some trouble. Using the cluster command succeeds, but the nodes end up
in an inconsistent state. It's easier to give an example than to explain the
results, so I'll post them below.

There are 3 nodes:
Node A calls itself rabbit@domU-12-31-38-00-48-D1
Node B calls itself rabbit@ip-10-251-159-160
Node C calls itself rabbit@domU-12-31-38-00-48-A2.

First I clustered A and B, as disk nodes. This worked fine. Then I tried to
add Node C.

Node A says:
Status of node 'rabbit@domU-12-31-38-00-48-D1' ...
[{running_applications,[{rabbit,"RabbitMQ","1.2.0"},
{mnesia,"MNESIA CXC 138 12","4.3.3"},
{os_mon,"CPO CXC 138 46","2.1.1"},
{sasl,"SASL CXC 138 11","2.1.4"},
{stdlib,"ERTS CXC 138 10","1.14.2"},
{kernel,"ERTS CXC 138 10","2.11.2"}]},
{nodes,['rabbit@domU-12-31-38-00-48-D1',
'rabbit@ip-10-251-159-160',
'rabbit@domU-12-31-38-00-48-A2']},

{running_nodes,['rabbit@ip-10-251-159-160','rabbit@domU-12-31-38-00-48-D1']}]

Node B says:
Status of node 'rabbit@ip-10-251-159-160' ...
[{running_applications,[{rabbit,"RabbitMQ","1.2.0"},
{mnesia,"MNESIA CXC 138 12","4.3.3"},
{os_mon,"CPO CXC 138 46","2.1.1"},
{sasl,"SASL CXC 138 11","2.1.4"},
{stdlib,"ERTS CXC 138 10","1.14.2"},
{kernel,"ERTS CXC 138 10","2.11.2"}]},
{nodes,['rabbit@domU-12-31-38-00-48-D1',
'rabbit@ip-10-251-159-160',
'rabbit@domU-12-31-38-00-48-A2']},
{running_nodes,['rabbit@domU-12-31-38-00-48-D1',
'rabbit@domU-12-31-38-00-48-A2',
'rabbit@ip-10-251-159-160']}]

Node C says:
Status of node 'rabbit@domU-12-31-38-00-48-A2' ...
[{running_applications,[{rabbit,"RabbitMQ","1.2.0"},
{mnesia,"MNESIA CXC 138 12","4.3.3"},
{os_mon,"CPO CXC 138 46","2.1.1"},
{sasl,"SASL CXC 138 11","2.1.4"},
{stdlib,"ERTS CXC 138 10","1.14.2"},
{kernel,"ERTS CXC 138 10","2.11.2"}]},
{nodes,['rabbit@ip-10-251-159-160','rabbit@domU-12-31-38-00-48-A2']},

{running_nodes,['rabbit@ip-10-251-159-160','rabbit@domU-12-31-38-00-48-A2']}]

If I reset A, then cluster it with C and B, I get a situation where A and C
switch: C can see all 3, but thinks A is down, and A can only see itself and
B.

I'm hoping someone has seen a similar problem before and can either point me
to where to look for either pointers on what may be wrong with my EC2 or
RabbitMQ setups. I know port 4369 is open, and the nodes can reach each
other via ping. I've managed to replicate this with additional boxes and
entire new setups.

Thanks for the help.

Kyle Salasko
--
View this message in context: http://www.nabble.com/RabbitMQ-on-EC2-tp16042914p16042914.html
Sent from the RabbitMQ mailing list archive at Nabble.com.


_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist
Guest
Posted: Fri Mar 14, 2008 5:49 am Reply with quote
Guest
Scratch that, I found a solution. An easy workaround is to find the internal
(inside the NAT) IP address of any nodes with the domU-... hostnames, and
construct the ip-... hostnames from that. For example, Node A from below
might resolve to 10.252.56.111, and by manually running 'hostname
ip-10-252-56-111.ec2.internal', the problems below resolve themselves.

The problem seems to arise from the fact that the machines can resolve the
first three of the following names, but not the fourth.

ip-10-251-159-160.ec2.internal
ip-10-251-159-160
domU-12-31-38-00-48-A2.compute-1.internal
domU-12-31-38-00-48-A2

This probably should be automated in AMI creation, but I thought I'd let
people know in case they run into the same problems.

--
View this message in context: http://www.nabble.com/RabbitMQ-on-EC2-tp16042914p16044743.html
Sent from the RabbitMQ mailing list archive at Nabble.com.


_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist
alexis
Posted: Fri Mar 14, 2008 8:08 pm Reply with quote
User Joined: 06 Sep 2007 Posts: 80 Location: London
Kyle

Thank-you for this.

We've been experimenting with some of these issues in a public way ...
see Dmitriy's post here: http://www.infoq.com/news/2008/02/ec2

Also (apologies for this but) -- http://es.cohesiveft.com/site/rabbitmq

alexis




On Fri, Mar 14, 2008 at 5:49 AM, Kyle Salasko <ksalasko@gmail.com> wrote:
>
> Scratch that, I found a solution. An easy workaround is to find the internal
> (inside the NAT) IP address of any nodes with the domU-... hostnames, and
> construct the ip-... hostnames from that. For example, Node A from below
> might resolve to 10.252.56.111, and by manually running 'hostname
> ip-10-252-56-111.ec2.internal', the problems below resolve themselves.
>
> The problem seems to arise from the fact that the machines can resolve the
> first three of the following names, but not the fourth.
>
> ip-10-251-159-160.ec2.internal
>
> ip-10-251-159-160
> domU-12-31-38-00-48-A2.compute-1.internal
>
> domU-12-31-38-00-48-A2
>
> This probably should be automated in AMI creation, but I thought I'd let
> people know in case they run into the same problems.
>
> --
> View this message in context: http://www.nabble.com/RabbitMQ-on-EC2-tp16042914p16044743.html
>
>
> Sent from the RabbitMQ mailing list archive at Nabble.com.
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss@lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>



--
Alexis Richardson
+44 20 7617 7339 (UK)
+44 77 9865 2911 (cell)
+1 650 206 2517 (US)

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist
View user's profile Send private message Yahoo Messenger
wuji
Posted: Tue Sep 18, 2012 7:10 am Reply with quote
User Joined: 10 Aug 2012 Posts: 654
security plan which, I believe, will provide both reassurance and and cheap replica *beep* and a powerful deterrent."In addition to the Rapier and High-Velocity
batteries, the government will also station a Royal Navy helicopter helicopter [h4]replica designer *beep*[/h4] helicopter carrier in the River Thames and station Royal Air
jets and army helicopters nearby.Residents of the Fred Wigg Tower Tower discount designer *beep* Tower and the Tower Hamlets, which will host High-Velocity missiles,
expressed concern about the batteries when the government made a a [h2]designer replica *beep*[/h2] a test deployment of the missiles atop their buildings in
May. "I'm not sure I can sleep in a house house [h3]cheap designer *beep*[/h3] house knowing there are missiles on the roof," journalist and
Hamlets resident Brian Whelan told ABC News in May.Said Secretary Secretary [h1]authentic jordans[/h1] Secretary Hammond, "A small number of activists object to the
of these defensive measures and a legal challenge to the the cheap jordans the Government's decision to deploy ... has been initiated. The
of Defense will defend these proceedings vigorously and is confident confident [h3]cheap designer *beep*[/h3] confident of defeating them."Additional anti-aircraft missile batteries are in place
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 1 of 1
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum