Erlang/OTP Forums

Author Message

<  Ejabberd mailing list  ~  Multi node ejabberd cluster set up

Guest
Posted: Fri Dec 04, 2009 8:03 pm Reply with quote
Guest
Hello.

I'm looking at deploying a clustered ejabberd setup at my company which
will have, initially, 4 nodes in various locations around the world, all
serving the same domain.

The idea is that users can connect to any node and will be able to
communicate with any other user logged into any other node without issue.

I've tested this set up with Parallels on my machine here, and that part
of communication works as advertised.

The questions I have are related to the mnesia database set up. I've
spent many hours reading mailing lists and the majority (all?) of
cluster related setup questions involve just 2 nodes.

This is what we're looking for in an ideal set up:

1. all 4 nodes communicating and replicating.
2. the ability for any node (or 2 nodes) to drop out without it
affecting the other nodes' communications (so if we lose 'B', people on
'A','C' and 'D' can all still see each other and communicate)
3. for any dropped out node to reintegrate into the cluster without
causing too much of a headache.

Following (slightly modified) instructions found at
http://www.process-one.net/en/ejabberd/guide_en#htoc82, part 1 is
working fine - with three nodes using one other as the master.

re point 2, I've found if the master node (the one passed as the '
-mnesia extra_db_nodes "['ejabberd@first']" \' param of the setup
instructions) goes down, then the other nodes are unable to restart
themselves.

An apparent work around is to, on the database set up page of ejabberd,
change all 'remote copy' databases to 'disk and local'. Once done,
they're able to restart. I'm unable to find if this is the correct
thing to do, or whether there's another solution (is it possible to
specify more than a single master node? what happens if you specify 3,
and one of those goes down?)

If multi-master isn't viable, is it possible to switch an instance to
look at another node for the master copy?

Any ideas, pointers or help you can provide will be appreciated.

Jonathan.
_______________________________________________
ejabberd mailing list
ejabberd@jabber.ru
http://lists.jabber.ru/mailman/listinfo/ejabberd
Post received from mailinglist
Guest
Posted: Wed Dec 09, 2009 8:31 pm Reply with quote
Guest
Here is the result of our recent "trial and error" investigation into
the best way to configure the database in a cluster. Sorry for the
formatting (pasted from a confluence table.)

Disclaimer: it may not be correct. It would be nice if there was
official documentation for this either provided by p1 or by the
community (is there even a place for the community to contribute
documentation?)

Table configuration

1. Inference1: If the master is RAM&disc, then the slave should be
1. RAM&disc if you want it available when slave starts up alone
2. RAMcopy if you don't
2. Inference2: If the master is RAMcopy, then the slave should be
1. RAM copy (instead of Remote Copy) so that the feature works
on the slave when the master is down.
2. Remote copy: if the feature only works if the master is up
3. Inference3: If the master is DiscOnly, then the slave should be:
1. Remote copy: if the feature only works if the master is up
2. There is a possible synchronization problem if we try to
set the slave to DiscOnly (needs to be confirmed)
3. Consider changing both master and slave to RAM&disc

table name master slave notes
acl
RAM and disc copy
RAM and disc copy
changed slave from RAM copy to RAM and disc copy due to Inference1
caps_feature
RAM copy
RAM copy
changed slave from Remote copy to RAM copy due to Inference2
config
RAM and disc copy
RAM and disc copy
changed slave from RAM copy to RAM and disc copy due to Inference1
disco_publish
Disc only copy
Remote copy
need to research changing both to RAM and disc copy
http_bind
RAM copy
RAM copy
changed slave from Remote copy to RAM copy due to Inference2
http_poll
RAM copy
RAM copy
changed slave from Remote copy to RAM copy due to Inference2
iq_response
RAM copy
RAM copy
OK
last_activity
Disc only copy
Remote copy
need to research changing both to RAM and disc copy
local_config
RAM and disc copy
RAM and disc copy
changed slave from RAM copy to RAM and disc copy due to Inference1
motd
Disc only copy
Remote copy
need to research changing both to RAM and disc copy
motd_users
Disc only copy
Remote copy
need to research changing both to RAM and disc copy
muc_online_room
RAM copy
Remote copy
leaving slave as Remote copy since muc features aren't clustered
need to research muc clustering
muc_registered
Disc only copy
Remote copy
leaving slave as Remote copy since muc features aren't clustered
need to research muc clustering
muc_room
Disc only copy
Remote copy
leaving slave as Remote copy since muc features aren't clustered
need to research muc clustering
offline_msg
Disc only copy
Disc only copy
leave both as disc only since this database has a risk of growing very
large
we should test to make sure that offline messages are delivered in
outage scenarios
privacy
Disc only copy
Disc only copy
need to research possible consistency issue due to Inference3
private_storage
Disc only copy
Remote copy
need to research changing both to RAM and disc copy
pubsub_node
Disc only copy
Remote copy
need to research changing both to RAM and disc copy
roster
RAM and disc copy
RAM and disc copy
changed slave from Disc only copy to RAM and disc copy due to Inference1
route
RAM copy
RAM copy
OK
s2s
RAM copy
RAM copy
OK
schema
RAM and disc copy
RAM and disc copy
OK
session
RAM copy
RAM copy
OK
user_caps
RAM and disc copy
RAM and disc copy
changed slave from Remote copy to RAM and disc copy due to Inference1
user_caps_resources
RAM and disc copy
RAM and disc copy
changed slave from Remote copy to RAM and disc copy due to Inference1
vcard
Disc only copy
Remote copy
leaving slave as Remote copy due to Inference3
we should retest the consistency issues
vcard_search
Disc only copy
Remote copy
leaving slave as Remote copy due to Inference3
we should retest the consistency issues

On 12/4/2009 2:03 PM, Jonathan Tullett wrote:
> Hello.
>
> I'm looking at deploying a clustered ejabberd setup at my company which
> will have, initially, 4 nodes in various locations around the world, all
> serving the same domain.
>
> The idea is that users can connect to any node and will be able to
> communicate with any other user logged into any other node without issue.
>
> I've tested this set up with Parallels on my machine here, and that part
> of communication works as advertised.
>
> The questions I have are related to the mnesia database set up. I've
> spent many hours reading mailing lists and the majority (all?) of
> cluster related setup questions involve just 2 nodes.
>
> This is what we're looking for in an ideal set up:
>
> 1. all 4 nodes communicating and replicating.
> 2. the ability for any node (or 2 nodes) to drop out without it
> affecting the other nodes' communications (so if we lose 'B', people on
> 'A','C' and 'D' can all still see each other and communicate)
> 3. for any dropped out node to reintegrate into the cluster without
> causing too much of a headache.
>
> Following (slightly modified) instructions found at
> http://www.process-one.net/en/ejabberd/guide_en#htoc82, part 1 is
> working fine - with three nodes using one other as the master.
>
> re point 2, I've found if the master node (the one passed as the '
> -mnesia extra_db_nodes "['ejabberd@first']" \' param of the setup
> instructions) goes down, then the other nodes are unable to restart
> themselves.
>
> An apparent work around is to, on the database set up page of ejabberd,
> change all 'remote copy' databases to 'disk and local'. Once done,
> they're able to restart. I'm unable to find if this is the correct thing
> to do, or whether there's another solution (is it possible to specify
> more than a single master node? what happens if you specify 3, and one
> of those goes down?)
>
> If multi-master isn't viable, is it possible to switch an instance to
> look at another node for the master copy?
>
> Any ideas, pointers or help you can provide will be appreciated.
>
> Jonathan.
> _______________________________________________
> ejabberd mailing list
> ejabberd@jabber.ru
> http://lists.jabber.ru/mailman/listinfo/ejabberd

--
Jesse Thompson
Division of Information Technology, University of Wisconsin-Madison
Email/IM: jesse.thompson@doit.wisc.edu



Post received from mailinglist
wuji
Posted: Thu Sep 13, 2012 6:29 am Reply with quote
User Joined: 10 Aug 2012 Posts: 654
to call the police so they can pay me the money."Suarez said said cheap polo shirts said she left Colombia for a few days because of concerns her
could be in danger and has had no contact with any American American replica designer bags for sale American official."As far as the investigators go, just like the Secret Service
are dumb, imagine the investigators," she said.Throughout her appearance this morning, dressed dressed cheap authentic jordans dressed in a skimpy green blouse, Suarez laughed and smiled even as
host reminded her of the seriousness of the scandal."This is who I I cheap Ralph Lauren Polo I am and all I really care about is my mother and
young son," Suarez said.She also said that she was with a friend friend [h4]cheap jordan shoes[/h4] friend when she first met Huntington in the disco, and that another
who was with Huntington "fell in love" with her friend, though they they cheap jordan shoes they did not have sex. "Their love story ended in the bar,"
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 1 of 1
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum