| Author |
Message |
|
| Guest |
 Posted: Mon Sep 28, 2009 9:51 pm |
 |
|
|
Guest
|
Will the RabbitMQ 1.7.0 disk layout (persister.log, mnesia schema,
etc) be compatible with 1.6.0, or will it clear out the old schema
like the pre-1.5 to 1.5 upgrade did? If the 1.7.0 layout isn't
compatible, is there a guide for migrating from 1.6 to 1.7? A search
on rabbitmq upgrade just brought up the 1.5.4 announcement that said
to contact support@rabbitmq if help was needed.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Sep 29, 2009 2:23 pm |
|
|
|
Guest
|
Hi,
1.7 will have a different mnesia schema and a totally different
persister layout. Whilst we have considered being able to migrate the
persister contents, migrating between mnesia schemas is far more
painful. It's likely that we'll recommend using a client that connects
to both versions of Rabbit, and copies messages from the old to the
new.
Matthew
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Sep 29, 2009 2:35 pm |
|
|
|
Guest
|
On Tue, Sep 29, 2009 at 03:22:50PM +0100, Matthew Sackman wrote:
> 1.7 will have a different mnesia schema and a totally different
> persister layout. Whilst we have considered being able to migrate the
> persister contents, migrating between mnesia schemas is far more
> painful. It's likely that we'll recommend using a client that connects
> to both versions of Rabbit, and copies messages from the old to the
> new.
Ahh, Matthias has just reminded me that 1.7 is being released before
the new persister, and so will be able to use the persister.log file
from 1.6 installations. So far, there have been no reasons to change
the mnesia schema either so unless that changes between now and 1.7
being released, then you should be able to upgrade seemlessly.
My comments above really refer to when the new persister gets released.
Sorry for creating confusion.
Matthew
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Sep 29, 2009 2:48 pm |
|
|
|
Guest
|
> Ahh, Matthias has just reminded me that 1.7 is being released before
> the new persister, and so will be able to use the persister.log file
> from 1.6 installations. So far, there have been no reasons to change
> the mnesia schema either so unless that changes between now and 1.7
> being released, then you should be able to upgrade seemlessly.
>
> My comments above really refer to when the new persister gets released.
I guess that still answers my question well enough; I was assuming the
new persister was part of 1.7, so whenever the new persister comes
out, I'll have to do that. Why was it pushed back? I thought it had
passed QA a few months ago.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Sep 29, 2009 3:38 pm |
|
|
|
Guest
|
On Tue, Sep 29, 2009 at 09:47:55AM -0500, tsuraan wrote:
> I guess that still answers my question well enough; I was assuming the
> new persister was part of 1.7, so whenever the new persister comes
> out, I'll have to do that. Why was it pushed back? I thought it had
> passed QA a few months ago.
During the process of going through QA, we realised that mnesia is too
flawed for us to use it as part of the persister. We were never going
to store messages directly in mnesia, merely accountancy information.
However, when operating in "disc_copies", mnesia keeps the entire table
in RAM, and periodically writes the table out to disk. Unfortunately,
it just writes the entire table out to disk, which means that for a
large table with 99% static entries, but 1% rapidly changing entries,
it will rewrite vast amounts of data unnecessarily, whenever it decides
to write out the table. The other problem is that when running in
"disc_only_copies", the entire table is disk only, using dets. Dets has
an upper limit of 2GB of data, and performs very very badly - I've
never seen it push a disk past about 10MB/s. These problems together
have meant that in order to achieve the goals of zero per message RAM
cost, and a sensible bound on the number of times data is rewritten, we
have decided to move away from using mnesia for the persister.
Unfortunately, these issues were only realised fairly late in the day,
hence the delay whilst we do some redesigning and recoding.
Matthew
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Sep 29, 2009 5:42 pm |
|
|
|
Guest
|
> During the process of going through QA, we realised that mnesia is too
> flawed for us to use it as part of the persister. We were never going
> to store messages directly in mnesia, merely accountancy information.
> However, when operating in "disc_copies", mnesia keeps the entire table
> in RAM, and periodically writes the table out to disk. Unfortunately,
> it just writes the entire table out to disk, which means that for a
> large table with 99% static entries, but 1% rapidly changing entries,
> it will rewrite vast amounts of data unnecessarily, whenever it decides
> to write out the table. The other problem is that when running in
> "disc_only_copies", the entire table is disk only, using dets. Dets has
> an upper limit of 2GB of data, and performs very very badly - I've
> never seen it push a disk past about 10MB/s. These problems together
> have meant that in order to achieve the goals of zero per message RAM
> cost, and a sensible bound on the number of times data is rewritten, we
> have decided to move away from using mnesia for the persister.
I'm not sure if you saw the Dukes of Erl mnesiaex project
(http://code.google.com/p/mnesiaex). I'm guessing you're well on your
way with a different strategy, but I've found tokyo to be really high
performance, and I'd be curious to hear how mnesia with a tokyo
backend works in a real project like rabbit.
> Unfortunately, these issues were only realised fairly late in the day,
> hence the delay whilst we do some redesigning and recoding.
That's always depressing. Good luck getting this thing working; I'm
sure looking forward to it 
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
