| Author |
Message |
|
| Guest |
 Posted: Sun Oct 04, 2009 7:44 pm |
 |
|
|
Guest
|
I had this problem earlier today and it seems to continue:
I have rabbitmq working great and the producer alongside but then out of nowhere all of a sudden the producer consumes my complete 4 GB of RAM all at once causing swapping to occur amongst other things.
Any ideas why my producer would suddenly jump and consume all the RAM available? I am getting a serious amount of volume of items hitting the queue.
I run rabbitmq in persistence alongside with the producer and using acking for my producers.
Suhail
Post received from mailinglist |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Sun Oct 04, 2009 9:18 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Suhail,
On Sun, Oct 4, 2009 at 8:43 PM, Suhail Doshi <suhail@mixpanel.com> wrote:
> Any ideas why my producer would suddenly jump and consume all the RAM
> available? I am getting a serious amount of volume of items hitting the
> queue.
In general, a producer is a client process running outside of
RabbitMQ, so it is difficult to see how RabbitMQ is affecting the
memory consumption of this process. Maybe you can post a cut down
version of your producer application that reproduces the symptoms.
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
 
|
| Guest |
| Posted: Mon Oct 05, 2009 1:05 am |
|
|
|
Guest
|
Problem is, it only happens every 5+ hours and not sure how to duplicate it but here is a crash dump:
=erl_crash_dump:0.1
Sat Oct |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Oct 05, 2009 1:08 am |
|
|
|
Guest
|
Here's part of the queue code we use:
-export([amqp_lifecycle/0, send_message/5, log/2]).
-include_lib("rabbitmq-erlang-client/rabbitmq_server/include/rabbit.hrl").
-include_lib("rabbitmq-erlang-client/rabbitmq_server/include/rabbit_framing.hrl").
-include("rabbitmq-erlang-client/include/amqp_client.hrl").
-record(rabbit_info, {channel, ticket, exchange, routing_key}).
amqp_lifecycle() ->
|
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Mon Oct 05, 2009 11:03 am |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Suhail,
On Mon, Oct 5, 2009 at 2:08 AM, Suhail Doshi <suhail@mixpanel.com> wrote:
> Here's part of the queue code we use:
A few observations:
- Tickets and realms are toast in Rabbit now, so you can nuke that code;
- You set a lot of fields that are identical to the default values,
hence adding unnecessary extra code;
- You are also using a fairly old version of the Erlang client - you
are less likely to get any serious feedback when you are not using the
latest version;
In general the idea is to write the most concise piece of code that
you can that replicates the issue, even if it does take 5 hours to
reproduce - we can run the code for 5 hours locally if we need to as
well. Otherwise we have little hope of being able to help you.
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Oct 05, 2009 9:24 pm |
|
|
|
Guest
|
I have actually traced to the error_logger process:
State: Waiting
Name: error_logger
Spawned as: proc_lib:init_p/5
Spawned by: <0.1.0>
Started: Mon Oct |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Mon Oct 05, 2009 9:31 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Suhail,
On Mon, Oct 5, 2009 at 10:23 PM, Suhail Doshi <suhail@mixpanel.com> wrote:
> Anyone else seen this problem? Seems like an erlang thing not a rabbitmq
> thing.
I've never seen it myself. But if you don't send some concise runnable
code that reproduces the issue, you are not going to get much help
much from anybody.
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Oct 06, 2009 10:45 am |
|
|
|
Guest
|
|
| Back to top |
|
| Guest |
| Posted: Wed Oct 07, 2009 10:28 am |
|
|
|
Guest
|
Running it now, will let you know the results of it--so far so good.
Suhail
On Tue, Oct 6, 2009 at 3:45 AM, Alexis Richardson <alexis.richardson@gmail.com (alexis.richardson@gmail.com)> wrote:
Quote: Suhail
Have you tried running this with the latest erlang client yet?
alexis
On Mon, Oct 5, 2009 at 10:30 PM, Ben Hood <0x6e6562@gmail.com (0x6e6562@gmail.com)> wrote:
> Suhail,
>
> On Mon, Oct 5, 2009 at 10:23 PM, Suhail Doshi <suhail@mixpanel.com (suhail@mixpanel.com)> wrote:
>> Anyone else seen this problem? Seems like an erlang thing not a rabbitmq
>> thing.
>
> I've never seen it myself. But if you don't send some concise runnable
> code that reproduces the issue, you are not going to get much help
> much from anybody.
>
> Ben
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss@lists.rabbitmq.com (rabbitmq-discuss@lists.rabbitmq.com)
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
--
http://mixpanel.com
Blog: http://blog.mixpanel.com
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Oct 07, 2009 7:01 pm |
|
|
|
Guest
|
Seems to have fixed the problem, I guess it was a bug in an older version of erlang.
Suhail
On Wed, Oct 7, 2009 at 3:27 AM, Suhail Doshi <suhail@mixpanel.com (suhail@mixpanel.com)> wrote:
Quote: Running it now, will let you know the results of it--so far so good.
Suhail
On Tue, Oct 6, 2009 at 3:45 AM, Alexis Richardson <alexis.richardson@gmail.com (alexis.richardson@gmail.com)> wrote:
Quote: Suhail
Have you tried running this with the latest erlang client yet?
alexis
On Mon, Oct 5, 2009 at 10:30 PM, Ben Hood <0x6e6562@gmail.com (0x6e6562@gmail.com)> wrote:
> Suhail,
>
> On Mon, Oct 5, 2009 at 10:23 PM, Suhail Doshi <suhail@mixpanel.com (suhail@mixpanel.com)> wrote:
>> Anyone else seen this problem? Seems like an erlang thing not a rabbitmq
>> thing.
>
> I've never seen it myself. But if you don't send some concise runnable
> code that reproduces the issue, you are not going to get much help
> much from anybody.
>
> Ben
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss@lists.rabbitmq.com (rabbitmq-discuss@lists.rabbitmq.com)
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
--
http://mixpanel.com
Blog: http://blog.mixpanel.com
--
http://mixpanel.com
Blog: http://blog.mixpanel.com
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Thu Oct 08, 2009 5:20 pm |
|
|
|
Guest
|
I replied to this too soon and sadly is still a problem. Will post code later.
Suhail
On Wed, Oct 7, 2009 at 12:00 PM, Suhail Doshi <suhail@mixpanel.com (suhail@mixpanel.com)> wrote:
Quote: Seems to have fixed the problem, I guess it was a bug in an older version of erlang.
Suhail
On Wed, Oct 7, 2009 at 3:27 AM, Suhail Doshi <suhail@mixpanel.com (suhail@mixpanel.com)> wrote:
Quote: Running it now, will let you know the results of it--so far so good.
Suhail
On Tue, Oct 6, 2009 at 3:45 AM, Alexis Richardson <alexis.richardson@gmail.com (alexis.richardson@gmail.com)> wrote:
Quote: Suhail
Have you tried running this with the latest erlang client yet?
alexis
On Mon, Oct 5, 2009 at 10:30 PM, Ben Hood <0x6e6562@gmail.com (0x6e6562@gmail.com)> wrote:
> Suhail,
>
> On Mon, Oct 5, 2009 at 10:23 PM, Suhail Doshi <suhail@mixpanel.com (suhail@mixpanel.com)> wrote:
>> Anyone else seen this problem? Seems like an erlang thing not a rabbitmq
>> thing.
>
> I've never seen it myself. But if you don't send some concise runnable
> code that reproduces the issue, you are not going to get much help
> much from anybody.
>
> Ben
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss@lists.rabbitmq.com (rabbitmq-discuss@lists.rabbitmq.com)
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
--
http://mixpanel.com
Blog: http://blog.mixpanel.com
--
http://mixpanel.com
Blog: http://blog.mixpanel.com
--
http://mixpanel.com
Blog: http://blog.mixpanel.com
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
