|
|
| Author |
Message |
|
| Guest |
 Posted: Wed May 07, 2008 3:15 pm |
 |
|
|
Guest
|
Hi all
I have looked at the FAQs, and the mailing list about this topic. This
is what I know so far:
1. I have an Erlang consumer (using the RabbitMQ Erlang client) that
has registered using a basic.consume.
2. The consumer often gets flooded with messages from the producer
when it does some time-based operations. In Erlang, this means that
the consumer's Erlang message queue grows to a couple hundred
messages, which is not what I want. I want the producer to stop
sending messages when the consumer is too busy.
Messages must not be lost and are marked persistent, so I want them to
stay on disk until the consumer acknowledges successful processing. Am
I correct to assume RabbitMQ does this if I have marked the queue as
persistent and have not replied to the basic.deliver with an ack?
3. The channel.flow object is not currently implemented in RabbitMQ; I
think this would be perfect to do this.
4. IIRC, a suggestion was for the consumer to basic.cancel itself
until the load reduces, and then re-subscribe (basic.consume) once the
overflow condition has abated. This seems heavy-handed, and possibly
resource-intensive, especially since it might happen relatively
frequently.
I'd appreciate any suggestions or clarifications.
Regards,
Edwin Fine
Fine Computer Consultants
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed May 07, 2008 9:40 pm |
|
|
|
Guest
|
Edwin,
Edwin Fine wrote:
> 2. The consumer often gets flooded with messages from the producer
> when it does some time-based operations. In Erlang, this means that
> the consumer's Erlang message queue grows to a couple hundred
> messages, which is not what I want. I want the producer to stop
> sending messages when the consumer is too busy.
Do you require end-to-end flow control, i.e. you need to throttle the
publisher of the messages when the consumer cannot keep up, or just flow
control between the broker and the consumer?
> Messages must not be lost and are marked persistent, so I want them to
> stay on disk until the consumer acknowledges successful processing. Am
> I correct to assume RabbitMQ does this if I have marked the queue as
> persistent and have not replied to the basic.deliver with an ack?
The messages themselves need to be marked as persistent, and the queue
needs to be marked as durable.
> 3. The channel.flow object is not currently implemented in RabbitMQ; I
> think this would be perfect to do this.
channel.flow is rather imperfect  For starters, it operates on an
entire channel, whereas it looks like you want it to operate on a
per-consumer basis.
> 4. IIRC, a suggestion was for the consumer to basic.cancel itself
> until the load reduces, and then re-subscribe (basic.consume) once the
> overflow condition has abated. This seems heavy-handed, and possibly
> resource-intensive, especially since it might happen relatively
> frequently.
The frequency is unlikely to be an issue - basic.consume/cancel are
pretty efficient.
The more serious problem may be prefetching. By the time your consumer
wants to start throttling there may already be a large number of
messages in flight. The client will still receive these after issuing a
basic.cancel, or, for that matter, channel.flow.
You can address that by switching to use basic.get, though that is
non-blocking.
Matthias
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
