| Author |
Message |
< RabbitMQ mailing list ~ Publish with immediate for dynamic worker creation? |
| Guest |
 Posted: Wed Oct 07, 2009 11:28 am |
 |
|
|
Guest
|
Hi,
I'm wondering if I'm wandering the right path here. (I'm using 1.6.0 with erlang client.)
I was intending to use a basic.publish with mandatory and immediate set to true, and a return handler to allow me to add new workers reading from a queue.
I create a work queue with a return handler and a reply queue. I also create one initial worker, subscribing to the work queue with no_ack set to false.
I then publish a work message onto the work queue and the worker picks it up, does the work and publishes the answer back into the reply queue.
My intention was that my work message publish uses the immediate flag to generate a basic.return if the message cannot be immediately delivered. I thought that if my worker is busy it won't get the next message, leading to the basic.return. But I believe my thinking is wrong. It looks like the second work message is routed to the initial work consumer immediately, even if it is still busy.
Have I overlooked something obvious, or would I need to implement a 1 message flow control on the channel that the work consumer is using?
Thanks for any pointers,
Robby
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Oct 07, 2009 12:11 pm |
|
|
|
Guest
|
Hi Robert,
If you set the QOS on the consumer to 1 (as you say), then this should prevent the message being queued at the client.
Thanks,
Paul.
On Wed, Oct 7, 2009 at 12:28 PM, Robert Raschke <rtrlists@googlemail.com (rtrlists@googlemail.com)> wrote:
Quote: Hi,
I'm wondering if I'm wandering the right path here. (I'm using 1.6.0 with erlang client.)
I was intending to use a basic.publish with mandatory and immediate set to true, and a return handler to allow me to add new workers reading from a queue.
I create a work queue with a return handler and a reply queue. I also create one initial worker, subscribing to the work queue with no_ack set to false.
I then publish a work message onto the work queue and the worker picks it up, does the work and publishes the answer back into the reply queue.
My intention was that my work message publish uses the immediate flag to generate a basic.return if the message cannot be immediately delivered. I thought that if my worker is busy it won't get the next message, leading to the basic.return. But I believe my thinking is wrong. It looks like the second work message is routed to the initial work consumer immediately, even if it is still busy.
Have I overlooked something obvious, or would I need to implement a 1 message flow control on the channel that the work consumer is using?
Thanks for any pointers,
Robby
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com (rabbitmq-discuss@lists.rabbitmq.com)
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Oct 07, 2009 12:55 pm |
|
|
|
Guest
|
Superb.
Thank you. I got lost looking at flow control, which is wrong for my model.
Setting the QoS prefetch-count to 1 in my workers leads to my desired return notification. I can then set up a new worker and requeue the work message. Works like a charm.
Splendid,
Robby
On Wed, Oct 7, 2009 at 1:10 PM, Paul Jones <pauljones23@gmail.com (pauljones23@gmail.com)> wrote:
Quote: Hi Robert,
If you set the QOS on the consumer to 1 (as you say), then this should prevent the message being queued at the client.
Thanks,
Paul.
On Wed, Oct 7, 2009 at 12:28 PM, Robert Raschke <rtrlists@googlemail.com (rtrlists@googlemail.com)> wrote:
Quote:
Hi,
I'm wondering if I'm wandering the right path here. (I'm using 1.6.0 with erlang client.)
I was intending to use a basic.publish with mandatory and immediate set to true, and a return handler to allow me to add new workers reading from a queue.
I create a work queue with a return handler and a reply queue. I also create one initial worker, subscribing to the work queue with no_ack set to false.
I then publish a work message onto the work queue and the worker picks it up, does the work and publishes the answer back into the reply queue.
My intention was that my work message publish uses the immediate flag to generate a basic.return if the message cannot be immediately delivered. I thought that if my worker is busy it won't get the next message, leading to the basic.return. But I believe my thinking is wrong. It looks like the second work message is routed to the initial work consumer immediately, even if it is still busy.
Have I overlooked something obvious, or would I need to implement a 1 message flow control on the channel that the work consumer is using?
Thanks for any pointers,
Robby
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com (rabbitmq-discuss@lists.rabbitmq.com)
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Oct 07, 2009 1:24 pm |
|
|
|
Guest
|
Robert,
Robert Raschke wrote:
> Setting the QoS prefetch-count to 1 in my workers leads to my desired
> return notification. I can then set up a new worker and requeue the work
> message. Works like a charm.
Glad to hear you got things to work. It's an interesting and novel use
of the 'immediate' flag.
An alternative approach would be to monitor the queue length (e.g. by
issuing a passive queue.declare) and create/destroy workers based on
that information. That way queues can still do queuing, which helps in
smoothing out temporary spikes.
Regards,
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Oct 07, 2009 3:34 pm |
|
|
|
Guest
|
Hi Matthias,
On Wed, Oct 7, 2009 at 2:24 PM, Matthias Radestock <matthias@lshift.net (matthias@lshift.net)> wrote:
Quote: An alternative approach would be to monitor the queue length (e.g. by issuing a passive queue.declare) and create/destroy workers based on that information. That way queues can still do queuing, which helps in smoothing out temporary spikes.
Regards,
Matthias.
I was trying hard (maybe a little too hard  not to have any busy loops in my code.
Getting used to dynamic queues will take me a bit of time. It's ever so slightly too flexible. I've read about various usage scenarios in various blogs. But in general, they all tend to be a bit on the generic and vague side. Does anyone have pointers to real life writeups? I guess I should attempt to find time to write one myself, once my project is nearing completion.
Thanks,
Robby
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
