Erlang/OTP Forums

Author Message

<  RabbitMQ mailing list  ~  Process N messages / VM concurrently

Guest
Posted: Wed Jul 02, 2008 6:58 pm Reply with quote
Guest
Dear list,

I'm new to AMQP and RabbitMQ and like to solve the following problem.
I am going to deploy a couple of JVMs on different nodes. Depending on
certain characteristics of these nodes they will be capable of
handling a node-specific N of messages concurrently. For each message
processed a worker Thread is used (probably using Java's Executor
framework). A message specific to the outcome of the work done in the
thread is put into the queue after the job is done and the transaction
is finished / the original message is acknowledged (I'm still a bit
unsure of the appropriate semantics here).

Is there a recommended strategy how to combine the Executor framework
of Java 5+ with the BlockingConsumer of Rabbit? I'm wiring the
application using Spring and use the latest version of RabbitMQ.

Best regards,


Joern
--

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist
0x6e6562
Posted: Thu Jul 03, 2008 3:36 pm Reply with quote
User Joined: 12 Jul 2007 Posts: 250
Joern,

On Wed, Jul 2, 2008 at 7:58 PM, Joern <opendev@gmail.com> wrote:
> I'm new to AMQP and RabbitMQ and like to solve the following problem.
> I am going to deploy a couple of JVMs on different nodes. Depending on
> certain characteristics of these nodes they will be capable of
> handling a node-specific N of messages concurrently. For each message
> processed a worker Thread is used (probably using Java's Executor
> framework). A message specific to the outcome of the work done in the
> thread is put into the queue after the job is done and the transaction
> is finished / the original message is acknowledged (I'm still a bit
> unsure of the appropriate semantics here).
>
> Is there a recommended strategy how to combine the Executor framework
> of Java 5+ with the BlockingConsumer of Rabbit? I'm wiring the
> application using Spring and use the latest version of RabbitMQ.

I'm not sure about the Executor framework but I think one thing that
you may want to consider is binding one AMQP channel to each worker
thread. The connection is thread safe and hence can be shared, but the
channel is designed to be the smallest unit of parallelism.
Furthermore, they are not threadsafe for this very reason.

Whilst it is possible to spawn a new thread in your own consumer for
each message, but going down the multi-channel route is more in line
with the AMQP model.

HTH,

Ben

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist
View user's profile Send private message
alexis
Posted: Thu Jul 03, 2008 3:41 pm Reply with quote
User Joined: 06 Sep 2007 Posts: 80 Location: London
Joern

>> Is there a recommended strategy how to combine the Executor framework
>> of Java 5+ with the BlockingConsumer of Rabbit? I'm wiring the
>> application using Spring and use the latest version of RabbitMQ.

To add one thing to Ben's reply, may I request that you share with us
your experiences of using Spring with RabbitMQ BlockingConsumer
please. Other Spring users may find this beneficial when looking at
RabbitMQ.

Cheers

alexis



--
Alexis Richardson
+44 20 7617 7339 (UK)
+44 77 9865 2911 (cell)
+1 650 206 2517 (US)

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist
View user's profile Send private message Yahoo Messenger
0x6e6562
Posted: Thu Jul 03, 2008 3:52 pm Reply with quote
User Joined: 12 Jul 2007 Posts: 250
Joern,

On Thu, Jul 3, 2008 at 4:40 PM, Alexis Richardson
<alexis.richardson@cohesiveft.com> wrote:
> To add one thing to Ben's reply, may I request that you share with us
> your experiences of using Spring with RabbitMQ BlockingConsumer
> please. Other Spring users may find this beneficial when looking at
> RabbitMQ.

I think I may have misunderstood your question. Were you perhaps
referring to the QueueingConsumer class that buffers messages into a
client side queue?

Ben

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 1 of 1
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum