| Author |
Message |
|
| Guest |
 Posted: Wed Apr 30, 2008 4:40 pm |
 |
|
|
Guest
|
Hi,
I'm evaluating RabbitMQ as a replacement for our JMS code but I've run into
some slightly odd behaviour.
I've managed to replicate it in a simple example. So:
1. Run the HelloClient RabbitMQ example modified slightly so that it sends
20 messages instead of 1.
2. Run the HelloServer example but with a slight change to the code. Add a
Thread.sleep(10*1000) before the return in handleStringCall().
3. Now run a normal HelloServer example, without any sleep.
What I noticed here is that the normal example doesn't run at full speed.
It looks like the slow server is slowing down the normal one.
5. Kill the slow server (the one with the sleep) and the normal HelloServer
now runs at full speed and finishes the rest of the messages within the
second.
Any help would be great.
Thanks,
Dave
*********************************************************************
This communication contains confidential information, some or all of which may be privileged. It is for the intended recipient only and others must not disclose, distribute, copy, print or rely on this communication. If an addressing or transmission error has misdirected this communication, please notify the sender by replying to this e-mail and then delete the e-mail. E-mail sent to EDF Trading may be monitored by the company. Thank you.
EDF Trading Limited
80 Victoria Street, 3rd Floor, Cardinal Place, London, SW1E 5JL
A Company registered in England No. 4255974.
Switchboard: 020 7061 4000
EDF Trading Markets Limited is a member of the EDF Trading Limited Group and is authorised and regulated by the Financial Services Authority.
VAT number: GB 735 5479 07
*********************************************************************
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Thu May 01, 2008 6:16 pm |
|
|
|
Guest
|
David,
David.Corcoran@edftrading.com wrote:
> 1. Run the HelloClient RabbitMQ example modified slightly so that it sends
> 20 messages instead of 1.
> 2. Run the HelloServer example but with a slight change to the code. Add a
> Thread.sleep(10*1000) before the return in handleStringCall().
> 3. Now run a normal HelloServer example, without any sleep.
> What I noticed here is that the normal example doesn't run at full speed.
> It looks like the slow server is slowing down the normal one.
> 5. Kill the slow server (the one with the sleep) and the normal HelloServer
> now runs at full speed and finishes the rest of the messages within the
> second.
How exactly did you modify the HelloClient? Did you just put the original
System.out.println(service.stringCall(request));
in a 20-iteration loop?
If so, then you are issuing 20 RPCs sequentially, i.e. the client will
be waiting for the result of the previous call before issuing the next.
Each call will be handled by one of the two HelloServers you are
running. If a call gets handled by the "slow" server then it will take
10 seconds to complete. If it happens to get handled by the fast server
then it will complete nearly instantly. As soon as you stop the slow
server all remaining requests will get handled by the fast server and
hence complete quickly.
So the behaviour you are seeing is to be expected.
Were you trying to construct a load balancing test? If so you'd need to
be issuing RPCs *concurrently*. Also, the RpcServer wasn't designed for
load balancing - it will actually consume messages from the underlying
AMQP queue as fast as it can without waiting for them to be processed.
You'd have to change that.
Regards,
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri May 02, 2008 10:52 am |
|
|
|
Guest
|
Hi Matthias,
That makes perfect sense. Thanks for your help. I think my brain was a
little fried when I was writing the example. I was actually trying to write
a simple piece of code that exhibited a problem I was having and the
example I gave was obviously wrong.
The problem I was actually having I suspect is related to prefetching. I'll
try and explain it and perhaps you can confirm if it sounds likely?
I'm testing rabbitmq as a new messaging system for our job control. So I
write a job to the queue and it gets picked up by a worker and run.
Unfortunately each job can start more jobs, down to a level of about 3
deep. So with one job runner active this would lead to deadlock, as a job
would start a new job and block waiting for it to finish but no other job
runner would pick it up.
The way we work around this is when a parent job starts its own child jobs
and blocks, a new thread is started with a new channel pointing to the same
queue. Then when the parent job finishes the thread it was running in
finishes too. This way we only have 1 thread doing any work at a time.
In our current system for this to work I have to turn off prefetching by
the clients. Otherwise the first thread can start a job with dependencies,
then block, but the dependent jobs are prefetched into its local buffer
which leads to deadlock. With prefetching off another client can pick up
the job and work on it.
I think I've hit a problem with prefetching and RabbitMQ. Do you think this
might be the case?
Thanks for your help,
Dave
Matthias
Radestock
<matthias@lshift. To
net> David.Corcoran@edftrading.com
cc
01/05/2008 19:22 rabbitmq-discuss@lists.rabbitmq.com
Subject
Re: [rabbitmq-discuss] odd consumer
behaviour
David,
David.Corcoran@edftrading.com wrote:
> 1. Run the HelloClient RabbitMQ example modified slightly so that it
sends
> 20 messages instead of 1.
> 2. Run the HelloServer example but with a slight change to the code. Add
a
> Thread.sleep(10*1000) before the return in handleStringCall().
> 3. Now run a normal HelloServer example, without any sleep.
> What I noticed here is that the normal example doesn't run at full speed.
> It looks like the slow server is slowing down the normal one.
> 5. Kill the slow server (the one with the sleep) and the normal
HelloServer
> now runs at full speed and finishes the rest of the messages within the
> second.
How exactly did you modify the HelloClient? Did you just put the original
System.out.println(service.stringCall(request));
in a 20-iteration loop?
If so, then you are issuing 20 RPCs sequentially, i.e. the client will
be waiting for the result of the previous call before issuing the next.
Each call will be handled by one of the two HelloServers you are
running. If a call gets handled by the "slow" server then it will take
10 seconds to complete. If it happens to get handled by the fast server
then it will complete nearly instantly. As soon as you stop the slow
server all remaining requests will get handled by the fast server and
hence complete quickly.
So the behaviour you are seeing is to be expected.
Were you trying to construct a load balancing test? If so you'd need to
be issuing RPCs *concurrently*. Also, the RpcServer wasn't designed for
load balancing - it will actually consume messages from the underlying
AMQP queue as fast as it can without waiting for them to be processed.
You'd have to change that.
Regards,
Matthias.
*********************************************************************
This communication contains confidential information, some or all of which may be privileged. It is for the intended recipient only and others must not disclose, distribute, copy, print or rely on this communication. If an addressing or transmission error has misdirected this communication, please notify the sender by replying to this e-mail and then delete the e-mail. E-mail sent to EDF Trading may be monitored by the company. Thank you.
EDF Trading Limited
80 Victoria Street, 3rd Floor, Cardinal Place, London, SW1E 5JL
A Company registered in England No. 4255974.
Switchboard: 020 7061 4000
EDF Trading Markets Limited is a member of the EDF Trading Limited Group and is authorised and regulated by the Financial Services Authority.
VAT number: GB 735 5479 07
*********************************************************************
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri May 02, 2008 11:20 am |
|
|
|
Guest
|
David,
David.Corcoran@edftrading.com wrote:
> I think I've hit a problem with prefetching and RabbitMQ. Do you think this
> might be the case?
As I said in my previous email: "the RpcServer [...] will actually
consume messages from the underlying AMQP queue as fast as it can
without waiting for them to be processed."
So in effect the RpcServer is doing prefetching.
The easiest way to change that is to switch the way the RpcServer gets
the messages from a push to a pull model, i.e. using AMQP's basic.get
rather than basic.consume.
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri May 02, 2008 1:48 pm |
|
|
|
Guest
|
Hey,
Thanks again Matthias. That sounds perfect. In some other message queue
systems I've come across prefetching happens behind the scenes no matter
what methods you call and I was hoping that didn't happen with RabbitMq.
I'll give basic.get a try.
Regards,
Dave
Matthias
Radestock
<matthias@lshift. To
net> David.Corcoran@edftrading.com
Sent by: cc
rabbitmq-discuss- rabbitmq-discuss@lists.rabbitmq.com
bounces@lists.rab Subject
bitmq.com Re: [rabbitmq-discuss] odd consumer
behaviour
02/05/2008 12:19
David,
David.Corcoran@edftrading.com wrote:
> I think I've hit a problem with prefetching and RabbitMQ. Do you think
this
> might be the case?
As I said in my previous email: "the RpcServer [...] will actually
consume messages from the underlying AMQP queue as fast as it can
without waiting for them to be processed."
So in effect the RpcServer is doing prefetching.
The easiest way to change that is to switch the way the RpcServer gets
the messages from a push to a pull model, i.e. using AMQP's basic.get
rather than basic.consume.
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
*********************************************************************
This communication contains confidential information, some or all of which may be privileged. It is for the intended recipient only and others must not disclose, distribute, copy, print or rely on this communication. If an addressing or transmission error has misdirected this communication, please notify the sender by replying to this e-mail and then delete the e-mail. E-mail sent to EDF Trading may be monitored by the company. Thank you.
EDF Trading Limited
80 Victoria Street, 3rd Floor, Cardinal Place, London, SW1E 5JL
A Company registered in England No. 4255974.
Switchboard: 020 7061 4000
EDF Trading Markets Limited is a member of the EDF Trading Limited Group and is authorised and regulated by the Financial Services Authority.
VAT number: GB 735 5479 07
*********************************************************************
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri May 02, 2008 3:56 pm |
|
|
|
Guest
|
Hi Matthias,
basic.get works great. The jobs with nested dependencies run with no
problems.
The only problem is that Channel basicGet is a non-blocking operation so to
get my test running I'm polling and sleeping for a short amount of time if
I get null. Is there a blocking, non-prefetching, version of basicGet?
Thanks,
Dave
Matthias
Radestock
<matthias@lshift. To
net> David.Corcoran@edftrading.com
Sent by: cc
rabbitmq-discuss- rabbitmq-discuss@lists.rabbitmq.com
bounces@lists.rab Subject
bitmq.com Re: [rabbitmq-discuss] odd consumer
behaviour
02/05/2008 12:19
David,
David.Corcoran@edftrading.com wrote:
> I think I've hit a problem with prefetching and RabbitMQ. Do you think
this
> might be the case?
As I said in my previous email: "the RpcServer [...] will actually
consume messages from the underlying AMQP queue as fast as it can
without waiting for them to be processed."
So in effect the RpcServer is doing prefetching.
The easiest way to change that is to switch the way the RpcServer gets
the messages from a push to a pull model, i.e. using AMQP's basic.get
rather than basic.consume.
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
*********************************************************************
This communication contains confidential information, some or all of which may be privileged. It is for the intended recipient only and others must not disclose, distribute, copy, print or rely on this communication. If an addressing or transmission error has misdirected this communication, please notify the sender by replying to this e-mail and then delete the e-mail. E-mail sent to EDF Trading may be monitored by the company. Thank you.
EDF Trading Limited
80 Victoria Street, 3rd Floor, Cardinal Place, London, SW1E 5JL
A Company registered in England No. 4255974.
Switchboard: 020 7061 4000
EDF Trading Markets Limited is a member of the EDF Trading Limited Group and is authorised and regulated by the Financial Services Authority.
VAT number: GB 735 5479 07
*********************************************************************
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Sat May 03, 2008 5:45 am |
|
|
|
Guest
|
David,
David.Corcoran@edftrading.com wrote:
> basic.get works great. The jobs with nested dependencies run with no
> problems.
>
> The only problem is that Channel basicGet is a non-blocking operation so to
> get my test running I'm polling and sleeping for a short amount of time if
> I get null. Is there a blocking, non-prefetching, version of basicGet?
There is no blocking basicGet. However, there is a way of controlling
the prefetch window used by basicConsume, namely AMQP's basicQos method.
Unfortunately this feature is as yet unimplemented in RabbitMQ. It's on
our todo list.
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
