| Author |
Message |
|
| Guest |
 Posted: Sat May 10, 2008 12:03 am |
 |
|
|
Guest
|
Thanks, Ben, I will take a look and give you some feedback.
In the meantime, I have done the following:
- Changed my consumer code (I use the term "consumer" loosely as "anything that eats the output of a producer") to use basic.get instead of basic.consume. Actually, it's set up so that I can select basic.get or basic.consume behavior at run-time. I didn't want to throw away working basic.consume code
- Changed the process that creates consumers so that it now creates one channel per consumer. Previously, there was one channel only for all consumers. One-channel-per-consumer was the only way I could get the code to work with the network client; in the one-channel scenario I was getting back responses to messages destined for different consumers. I assume that with your changes I will be able to again use one channel for all consumers.
- I tested with 50 queues (each with its own consumer and channel) and it seemed reasonably performant, even with the get. I need to try a full-blast test soon.
Again, thanks for your help and patience, and do let me know if you see any merit in the rbmq_admin module I wrote, or if maybe you guys are releasing an official one.
Regards,
Ed
On Fri, May 9, 2008 at 7:07 PM, Ben Hood <0x6e6562@gmail.com (0x6e6562@gmail.com)> wrote:
Quote: Ed,
On 8 May 2008, at 21:11, Ben Hood wrote:
Quote:
On 8 May 2008, at 15:04, Edwin Fine wrote:
Quote: Thanks for looking into this. In the "real" code, all the setup is actually done within a single Erlang process, and the only thing the consumers do is handle the basic.deliver, as well as any basic.consume_ok |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Mon May 12, 2008 6:07 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Ed,
On 10 May 2008, at 01:02, Edwin Fine wrote:
Quote: Thanks, Ben, I will take a look and give you some feedback.
In the meantime, I have done the following:
- Changed my consumer code (I use the term "consumer" loosely as "anything that eats the output of a producer") to use basic.get instead of basic.consume. Actually, it's set up so that I can select basic.get or basic.consume behavior at run-time. I didn't want to throw away working basic.consume code
- Changed the process that creates consumers so that it now creates one channel per consumer. Previously, there was one channel only for all consumers. One-channel-per-consumer was the only way I could get the code to work with the network client; in the one-channel scenario I was getting back responses to messages destined for different consumers. I assume that with your changes I will be able to again use one channel for all consumers.
- I tested with 50 queues (each with its own consumer and channel) and it seemed reasonably performant, even with the get. I need to try a full-blast test soon.
I have now commited fix 2 of 3 to the mtn repo which addresses the issue of not being able to subscribe concurrently. So the 2 issues you mention here should be addressed.
The outstanding issue is to close the writer down properly in the network case.
HTH,
Ben
Post received from mailinglist |
|
|
| Back to top |
 
|
| Guest |
| Posted: Mon May 12, 2008 7:57 pm |
|
|
|
Guest
|
Ben,
I was monitoring my running system, and unless I am mistaken, it looks as if amqp_network_connection:reader_loop/4 has a non-tail recursive call, which is making it eat stack and heap like there's no tomorrow. There's a call to gen_tcp:close(Sock) at the end of the function. I suggest you move this to the last line of start_reader as shown:
start_reader(Sock, FramingPid) ->
|
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Mon May 12, 2008 8:45 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Ed,
On 12 May 2008, at 20:56, Edwin Fine wrote:
> Ben,
>
> I was monitoring my running system, and unless I am mistaken, it
> looks as if amqp_network_connection:reader_loop/4 has a non-tail
> recursive call, which is making it eat stack and heap like there's
> no tomorrow. There's a call to gen_tcp:close(Sock) at the end of the
> function. I suggest you move this to the last line of start_reader
> as shown:
Fair point. I've applied this change. Thanks for pointing this one out.
> The outstanding issue is to close the writer down properly in the
> network case.
I've fixed the issue with the network writer not being shut down
properly and it's in mtn.
HTH,
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon May 12, 2008 9:16 pm |
|
|
|
Guest
|
Ben,
Thanks for being so responsive to input. It's really great and much appreciated.
Regards,
Ed
On Mon, May 12, 2008 at 4:44 PM, Ben Hood <0x6e6562@gmail.com (0x6e6562@gmail.com)> wrote:
Quote: Ed,
On 12 May 2008, at 20:56, Edwin Fine wrote:
Quote: Ben,
I was monitoring my running system, and unless I am mistaken, it looks as if amqp_network_connection:reader_loop/4 has a non-tail recursive call, which is making it eat stack and heap like there's no tomorrow. There's a call to gen_tcp:close(Sock) at the end of the function. I suggest you move this to the last line of start_reader as shown:
Fair point. I've applied this change. Thanks for pointing this one out.
Quote: The outstanding issue is to close the writer down properly in the network case.
I've fixed the issue with the network writer not being shut down properly and it's in mtn.
HTH,
Ben
Post received from mailinglist |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Thu May 15, 2008 11:37 am |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Ed,
On 10 May 2008, at 01:02, Edwin Fine wrote:
Quote: Thanks, Ben, I will take a look and give you some feedback.
In the meantime, I have done the following:
- Changed my consumer code (I use the term "consumer" loosely as "anything that eats the output of a producer") to use basic.get instead of basic.consume. Actually, it's set up so that I can select basic.get or basic.consume behavior at run-time. I didn't want to throw away working basic.consume code
- Changed the process that creates consumers so that it now creates one channel per consumer. Previously, there was one channel only for all consumers. One-channel-per-consumer was the only way I could get the code to work with the network client; in the one-channel scenario I was getting back responses to messages destined for different consumers. I assume that with your changes I will be able to again use one channel for all consumers.
- I tested with 50 queues (each with its own consumer and channel) and it seemed reasonably performant, even with the get. I need to try a full-blast test soon.
Again, thanks for your help and patience, and do let me know if you see any merit in the rbmq_admin module I wrote, or if maybe you guys are releasing an official one.
Sorry for not responding to this sooner, the offer is appreciated. Having said that, part of the functionality you have written is a wrapper around the rabbit_access_control module, which is pretty much what the rabbit_control module is (see the rabbitmqctl script). That's not to say there is no merit in improving the whole management aspect of RabbitMQ, which I think is the important point you making with the rbmq_admin module you have written. I know there have been a few posts about the reshaping the current management |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
