| Author |
Message |
|
| Guest |
 Posted: Wed May 07, 2008 5:58 pm |
 |
|
|
Guest
|
Using Rabbit 1.3.0, Erlang client, Ubuntu Linux Feisty, Erlang R12B-2 64-bit
Consumer channel crashes as follows when I do the following:
1. Subscribe (basic.consume)
2. Unsubscribe (basic.cancel)
The channel crashes handling the basic.cancel_ok. I can't figure out
why. The channel is under some load (about 100 - 130 messages/second).
I haven't yet built a standalone test case to try to reproduce this,
but it happens consistently in my code.
The code to subscribe is:
#'basic.consume_ok'{consumer_tag = ConsumerTag} =
amqp_channel:call(Channel, BasicConsume, self())
and this works fine.
The code to cancel is
BasicCancel = #'basic.cancel'{consumer_tag = ConsumerTag, nowait = false},
#'basic.cancel_ok'{consumer_tag = ConsumerTag} =
amqp_channel:call(Channel, BasicCancel).
The problem seems to be
** {function_clause,
[{gen_server,reply,
[<<>>,
{'basic.cancel_ok',<<"XHG.DELIVERY.Q.HTTP031.HC031">>}]},
What am I doing wrong? Any ideas?
Regards,
Edwin Fine
----------------------------------
=ERROR REPORT==== 7-May-2008::13:42:16 ===
** Generic server <0.138.0> terminating
** Last message in was {method,
{'basic.cancel_ok',
<<"XHG.DELIVERY.Q.HTTP031.HC031">>},
none}
** When Server state == {channel_state,1,<0.132.0>,<0.136.0>,<0.139.0>,
#Fun<amqp_network_driver.do.2>,
#Fun<amqp_network_driver.do.3>,<<>>,<<>>,false,
undefined,
{dict,48,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
[]},
{{[],
[[<<"XHG.DELIVERY.Q.HTTP016.HC016">>|
<0.145.0>],
[<<"XHG.DELIVERY.Q.HTTP038.HC038">>|
<0.295.0>],
[<<"XHG.DELIVERY.Q.HTTP009.HC009">>|
<0.358.0>],
[<<"XHG.DELIVERY.Q.HTTP023.HC023">>|
<0.522.0>],
[<<"XHG.DELIVERY.Q.HTTP001.HC001">>|
<0.905.0>],
[<<"XHG.DELIVERY.Q.HTTP030.HC030">>|
<0.1109.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP002.HC002">>|
<0.150.0>],
[<<"XHG.DELIVERY.Q.HTTP017.HC017">>|
<0.1014.0>],
[<<"XHG.DELIVERY.Q.HTTP024.HC024">>|
<0.1062.0>],
[<<"XHG.DELIVERY.Q.HTTP031.HC031">>|
<0.1115.0>],
[<<"XHG.DELIVERY.Q.HTTP039.HC039">>|
<0.1170.0>],
[<<"XHG.DELIVERY.Q.HTTP046.HC046">>|
<0.1220.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP003.HC003">>|
<0.919.0>],
[<<"XHG.DELIVERY.Q.HTTP010.HC010">>|
<0.967.0>],
[<<"XHG.DELIVERY.Q.HTTP018.HC018">>|
<0.1025.0>],
[<<"XHG.DELIVERY.Q.HTTP025.HC025">>|
<0.1073.0>],
[<<"XHG.DELIVERY.Q.HTTP032.HC032">>|
<0.1118.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP019.HC019">>|
<0.154.0>],
[<<"XHG.DELIVERY.Q.HTTP033.HC033">>|
<0.354.0>],
[<<"XHG.DELIVERY.Q.HTTP004.HC004">>|
<0.927.0>],
[<<"XHG.DELIVERY.Q.HTTP011.HC011">>|
<0.970.0>],
[<<"XHG.DELIVERY.Q.HTTP026.HC026">>|
<0.1076.0>],
[<<"XHG.DELIVERY.Q.HTTP040.HC040">>|
<0.1181.0>],
[<<"XHG.DELIVERY.Q.HTTP048.HC048">>|
<0.1233.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP049.HC049">>|
<0.365.0>],
[<<"XHG.DELIVERY.Q.HTTP005.HC005">>|
<0.930.0>],
[<<"XHG.DELIVERY.Q.HTTP012.HC012">>|
<0.980.0>],
[<<"XHG.DELIVERY.Q.HTTP027.HC027">>|
<0.1088.0>],
[<<"XHG.DELIVERY.Q.HTTP034.HC034">>|
<0.1130.0>],
[<<"XHG.DELIVERY.Q.HTTP041.HC041">>|
<0.1185.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP006.HC006">>|
<0.940.0>],
[<<"XHG.DELIVERY.Q.HTTP013.HC013">>|
<0.988.0>],
[<<"XHG.DELIVERY.Q.HTTP020.HC020">>|
<0.1039.0>],
[<<"XHG.DELIVERY.Q.HTTP028.HC028">>|
<0.1092.0>],
[<<"XHG.DELIVERY.Q.HTTP035.HC035">>|
<0.1147.0>],
[<<"XHG.DELIVERY.Q.HTTP042.HC042">>|
<0.1188.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP050.HC050">>|
<0.226.0>],
[<<"XHG.DELIVERY.Q.HTTP007.HC007">>|
<0.299.0>],
[<<"XHG.DELIVERY.Q.HTTP014.HC014">>|
<0.998.0>],
[<<"XHG.DELIVERY.Q.HTTP021.HC021">>|
<0.1042.0>],
[<<"XHG.DELIVERY.Q.HTTP029.HC029">>|
<0.1097.0>],
[<<"XHG.DELIVERY.Q.HTTP036.HC036">>|
<0.1151.0>],
[<<"XHG.DELIVERY.Q.HTTP043.HC043">>|
<0.1202.0>]],
[],
[[<<"XHG.DELIVERY.Q.HTTP008.HC008">>|
<0.953.0>],
[<<"XHG.DELIVERY.Q.HTTP015.HC015">>|
<0.1001.0>],
[<<"XHG.DELIVERY.Q.HTTP022.HC022">>|
<0.1049.0>],
[<<"XHG.DELIVERY.Q.HTTP037.HC037">>|
<0.1153.0>],
[<<"XHG.DELIVERY.Q.HTTP044.HC044">>|
<0.1205.0>]]}}}}
** Reason for termination ==
** {function_clause,
[{gen_server,reply,
[<<>>,
{'basic.cancel_ok',<<"XHG.DELIVERY.Q.HTTP031.HC031">>}]},
{amqp_channel,rpc_bottom_half,2},
{gen_server,handle_msg,5},
{proc_lib,init_p,5}]}
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Wed May 07, 2008 8:30 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Edwin,
On 7 May 2008, at 18:57, Edwin Fine wrote:
> Using Rabbit 1.3.0, Erlang client, Ubuntu Linux Feisty, Erlang
> R12B-2 64-bit
>
> Consumer channel crashes as follows when I do the following:
>
> 1. Subscribe (basic.consume)
> 2. Unsubscribe (basic.cancel)
>
> The channel crashes handling the basic.cancel_ok. I can't figure out
> why. The channel is under some load (about 100 - 130 messages/second).
> I haven't yet built a standalone test case to try to reproduce this,
> but it happens consistently in my code.
>
> The code to subscribe is:
>
> #'basic.consume_ok'{consumer_tag = ConsumerTag} =
> amqp_channel:call(Channel, BasicConsume, self())
>
> and this works fine.
>
> The code to cancel is
>
> BasicCancel = #'basic.cancel'{consumer_tag = ConsumerTag, nowait
> = false},
> #'basic.cancel_ok'{consumer_tag = ConsumerTag} =
> amqp_channel:call(Channel, BasicCancel).
>
> The problem seems to be
>
> ** {function_clause,
> [{gen_server,reply,
> [<<>>,
> {'basic.cancel_ok',<<"XHG.DELIVERY.Q.HTTP031.HC031">>}]},
>
>
> What am I doing wrong? Any ideas?
What you are seeing is a symptom of an error in the RPC handling.
Essentially, the channel process is receiving a notification that it
thinks it has already sent to the invoking process.
It would be really good if you could send some code that might help me
reproduce this myself.
In the meantime I will look into this and see if I can come up with a
solution.
HTH,
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
 
|
| 0x6e6562 |
| Posted: Wed May 07, 2008 9:03 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Just quickly before you start doing stuff:
Does it make a difference if you just the direct client rather than the TCP client?
Ben
On 7 May 2008, at 21:50, Edwin Fine wrote:
Quote: Thanks for the response, Ben. I will try to create a standalone program that reproduces |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Wed May 07, 2008 9:29 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
BTW Edwin, I reposting this to the list, so it can be followed.
On 7 May 2008, at 21:50, Edwin Fine wrote:
Quote: Thanks for the response, Ben. I will try to create a standalone program that reproduces |
|
|
| Back to top |
|
| Guest |
| Posted: Wed May 07, 2008 10:06 pm |
|
|
|
Guest
|
Ben,
Umm, I haven't used the direct client at all because I have a whole complex application set up and it would have to run on the same Erlang node as RabbitMQ, if I understand how the direct client works. I would have to attach to the Rabbit node, set up all the right paths, and then start my application. I'll try that and if there are not too many complications (e.g. something silly like something in my application might expect a specific node name), I will let you know what happens.
Thanks for your attention!
Regards,
Edwin
On Wed, May 7, 2008 at 5:03 PM, Ben Hood <0x6e6562@gmail.com (0x6e6562@gmail.com)> wrote:
Quote: Just quickly before you start doing stuff:
Does it make a difference if you just the direct client rather than the TCP client?
Ben
On 7 May 2008, at 21:50, Edwin Fine wrote:
Quote: Thanks for the response, Ben. I will try to create a standalone program that reproduces |
|
|
| Back to top |
|
| Guest |
| Posted: Wed May 07, 2008 10:12 pm |
|
|
|
Guest
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Wed May 07, 2008 10:16 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Edwin,
On 7 May 2008, at 23:05, Edwin Fine wrote:
> Umm, I haven't used the direct client at all because I have a whole
> complex application set up and it would have to run on the same
> Erlang node as RabbitMQ, if I understand how the direct client works.
That's right. It uses native Erlang message passing instead of TCP in
a way that is transparent to the client. As a direct client you can
think of Rabbit as being an embedded broker, like the way ActiveMQ is
often run in the same process and the client.
> I would have to attach to the Rabbit node, set up all the right
> paths, and then start my application. I'll try that and if there are
> not too many complications (e.g. something silly like something in
> my application might expect a specific node name), I will let you
> know what happens.
You don't necessarily need to attach to a node. You *should* be able
to use the direct client in exactly the same way as the TCP client.
From the client's prespective the API calls are exactly the same. All
you need to do is to boot Rabbit in the same VM and the client
process, which means just adding a few arguments to the Erlang
runtime, e.g.
erl -pa $PATH_TO_RABBIT_EBIN_DIR $PATH_TO_YOUR_CLIENT_EBIN_DIR -mnesia
dir $MNESIA_DIR -boot start_sasl -s rabbit
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed May 07, 2008 10:26 pm |
|
|
|
Guest
|
Quote: You don't necessarily need to attach to a node. |
|
|
| Back to top |
|
| Guest |
| Posted: Thu May 08, 2008 4:22 am |
|
|
|
Guest
|
Edwin Fine wrote:
> ***** Actually, I believe I have turned off the no_ack flag and am
> actively sending acks when appropriate. [...]
> However, I must have done something wrong because RMQ continues to send
> even if I haven't acked.
You haven't done anything wrong. The no_ack flag controls whether the
server expects to see acks from the client, or just auto-acks the
messages itself. The flag does not affect the flow of messages.
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Thu May 08, 2008 5:16 am |
|
|
|
Guest
|
That's true. It seems that all that happens is that the messages (which are persistent in my case) are maintained in the queue and do not get removed until the acks are received. However, all the messages are still sent. I assume they would be retransmitted at some point. It's quite interesting  .
Ed
On Thu, May 8, 2008 at 12:22 AM, Matthias Radestock <matthias@lshift.net (matthias@lshift.net)> wrote:
|
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Thu May 08, 2008 10:07 am |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Ed,
On 8 May 2008, at 06:13, Edwin Fine wrote:
Quote:
Well, I have put most of the code I am using into a test program (two modules). This code is not for the mailing list's consumption, if that's ok with you. The problem is, I can't even make the code work for more than 1 consumer and I don't know why. I think I am too tired. In any case, the two modules included are:
- rabbit_chan_crash.erl - The actual test program, which contains most of the Rabbit-related working code;
By doing this though, you have uncovered a bug in the synchronous RPC handling code in the client that doesn't protect against this properly. So I've fixed this bug and will send it down to the repository. |
|
|
| Back to top |
|
| Guest |
| Posted: Thu May 08, 2008 2:05 pm |
|
|
|
Guest
|
Ben,
Thanks for looking into this. In the "real" code, all the setup is actually done within a single Erlang process, and the only thing the consumers do is handle the basic.deliver, as well as any basic.consume_ok |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Thu May 08, 2008 8:12 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Ed,
On 8 May 2008, at 15:04, Edwin Fine wrote:
Quote: Ben,
Thanks for looking into this. In the "real" code, all the setup is actually done within a single Erlang process, and the only thing the consumers do is handle the basic.deliver, as well as any basic.consume_ok |
|
|
| Back to top |
|
| Guest |
| Posted: Thu May 08, 2008 9:43 pm |
|
|
|
Guest
|
Edwin,
Edwin Fine wrote:
> That's true. It seems that all that happens is that the messages (which
> are persistent in my case) are maintained in the queue and do not get
> removed until the acks are received. However, all the messages are still
> sent. I assume they would be retransmitted at some point.
Yep. The unack'ed messages will get retransmitted or requeued after the
client calls basic.recover, or the channel or connection gets closed.
Matthias.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| 0x6e6562 |
| Posted: Fri May 09, 2008 11:08 pm |
|
|
|
User
Joined: 12 Jul 2007
Posts: 250
|
Ed,
On 8 May 2008, at 21:11, Ben Hood wrote:
>
> On 8 May 2008, at 15:04, Edwin Fine wrote:
>> Thanks for looking into this. In the "real" code, all the setup is
>> actually done within a single Erlang process, and the only thing
>> the consumers do is handle the basic.deliver, as well as any
>> basic.consume_ok or basic.cancel_ok messages that may arise. In
>> the test code, I rearranged some things specifically to be done
>> concurrently, which I now see was a mistake. However, this is a bug
>> in the test code, not in the production code. I will change the
>> test code so that there is no concurrent work being done on a
>> channel other than responding with basic.ack to basic.deliver
>> messages.
>
> I've begun a discussion on this topic and it looks like we will add
> the intelligence to the amqp_channel module to be able to serialize
> concurrent RPC requests to the channel. I understand that you have a
> workaround anyway, but on reflection, it seems like a good idea to
> put this capability into the client. will let you know when it is
> done.
This feature is now implemented in the trunk of the repo. There is
also a test case to test the serialization of concurrent RPC requests.
I will now look into the next 2 issues you raised.
BTW, I am also in the process of moving the mtn repo to hg, so this
might happen very soon as well.
HTH,
Ben
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
Page 1 of 2
Goto page 1, 2 Next
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
