| Author |
Message |
|
| spgarbet |
Posted: Wed Aug 06, 2008 12:25 pm |
|
|
|
Joined: 06 Aug 2008
Posts: 4
|
What am I doing wrong? The program is firing off several processes which make http requests (http:request) to do some load testing. It usually only happens when the concurrent process count is above 60. Should this be on the bug forum?
Several of the processes die a horrible death and read off the following monologue:
Code:
=ERROR REPORT==== 5-Aug-2008::21:42:16 ===
** Generic server <0.22109.0> terminating
** Last message in was {tcp_closed,#Port<0.21371>}
** When Server state == {state,
{request,#Ref<0.0.1.250636>,<0.21236.0>,0,http,
{"127.0.0.1",5984},
"/",[],get,
{http_request_h,undefined,"keep-alive",
undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,"127.0.0.1",undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,
[],undefined,undefined,undefined,
undefined,"0",undefined,undefined,
undefined,undefined,undefined,undefined,
[]},
{[],[]},
{http_options,"HTTP/1.1",infinity,true,[],
undefined,false},
"http://127.0.0.1:5984",[],none,[]},
{tcp_session,
{{"127.0.0.1",5984},<0.22109.0>},
false,http,#Port<0.21371>,1},
undefined,undefined,undefined,
{httpc_response,parse,[nolimit,false]},
{[],[]},
new,[],nolimit,nolimit,
{options,
{undefined,[]},
0,2,2,disabled,enabled,false},
{timers,[],undefined},
httpc_manager,undefined}
** Reason for termination ==
** session_remotly_closed
[/code] |
|
|
| Back to top |
|
| spgarbet |
Posted: Wed Aug 06, 2008 2:53 pm |
|
|
|
Joined: 06 Aug 2008
Posts: 4
|
This was written by Jonathan Hicks, and myself.
I was calling it as follows against a couchdb installation on the same box:
> dominatrix:torquemada("http://127.0.0.1:5984", 10000, 60).
Code:
-module(dominatrix).
-export([torquemada/3]).
-export([strike/1,whip/2,inquisitor/0]).
% Copyright 2008, Gnu Public License Version 3.0. Jonathan Hicks and Shawn Garbett
for(N, N, F) -> [F()];
for(I, N, F) -> [F()|for(I+1, N, F)].
strike(Site) ->
case http:request(Site) of
{ok, {{_, 200, _}, _, _}} -> success;
_ -> failure
end.
whip(Site, Inquisitor) ->
receive
stop -> true
after 0 ->
Inquisitor ! strike(Site),
whip(Site, Inquisitor)
end.
inquisitor() -> inquisitor({0,0}).
inquisitor({Successes, Failures}) ->
receive
success -> inquisitor({Successes+1,Failures});
failure -> inquisitor({Successes,Failures+1});
{results,Pid} -> Pid ! {Successes,Failures},
inquisitor({Successes,Failures});
stop -> true
end.
%% Why does this die above 60 concurrent requests? This shouldn't happen!
torquemada(Site, Time, Concurrent) ->
inets:start(),
Inquisitor = spawn(dominatrix, inquisitor, []),
Torturers = for(1, Concurrent, fun() -> spawn(dominatrix, whip, [Site, Inquisitor]) end),
timer:sleep(Time),
Inquisitor ! {results, self()},
lists:foreach(fun(Pid) -> Pid ! stop end, Torturers),
receive
{Confessions,Heretics} ->
io:format("~w Successes and ~w Failures in ~w seconds with ~w concurrent users => ~w requests/second", [Confessions, Heretics, Time/1000, Concurrent, Confessions/Time*1000])
end,
Inquisitor ! stop.
|
|
|
| Back to top |
|
| spgarbet |
Posted: Wed Aug 06, 2008 3:07 pm |
|
|
|
Joined: 06 Aug 2008
Posts: 4
|
Quote:
mog: does this happen CyberGarp at lower numbers?
[10:00am] HunterXHunterXXX: that happaens to me too
[10:00am] CyberGarp: no
[10:00am] mog: whats your ulimit?
[10:01am] mog: this is a gnu/linux box i assume?
[10:01am] CyberGarp: Mac OSX
[10:01am] mog: ohhhh
[10:02am] CyberGarp: Running otp_src_R12B-3
[10:02am] mog: look up open file descriptors mac os x
[10:02am] CyberGarp: okay, sec
[10:02am] mog: im sure your exhausting that
[10:02am] mog: as this on local host you probably have 20,000 or so open file descriptors
[10:02am] mog: and macosx doesnt like that
[10:02am] mog: i know how to change it on gnu/linux boxes but not macs
[10:03am] CyberGarp: spgarbet$ ulimit
[10:03am] CyberGarp: unlimited
[10:04am] CyberGarp: Found it.
[10:04am] CyberGarp: ulimit -a
[10:04am] CyberGarp: open files (-n) 2560
[10:04am] mog: oh cool, its same thing as gnu/linux
[10:04am] mog: ya you cant open more than 2560 sockets right now
[10:05am] CyberGarp: It's basically BSD running a Mach kernal.
[10:05am] mog: you can up it to infiinity
[10:05am] mog: ya i know
[10:05am] CyberGarp: http://archives.postgresql.org/pgsql-admin/2007-11/msg00133.php
[10:05am] mog: but didnt know bsd had same tool for that
[10:05am] MisterN: lol i have 1024
[10:05am] mog: most gnu/linux default to that
[10:05am] CyberGarp: So I need to trap the error and print something intelligent?
[10:05am] CyberGarp: Then up the limit to actually do it.
[10:05am] mog: or up the ulimit before start
[10:05am] mog: ya
[10:06am] CyberGarp: cool
[10:06am] mog: well try uping it
[10:06am] CyberGarp: thx
[10:06am] mog: thats just my gues
[10:06am] mog: it could be something else as well
|
|
|
| Back to top |
|
| spgarbet |
Posted: Wed Aug 06, 2008 3:57 pm |
|
|
|
Joined: 06 Aug 2008
Posts: 4
|
While fiddling with the ulimit parameters I can get the amount to go higher. But why is it failing in the first place. 20 processes continually requesting through http:request shouldn't be hitting 10,000 open files?
Is there some option about how it uses the tcp/ip stack that will make it 20 open files? |
|
|
| Back to top |
|
| danmil |
Posted: Wed Oct 29, 2008 2:23 pm |
|
|
|
Joined: 29 Oct 2008
Posts: 1
|
I ran into a very similar problem (lots of session_remotly_closed at high concurrency for inets/http client). Tried out the ibrowse http client, which seems to have solved the problem instantly. A bit hard to find the source -- had to check out all of the jungerl code collection from cvs at:
http://sourceforge.net/cvs/?group_id=73688
And then pull out the ibrowse source and build it separately.
-Dan Milstein |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum
|
|
|