#692 wisp occasionally slow

liamstask Mon 27 Jul 2009

I was doing a bit of playing around with the 1.0.45 build, and found that as I was running some more of those pesky apachebench tests on what was not much more than the hello.fan demo in the web examples dir, that sometimes wisp seems to hang a bit.

I thought it might have been my mucking about, but running the pristine demo will sometimes see response times of up to 20 seconds, while they're usually just a couple milliseconds. Might take a couple times to see but I usually observe this happening when running

ab -n 10000 http://127.0.0.1:8080/

3 or 4 times in a row. I don't see errors as was the case in my previous wisp report, just slow responses that sometimes timeout.

Any changes that might have affected this behavior?

tcolar Mon 27 Jul 2009

Maybe it's running a garbage collection ?

andy Mon 27 Jul 2009

I've been seeing issues this week as well with FF 3.5 - we'll look into.

brian Mon 27 Jul 2009

If the pause is 20sec, I doubt that is garbage collection - since I have socket timeout set to 10sec sounds suspiciously like a HTTP pipelining issue.

tcolar Tue 28 Jul 2009

I tried running visalvm (profiler) on this to see what's going on. The good news is that the profiler works with fan right out of the box.

The weird part: i have 0.44, even with profiler off, any request from "ab" takes at least 10 seconds (ie: 10000 + ~20ms), now when i run it through thee browser it's fast (~20ms) ... so guessing fan doesn't like the kind of request makes ?

For info profiler says all time(10+sec) spent in: fan.sys.SysInStream.r

Makes me think maybe ab doesn't close properly the request, or does a keepalive or something like that ?

I'll try with 0.45 to see if any different.

tcolar Tue 28 Jul 2009

Same with 0.45. ab takes always 10sec + ~20 ms per request.

tcolar Tue 28 Jul 2009

Actually running with ab in verbose mode, i see response code coming instantly but then it hangs for 10 sec. Could it be fan hello.fan is not ending properly the request (at least the way ab wants it)

tcolar Tue 28 Jul 2009

ab must be weird. I'm running the test in jmeter now and Fan is performing great so far. Will do a long run test and see if i get any slow response.

tcolar Tue 28 Jul 2009

Actually with jmeter the wisp/hello.fan test was pretty amazing.

Did 10,000 request in 10mn on my little laptop. all where <100ms, except 3

The first one was ~200ms (init) and two others where ~2 seconds (seem to have been during a GC - the profiler shows a growing Char[] that needs GC'ing after a while (up to 80MB)).

At a slower pace 100 requests over 2mn, all results where 2-5ms.

Either way those are some damn good results, faster than java helloworld in tomcat for sure.

andy Tue 28 Jul 2009

Good detective work tcolar ;) There is definitely an issue though, since I'm seeing invalid responses in FF 3.5 (the same issues I saw before with 3.0). Just haven't had a chance to dig into the code yet.

tcolar Tue 28 Jul 2009

When using ab the issue is:

In WispActor:126 it says:

// skip leading CRLF (4.1)
  in := req.socket.in

This is blocking io, and since line 52 says:

socket.options.receiveTimeout = 10sec

It waits 10 seconds before proceeding(socket timeout).

brian Wed 12 Aug 2009

I messed around with this, and I am not seeing any apache bench problems.

I'd like to understand what might be happening, but not sure I can really proceed further.

What OS are you guys running your apache bench tests on?

tcolar Wed 12 Aug 2009

On my laptop, it happens every time when using apachebench, even for just 1 request (but not with other tools like jmeter). This is under Linux Ubuntu jaunty.

I gave you most of the infos I could get, and the profiler / tracing told me that it was obvious the issue was that it would always "block" until the socket timed out (10s for this example), of course if you have multiple blocking such request it degraded even more.

I believe from looking at the code/testing that it's sprobably because ab does not send the first packet(CR/LF) in the request, which wisp is waiting for before proceeding(WispActor:126). or some other problem with the stream ?

Here is ab -V:

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
</pre

Login or Signup to reply.