| Author |
Message |
|
| Guest |
 Posted: Fri Oct 29, 2010 7:17 am |
 |
|
|
Guest
|
On Sun, Oct 24, 2010 at 4:59 PM, Joel Reymont <joelr1@gmail.com> wrote:
>
> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote:
>
>> I wonder how responsive the system is to other events when running the
>> benchmark.
>
> The benchmark simulates several hundred clients hitting a (mochiweb) web server to read and write couchdb (json) documents. The system seems to stay -highly- responsive, 10x so compared to the same system not using NIFs at all.
>
> If low write response time is taken as a measure of system responsiveness then the first graph shows that the responsiveness of the system has increased dramatically. The write response here is the take taken to process a web request to write a couch document.
>
> ---
> http://twitter.com/wagerlabs
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
>
>
Joel,
I think the issue here is the way that we sequester file io to a
single Erlang process. In your benchmark you'd basically be
sacrificing an Erlang scheduler thread to speed up the synchronous
writes while the other schedulers are freely available to handle
read/write requests.
I'm not sure if Mikeal has written a test for Relaximation that runs a
similar test that hits multiple databases. I think the way I'd try and
show the issue would be to have a large number of clients attempting
to write to their own database.
Also, a trivial way to prove to your self that the NIF's are indeed
called in the same scheduler threads is to create a NIF that has a
function that just does "while(1){}". If you call that function in
more processes than you have schedulers the VM will halt.
HTH,
Paul Davis
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Oct 29, 2010 7:27 am |
|
|
|
Guest
|
Hi,
Have you tried the very same benchmark with just using the 'raw'
option when opening the file.
You should then get about the same efficiency as you get with NIFs
with the difference that you
can use asynch threads as well (i.e . the +A option will be
effective). Without the +A you will get the same
setup as you get with your NIF example.
Another benefit with that is that you don't need to write any C-code.
If the NIF example still is much faster (which I doubt) please let me know.
Without use of the raw option all file operations is done via an extra
Erlang process which explains the difference in speed.
/Kenneth Erlang/OTP, Ericsson
On Sun, Oct 24, 2010 at 3:56 PM, Joel Reymont <joelr1@gmail.com> wrote:
> Simply switching to NIFs for file IO seems to have improved CouchDB write performance more than ten-fold.
>
> Compare the old graph
>
> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4
>
> to the new graph
>
> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7
>
> I was under the impression that the Erlang IO subsystem was highly optimized but there seems to be no limit to perfection.
>
> NIFs are a giant black hole that will subsume Erlang code as performance has to be improved. Start at the lowest level and keep moving up. All that will be left of Erlang in the end is 99.99999% uptime, fault tolerance and supervision... of optimized C code. It's swell and I'm all for it!
>
> Patch is here:
>
> http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83
>
> ---
> http://twitter.com/wagerlabs
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
>
>
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Oct 29, 2010 7:27 am |
|
|
|
Guest
|
Dave,
On Oct 24, 2010, at 6:12 PM, Dave Smith wrote:
> +A anything won't help you anymore. You are now doing the I/O on the
> VM scheduler thread and blocking ANY erlang code (on that scheduler)
> while the file op runs.
Are you sure of this? Can you refer me to the spot in the VM code where I can learn more?
The reason I'm surprised is that increasing the number of threads in the async thread pool had a clear effect on my benchmark. Spikes were, basically, eliminated.
Thanks in advance, Joel
---
http://twitter.com/wagerlabs
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| dizzyd |
| Posted: Fri Oct 29, 2010 7:27 am |
|
|
|
User
Joined: 09 Dec 2005
Posts: 47
|
On Wed, Oct 27, 2010 at 1:28 PM, Joel Reymont <joelr1@gmail.com> wrote:
> Kenneth,
>
> The difference on the Mac is 10x for writes. It's at least 2x on Linux.
Just to be pedantic, that 10x # is for a SSD, no? Otherwise, I've
never seen OS X/Darwin win any awards for I/O speed... 
D.
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
 
|
| Guest |
| Posted: Fri Oct 29, 2010 7:32 am |
|
|
|
Guest
|
On Wed, Oct 27, 2010 at 10:19 AM, Paul Davis
<paul.joseph.davis@gmail.com> wrote:
> Out of curiosity, do you have another Erlang app that would be a good
> candidate for using to test that other parts of the VM remain
> responsive. I was going to suggest various parts of CouchDB that don't
> touch IO as a smoke screen, but an app doing something real that we
> can measure in and out of couch and with and without the new file io
> would be a good help.
I don't know how realistic it is to run both on the same server, but
RabbitMQ is another Erlang app that I deploy in the same sites as
CouchDB.
-elliot
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Oct 29, 2010 7:32 am |
|
|
|
Guest
|
My and the OTP teams view in this matter is:
- Reimplementing standard functionality already written in C (but as a
driver and with asynch thread support) as NIFs is generally a bad
idea)
- Implementing potentially blocking function calls with NIFs is a bad idea.
- You should have VERY strong reasons for writing NIFs at all. It is a
big point in not writing anything in C if it can be avoided.
- The implementation of NIFs is more modern than the driver concept
and among other things the passing of data between Erlang and C-code
is more efficient for NIFs than for drivers. The driver concept does
still have its place and advantages especially for
handling external asynchronous input to Erlang processes. We plan to
improve the driver mechanisms and make solutions
from the NIFs to be available when writing drivers as well.
If it is correct that NIFs for file operations is 2 times faster than
the current file operations in raw mode we will do something about it
because that difference is not justified. But first we must
investigate if that really is the case and where the performance is
lost.
/Kenneth Erlang/OTP Ericsson
On Wed, Oct 27, 2010 at 9:46 AM, Edmond Begumisa
<ebegumisa@hysteria-tech.com> wrote:
> Hi,
>
> I hope the Couch team isn't planning on doing this by default -- something
> about it makes me nervous...
>
> When CouchDB is on it's own, it might not be alarming/noticeable, but I'm
> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. where
> Couch is just one of many OTP apps running in the *SAME* VM -- I have a few
> hacks for avoiding socket communication.) I too worry about the potential
> for NIF-endowed couch io disturbing the balance of Erlang's scheduling.
>
> It would be good to see similar benchmarking with the VM concurrently doing
> things other than handling couch-related requests (which are implicitly
> synchronised in your case.)
>
> - Edmond -
>
> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont <joelr1@gmail.com> wrote:
>
>>
>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote:
>>
>>> I wonder how responsive the system is to other events when running the
>>> benchmark.
>>
>> The benchmark simulates several hundred clients hitting a (mochiweb) web
>> server to read and write couchdb (json) documents. The system seems to stay
>> -highly- responsive, 10x so compared to the same system not using NIFs at
>> all.
>>
>> If low write response time is taken as a measure of system responsiveness
>> then the first graph shows that the responsiveness of the system has
>> increased dramatically. The write response here is the take taken to process
>> a web request to write a couch document.
>>
>> ---
>> http://twitter.com/wagerlabs
>>
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
>>
>
>
> --
> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Oct 29, 2010 7:37 am |
|
|
|
Guest
|
As a devout Couch user, what I'd like to see is a pragmatic approach as
the couch team investigates this -- possibly targeting an optional
configuration with a clear description of the trade-offs to Couch users.
For example, the Couch team could advise using this new feature strictly
in a private emulator devoted to couch since the benchmarks show that NIF
io doesn't seem to have a detrimental effect on the rest of couch.
Personally, for my current uses, Couch's existing io performance is
sufficient. The use of hovercraft, the Erlang view server and some
workarounds to avoid mochiweb have been effective in giving me performance
boosts where I need them.
That said, it cannot be denied that for a database server, a 2-10 fold
increment in io performance is a good enough reason to do things that you
should not really be doing or that are potentially disruptive. For some
couch users, the pros might outweigh the cons. What I'd like to see is a
clear description of what those cons are so I can decide for myself.
- Edmond -
On Thu, 28 Oct 2010 06:28:03 +1100, Joel Reymont <joelr1@gmail.com> wrote:
> Kenneth,
>
> The difference on the Mac is 10x for writes. It's at least 2x on Linux.
>
> How do you propose to investigate?
>
> Sent from my iPhone
>
> On 27/10/2010, at 20:13, Kenneth Lundin <kenneth.lundin@gmail.com> wrote:
>
>> My and the OTP teams view in this matter is:
>>
>> - Reimplementing standard functionality already written in C (but as a
>> driver and with asynch thread support) as NIFs is generally a bad
>> idea)
>> - Implementing potentially blocking function calls with NIFs is a bad
>> idea.
>> - You should have VERY strong reasons for writing NIFs at all. It is a
>> big point in not writing anything in C if it can be avoided.
>> - The implementation of NIFs is more modern than the driver concept
>> and among other things the passing of data between Erlang and C-code
>> is more efficient for NIFs than for drivers. The driver concept does
>> still have its place and advantages especially for
>> handling external asynchronous input to Erlang processes. We plan to
>> improve the driver mechanisms and make solutions
>> from the NIFs to be available when writing drivers as well.
>>
>> If it is correct that NIFs for file operations is 2 times faster than
>> the current file operations in raw mode we will do something about it
>> because that difference is not justified. But first we must
>> investigate if that really is the case and where the performance is
>> lost.
>>
>> /Kenneth Erlang/OTP Ericsson
>>
>> On Wed, Oct 27, 2010 at 9:46 AM, Edmond Begumisa
>> <ebegumisa@hysteria-tech.com> wrote:
>>> Hi,
>>>
>>> I hope the Couch team isn't planning on doing this by default --
>>> something
>>> about it makes me nervous...
>>>
>>> When CouchDB is on it's own, it might not be alarming/noticeable, but
>>> I'm
>>> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e.
>>> where
>>> Couch is just one of many OTP apps running in the *SAME* VM -- I have
>>> a few
>>> hacks for avoiding socket communication.) I too worry about the
>>> potential
>>> for NIF-endowed couch io disturbing the balance of Erlang's scheduling.
>>>
>>> It would be good to see similar benchmarking with the VM concurrently
>>> doing
>>> things other than handling couch-related requests (which are implicitly
>>> synchronised in your case.)
>>>
>>> - Edmond -
>>>
>>> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont <joelr1@gmail.com>
>>> wrote:
>>>
>>>>
>>>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote:
>>>>
>>>>> I wonder how responsive the system is to other events when running
>>>>> the
>>>>> benchmark.
>>>>
>>>> The benchmark simulates several hundred clients hitting a (mochiweb)
>>>> web
>>>> server to read and write couchdb (json) documents. The system seems
>>>> to stay
>>>> -highly- responsive, 10x so compared to the same system not using
>>>> NIFs at
>>>> all.
>>>>
>>>> If low write response time is taken as a measure of system
>>>> responsiveness
>>>> then the first graph shows that the responsiveness of the system has
>>>> increased dramatically. The write response here is the take taken to
>>>> process
>>>> a web request to write a couch document.
>>>>
>>>> ---
>>>> http://twitter.com/wagerlabs
>>>>
>>>>
>>>> ________________________________________________________________
>>>> erlang-questions (at) erlang.org mailing list.
>>>> See http://www.erlang.org/faq.html
>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
>>>>
>>>
>>>
>>> --
>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Oct 29, 2010 7:37 am |
|
|
|
Guest
|
One intersting thing in the graphs is the fluctuations occurring in the NIF'ed version. The old version is much more consistent, but slower. I wonder why it is so.
One problem with writing too much in C that there is a big risk that the code will contain more bugs. So while a system technically is "up" it will be less accessible if it has to restart sub-sections more often.
Robert
----- "Joel Reymont" <joelr1@gmail.com> wrote:
> Simply switching to NIFs for file IO seems to have improved CouchDB
> write performance more than ten-fold.
>
> Compare the old graph
>
> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4
>
> to the new graph
>
> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7
>
> I was under the impression that the Erlang IO subsystem was highly
> optimized but there seems to be no limit to perfection.
>
> NIFs are a giant black hole that will subsume Erlang code as
> performance has to be improved. Start at the lowest level and keep
> moving up. All that will be left of Erlang in the end is 99.99999%
> uptime, fault tolerance and supervision... of optimized C code. It's
> swell and I'm all for it!
>
> Patch is here:
>
> http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83
>
> ---
> http://twitter.com/wagerlabs
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Oct 29, 2010 7:37 am |
|
|
|
Guest
|
Writing to SSD, yes.
There's no award for speed, it's a relative 10x improvement, Darwin vs Darwin.
Sent from my iPhone
On 27/10/2010, at 21:18, Dave Smith <dizzyd@gmail.com> wrote:
> On Wed, Oct 27, 2010 at 1:28 PM, Joel Reymont <joelr1@gmail.com> wrote:
>> Kenneth,
>>
>> The difference on the Mac is 10x for writes. It's at least 2x on Linux.
>
> Just to be pedantic, that 10x # is for a SSD, no? Otherwise, I've
> never seen OS X/Darwin win any awards for I/O speed...
>
> D.
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
