|
|
| Author |
Message |
< Advanced Erlang/OTP ~ mnesia batch load in distributed environment |
| -dk- |
 Posted: Mon Jul 21, 2008 5:05 pm |
 |
|
|
Joined: 21 Jul 2008
Posts: 1
|
Hi,
We have a distributed system providing some service based on read-only db querying. There are 2 of master db nodes (disk_copies), and 4 servers which hold 'ram_copies' replicas of mnesia db. There are relatively slow network between some of nodes.
Db is updated hourly.
Update runs in a single transaction by initial design. But since data set grown enough memory consumed by transaction manager reached some limit, that made nearly impossible to make an update.
We decided to move to active/stand-by paradigm which allows us to not use transactions for update, but just dirty operations.
But that resulted in other problems.
1. Using sync_dirty activity context (to make sure all the nodes are updated at the end of activity) results in massive network traffic and update is very slow - more than 1 hour in our case, compare to about 5 minutes for "old-style" transactional update.
2. Using async_dirty results in much more memory consuming - probably for replica queues and no way to guarantee that update is completed on all nodes.
3. Test with async dirty load plus one sync dirty write at the end - to make sure replicas delivered - made funny result - first 5 minutes or so data was written to the 1st master node (where update is running) and part of the data distributed over other nodes, then more than 1 hour 2nd master database slowly updated with new replica, periodically reporting that mnesia is overloaded...
4. Next scenario tested: one sync dirty write per 1000 async dirty writes.... Result is acceptable: low memory consuming, moderate network traffic, almost acceptable time ~10 min...
The purpose of this post? I can't believe mnesia is not able to handle massive dirty load distributively... But the fact is that mnesia is very sensitive to inter-node bandwidth. Is there any way to control replication process to avoid overloading? Might be somebody encountered similar problems?
Thank you.
P.S. Data set size is roughly near 1000000 records, db memory size ~50-100 MB |
|
|
| Back to top |
 
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
