|
|
| Author |
Message |
|
| 5hundy |
 Posted: Sun Oct 14, 2007 8:15 am |
 |
|
|
User
Joined: 14 Oct 2007
Posts: 17
|
Current System: (Using Ruby/Rails) Many servers handle tens of thousands of small transactions every day. Each transaction is recorded to a sqlite file (write only, one per server). At the end of the day all sqlite files are aggregated, analyzed, and finally stored in a safe location.
The data is used in aggregate but it's important to keep all data for every transaction somewhere where I can access it later if I need to. The key is that I'll have many processes trying to write to the same data store but it's write only.
I'm interested in replacing the whole system with erlang. My question is, what technology is recommended to replace my sqlite storage scheme? I'm new to erlang but here are some thoughts:
1) Use DETS. maybe one table per box. I don't know how to backup or move these data. Would each file have a new name for each date? Is this a good solution if many processes are all trying to write to it?
2) Use mnesia with a new table for each date. I'd need a way to backup the data to a file, move the file, and delete the table every day.
3) Use one central mnesia disk-only database and forget the idea of separate files. I'm worried the amount of data will grow out of control and writes would be unnecessarily slow in this case.
This is a critical design decision and any thoughts would be greatly appreciated!!  |
|
|
| Back to top |
 
|
| rbucker881 |
| Posted: Mon Oct 15, 2007 12:22 pm |
|
|
|
User
Joined: 11 Oct 2007
Posts: 17
|
... every day. I have a client that has an application developed in perl. While it is likely that I'll refactor the application in perl in order to meet the assigned goals... I did consider a rewrite to erlang, however, now that I know they have a SQL db in the mix it's going to be impossible to migrate. It would be nice to have reliable SQL access (I happen to like SQLite but I also use postgresql).
Problem is the unixODBC and the related drivers and prerequisites are near impossible to install. And there are fewer docs for building same on Mac.
I'll be watching this thread closely. |
|
|
| Back to top |
|
| rbucker881 |
| Posted: Wed Oct 17, 2007 12:37 am |
|
|
|
User
Joined: 11 Oct 2007
Posts: 17
|
| then again try configuring on OpenBSD or OSX. Both are tough and ODBC has been a roadblock... no thanks to google. |
|
|
| Back to top |
|
| francesco |
| Posted: Sat Oct 20, 2007 8:27 am |
|
|
|
User
Joined: 07 Jul 2006
Posts: 249
Location: London
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
