From Erlang Community

Revision as of 08:07, 27 October 2007 (edit) Eranga (Talk | contribs) ← Previous diff		Current revision (03:04, 13 August 2010) (edit) (undo) Jj1bdx (Talk | contribs)
(4 intermediate revisions not shown.)
Line 4:		Line 4:
	==Requirements==		==Requirements==
-	Let's say I have a make book library index application. There's a table I use to record all the available library book. The record structure is as below.	+	Let's say I have to make a book library index application. There's a table I use to record all the available library books. The record structure is as below.
	Due to the high volume of data, I want this table to be fragmented in a single Erlang node. If you want to make this fragmented table distributed, you may refer to http://www.trapexit.org/Distributing_a_Mnesia_schema tutorial on making distributed table. All the rest of the work related to table framentation remains same.		Due to the high volume of data, I want this table to be fragmented in a single Erlang node. If you want to make this fragmented table distributed, you may refer to http://www.trapexit.org/Distributing_a_Mnesia_schema tutorial on making distributed table. All the rest of the work related to table framentation remains same.
Line 13:		Line 13:
	==Start an Erlang node==		==Start an Erlang node==
-	Our example node foo@example.com have a default disc storage paths set to the directory Mnesia.foo@example.com in the current directory.	+	Our example node foo@example has the default disc storage path set to the directory Mnesia.foo@example in the current directory.
		+	erl -sname foo
-	The directory can be overriden by using -mnesia dir '"/path/of/your/preference"' ' when starting the node.	+	The directory can be overridden by using -mnesia dir '"/path/of/your/preference"' ' when starting the node.
	Let create a disk based schema by running,		Let create a disk based schema by running,
Line 21:		Line 22:
	==Create the Fragmented table with 20 table fragments==		==Create the Fragmented table with 20 table fragments==
-	In this example all the 20 fragments are in the same Erlang/Mnesia node. Also the fragments are disc_copies.	+	In this example all 20 fragments are in the same Erlang/Mnesia node. Also the fragments are disc_copies.
	mnesia:create_table(book_info,		mnesia:create_table(book_info,
	[{frag_properties, [{node_pool, [node()]}, {n_fragments, 20}, {n_disc_copies, 1}]},		[{frag_properties, [{node_pool, [node()]}, {n_fragments, 20}, {n_disc_copies, 1}]},
Line 31:		Line 32:
	==Add records==		==Add records==
-	Create a function with mnesia:write/3 function.	+	Create a function which calls mnesia:write/3.
	AddFun = fun() ->		AddFun = fun() ->
	mnesia:write(book_info, Record, write)		mnesia:write(book_info, Record, write)
	end		end
-	Now call that function in inside mnesia:activity/4 as below.	+	Now call that function inside mnesia:activity/4 as below.
	ok = mnesia:activity(transaction, AddFun, [], mnesia_frag)		ok = mnesia:activity(transaction, AddFun, [], mnesia_frag)
-	Notice that I have used the activity access context as transaction. Transaction makes sure that the operation is all successfull or all fail (atomic). AccessContext I can use are,	+	Notice that I have used the activity access context as "transaction". Transaction makes sure that the operation is all successfull or all fail (atomic). AccessContext I can use are,
	{transaction, Retries}		{transaction, Retries}
	sync_transaction		sync_transaction
Line 53:		Line 54:
	==Select records with limit==		==Select records with limit==
-	Select books by Author "steve" with 10 books limit. Remember 10 is not a hard limit. Create a function with mnesia:select/4 function.	+	As an example let's select books by Author "steve" with 10 books limit. Remember 10 is not a hard limit. Create a function with mnesia:select/4 function.
	MatchHead = #book_info{author = "steve", _ = '_'},		MatchHead = #book_info{author = "steve", _ = '_'},
	Guard = [],		Guard = [],
Line 62:		Line 63:
	end,		end,
-	Now call that function in inside mnesia:activity/4 as below.	+	Now call that function inside mnesia:activity/4 as below.
	Result = mnesia:activity(transaction, AddFun, [], mnesia_frag)		Result = mnesia:activity(transaction, AddFun, [], mnesia_frag)
	Result -> '$end_of_table' | {[Objects], Cont} | transaction abort		Result -> '$end_of_table' | {[Objects], Cont} | transaction abort
-	In fragmented table, if it returns {[Objects], Cont} and the number of objects return is less than the number of expected objects (10), you need to run recursivly mnesia:select/1 with the return Cont (continuation) until you get the expected number of results or '$end_of_table'.	+	In a fragmented table, if it returns {[Objects], Cont} and the number of objects returned is less than the number of objects expected (10), you need to recursively run mnesia:select/1 with the return Cont (continuation) until you get the expected number of results or '$end_of_table'.
	SelFun2 = fun() ->		SelFun2 = fun() ->
	mnesia:select(Cont)		mnesia:select(Cont)
	end,		end,
-	Result2 = mnesia:activity(transaction, AddFun, [], mnesia_frag)	+	Result2 = mnesia:activity(transaction, SelFun2, [], mnesia_frag)
	Result2 -> '$end_of_table' | {[Objects], Cont} | transaction abort		Result2 -> '$end_of_table' | {[Objects], Cont} | transaction abort
		+	== List of fragmented tables ==
-	That's it. Now you know how to write your basic Mnesia fragmented tables program.	+	To obtain the list of tables, use mnesia:table_info/2 with the option frag_names:
		+	mnesia:activity(async_dirty, mnesia:table_info/2, [store, frag_names], mnesia_frag).
		+
		+
		+	That's it! Now you know how to write a basic Mnesia fragmented tables program.
	[[Category:HowTo]]		[[Category:HowTo]]

---

Current revision

Contents 1 Overview 2 Requirements 3 Sample Fragmented Table 4 Start an Erlang node 5 Create the Fragmented table with 20 table fragments 6 Data operations 7 Add records 8 Select records with limit 9 List of fragmented tables

[edit] Overview

This HOWTO describes how you make Fragmented Mnesia tables and how to use them.

[edit] Requirements

Let's say I have to make a book library index application. There's a table I use to record all the available library books. The record structure is as below. Due to the high volume of data, I want this table to be fragmented in a single Erlang node. If you want to make this fragmented table distributed, you may refer to http://www.trapexit.org/Distributing_a_Mnesia_schema tutorial on making distributed table. All the rest of the work related to table framentation remains same.

[edit] Sample Fragmented Table

I need this table to be disk_copies. Other modes also operate the same way.

-record(book_info, {isbn, name, author, keywords, category, description}).

[edit] Start an Erlang node

Our example node foo@example has the default disc storage path set to the directory Mnesia.foo@example in the current directory.

 erl -sname foo

The directory can be overridden by using -mnesia dir '"/path/of/your/preference"' ' when starting the node.

Let create a disk based schema by running,

 mnesia:create_schema([node()]). 
 

[edit] Create the Fragmented table with 20 table fragments

In this example all 20 fragments are in the same Erlang/Mnesia node. Also the fragments are disc_copies.

 mnesia:create_table(book_info,
   [{frag_properties, [{node_pool, [node()]}, {n_fragments, 20}, {n_disc_copies, 1}]},
   {index, [name, keywords, category]},
   {attributes, record_info(fields, book_info)}]),
 

[edit] Data operations

In order to be able to access a record in a fragmented table, Mnesia must determine to which fragment the actual record belongs. This is done by the mnesia_frag module, which implements the mnesia_access callback behaviour. Wrap standard Mnesia operation functions inside the function and pass to mnesia:activity/4 with callback module mnesia_frag.

[edit] Add records

Create a function which calls mnesia:write/3.

 AddFun = fun() ->
           mnesia:write(book_info, Record, write)
          end

Now call that function inside mnesia:activity/4 as below.

 ok = mnesia:activity(transaction, AddFun, [], mnesia_frag)
 

Notice that I have used the activity access context as "transaction". Transaction makes sure that the operation is all successfull or all fail (atomic). AccessContext I can use are,

 {transaction, Retries} 
 sync_transaction 
 {sync_transaction, Retries} 
 async_dirty 
 sync_dirty 
 ets 

For example if you want to do above activity in dirty mode, you can write,

 ok = mnesia:activity(async_dirty, AddFun, [], mnesia_frag)
 
 Refer to mnesia:activity/4 documentation for more info.
 

[edit] Select records with limit

As an example let's select books by Author "steve" with 10 books limit. Remember 10 is not a hard limit. Create a function with mnesia:select/4 function.

 MatchHead = #book_info{author = "steve", _ = '_'},
 Guard = [],
 Result = ['$_'],
 MatchSpec = [{MatchHead, Guard, Result}],
 SelFun = fun() ->
           mnesia:select(book_info, MatchSpec, 10, read)
          end,

Now call that function inside mnesia:activity/4 as below.

 Result = mnesia:activity(transaction, AddFun, [], mnesia_frag)

 Result -> '$end_of_table' | {[Objects], Cont} | transaction abort
 

In a fragmented table, if it returns {[Objects], Cont} and the number of objects returned is less than the number of objects expected (10), you need to recursively run mnesia:select/1 with the return Cont (continuation) until you get the expected number of results or '$end_of_table'.

 SelFun2 = fun() ->
             mnesia:select(Cont)
           end,
 Result2 = mnesia:activity(transaction, SelFun2, [], mnesia_frag)

 Result2 -> '$end_of_table' | {[Objects], Cont} | transaction abort

[edit] List of fragmented tables

To obtain the list of tables, use mnesia:table_info/2 with the option frag_names:

 mnesia:activity(async_dirty, mnesia:table_info/2, [store, frag_names], mnesia_frag).

That's it! Now you know how to write a basic Mnesia fragmented tables program.