| Author |
Message |
|
| yrashk |
 Posted: Tue Dec 22, 2009 6:44 am |
 |
|
|
Joined: 22 Dec 2009
Posts: 7
|
Hi,
I am happy to announce an early version of aqmpfs (I've been cooking it for
a little bit more than a week).
AMQPFS is a Filesystem as API that works through AMQPFS, all filesystem
providers are available through AMQP bus and can be distributed in the
network.
What's the point? Why one may need this? Well, this project started as a way
to integrate some third-party software while maintaining no real files or
directories for it.
For example, it could be used to serve configuration files, receive logs and
such. Itsuse is virtually unlimited.
Plus, it can also utilize multiple instances of providers for the same path
which leads us to interesting fault tolerance and availability algorithms.
Would be glad to know if anybody else will find it useful.
It is available at http://github.com/scallable/amqpfs
Please note that it is very immature and pretty much undocumented. Feel free
to contact
me (yrashk on FreeNode IRC or yrashk at gmail.com by e-mail).
Cheers,
Yurii
--
View this message in context: http://old.nabble.com/amqpfs-announcement-tp26884573p26884573.html
Sent from the RabbitMQ mailing list archive at Nabble.com.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
 
|
| Guest |
| Posted: Tue Dec 22, 2009 8:22 am |
|
|
|
Guest
|
Neat! Does this also work with HDFS (or could it)?
alexis
On Tue, Dec 22, 2009 at 6:43 AM, yrashk <yrashk@gmail.com> wrote:
>
> Hi,
>
> I am happy to announce an early version of aqmpfs (I've been cooking it for
> a little bit more than a week).
>
> AMQPFS is a Filesystem as API that works through AMQPFS, all filesystem
> providers are available through AMQP bus and can be distributed in the
> network.
>
> What's the point? Why one may need this? Well, this project started as a way
> to integrate some third-party software while maintaining no real files or
> directories for it.
>
> For example, it could be used to serve configuration files, receive logs and
> such. Itsuse is virtually unlimited.
>
> Plus, it can also utilize multiple instances of providers for the same path
> which leads us to interesting fault tolerance and availability algorithms.
>
> Would be glad to know if anybody else will find it useful.
>
> It is available at http://github.com/scallable/amqpfs
>
> Please note that it is very immature and pretty much undocumented. Feel free
> to contact
> me (yrashk on FreeNode IRC or yrashk at gmail.com by e-mail).
>
> Cheers,
> Yurii
> --
> View this message in context: http://old.nabble.com/amqpfs-announcement-tp26884573p26884573.html
> Sent from the RabbitMQ mailing list archive at Nabble.com.
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss@lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| yrashk |
| Posted: Tue Dec 22, 2009 8:28 am |
|
|
|
Joined: 22 Dec 2009
Posts: 7
|
Alexis,
Short answer is no.
Long answer is
1) I haven't used HDFS myself so I can't say much about it
2) It does not support HDFS
3) HDFS goal is to provide a distributed and fault-tolerant filesystem, while AMQPFS goal is to expose services available through AMQP bus as filesystems (hence Filesystem as API)
4) I can see how somebody may develop a hdfs_amqpfs_provider that will be available through AMQP and will integrate with HDFS (so, _yes it could_)
5) *However*, I am not sure is there any good reason for doing this
Yurii
On 2009-12-22, at 12:21 AM, Alexis Richardson wrote:
> Neat! Does this also work with HDFS (or could it)?
>
> alexis
>
>
> On Tue, Dec 22, 2009 at 6:43 AM, yrashk <yrashk@gmail.com> wrote:
>>
>> Hi,
>>
>> I am happy to announce an early version of aqmpfs (I've been cooking it for
>> a little bit more than a week).
>>
>> AMQPFS is a Filesystem as API that works through AMQPFS, all filesystem
>> providers are available through AMQP bus and can be distributed in the
>> network.
>>
>> What's the point? Why one may need this? Well, this project started as a way
>> to integrate some third-party software while maintaining no real files or
>> directories for it.
>>
>> For example, it could be used to serve configuration files, receive logs and
>> such. Itsuse is virtually unlimited.
>>
>> Plus, it can also utilize multiple instances of providers for the same path
>> which leads us to interesting fault tolerance and availability algorithms.
>>
>> Would be glad to know if anybody else will find it useful.
>>
>> It is available at http://github.com/scallable/amqpfs
>>
>> Please note that it is very immature and pretty much undocumented. Feel free
>> to contact
>> me (yrashk on FreeNode IRC or yrashk at gmail.com by e-mail).
>>
>> Cheers,
>> Yurii
>> --
>> View this message in context: http://old.nabble.com/amqpfs-announcement-tp26884573p26884573.html
>> Sent from the RabbitMQ mailing list archive at Nabble.com.
>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss@lists.rabbitmq.com
>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Dec 22, 2009 8:38 am |
|
|
|
Guest
|
Yurii
Thank-you. Makes sense.
As to (5), I am of course not sure if using your FS interface would be
the way to go. However, various folks have in the past listed issues
with HDFS that could be solved by putting a proper queue in front of
it.
I will show them this thread in case they wish to comment. Failing
that, I can probably dig around and pull out some examples.
alexis
On Tue, Dec 22, 2009 at 8:28 AM, Yurii Rashkovskii <yrashk@gmail.com> wrote:
> Alexis,
>
> Short answer is no.
>
> Long answer is
>
> 1) I haven't used HDFS myself so I can't say much about it
> 2) It does not support HDFS
> 3) HDFS goal is to provide a distributed and fault-tolerant filesystem, while AMQPFS goal is to expose services available through AMQP bus as filesystems (hence Filesystem as API)
> 4) I can see how somebody may develop a hdfs_amqpfs_provider that will be available through AMQP and will integrate with HDFS (so, _yes it could_)
> 5) *However*, I am not sure is there any good reason for doing this
>
> Yurii
>
> On 2009-12-22, at 12:21 AM, Alexis Richardson wrote:
>
>> Neat! |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
