My G+ Profile
« Chad's Under Attack! | Main | Legislative Podcasting »
Distributed Back-up Systems
I’ve been interested in distributed back-up systems for some time. For example, I’d love to see a P2P client given to BYU students that allows them to commit a percentage of their disk to a distributed back-up system in exchange for that much storage on the overall system. Rather than the University having to commit capital to a back-up system for students files, excess direct-attached disk and software would solve the problem.
I’ve also be enamored with erasure codes for reliability. Using erasure codes would allow the distributed back-up network to provide reliable storage in the face of a certain percentage of nodes going down, leaving the network for some reason, and so forth.
A couple of students in my Middleware class this semester picked this theme up and did some further exploration. There were a couple of items that caught my eye.
- PStore is a secure P2P storage solution from some researchers at MIT. Overall, the feature set seems quite nice, but the code is not available and it doesn’t incorporate erasure codes as fas as I know.
- DIBS is a similar idea written in python that does use erasure codes. The UI is something only a geek could love.
Apart from being genuinely useful in a campus environment where its difficult to provide effective back-up solutions for even critical files, this is an excellent example of a P2P network beyond mere “file sharing” which has grown to have negative connotations. I’d love to see the headline “BYU Embraces P2P Technology.”
Posted by windley on April 11, 2005 10:58 AM
Comment from Jack Krupansky at April 11, 2005 11:29 AM
I've considered this "backup" problem from a somewhat different angle, as a concept which I call Distributed Virtual Personal Computer (DVPC). A common feature between our approaches is that a backup is not at "a" place, but distributed across numerous, diverse servers (including geographically).
I've also integrated the concept of "smart versioning" since overwriting of files and undoing changes is one of the primary headaches many computing users face.
It's only a concept right now, but unless we can get the concept right, we're merely going to end up with yet more "band aid" solutions that cause more headaches than they cure.
See: http://BaseTechnology.com/dvpc.htm
-- Jack Krupansky
Comment from Vanessa Williams at April 11, 2005 3:37 PM
I haven't tried it myself, but along the same lines there is a JXTA-based product from Three Twelve Inc. that allows secure backup to a private network of peers. A couple of sysadmins I pointed it out to seemed completely uninterested in the concept. Maybe support for something like that really needs to come from the grass-roots. Cheers.
http://www.312inc.com/index.html
Comment from Eliot Jacobsen at April 11, 2005 6:37 PM
Ian Clark of FreeNet fame has some thoughts about "placeless data" you might find interesting:
http://www.openp2p.com/pub/a/p2p/2000/11/14/ian.html