Cloud storage possibilities in tc

[Ulysses_]

Noticed dropbox has been discussed here a bit.

In NFS the data is only stored in the server's hard drive. Not so with dropbox, where a duplicate of the data is stored in your local drive. If there is no local drive, dropbox is not an option, right? At least not with files larger than available ram.

Is there any other cloud storage solution that does not keep a duplicate of the data locally?

Any workarounds to make dropbox work this way? Any way to make it work with files larger than available ram when there is no hard drive?

[curaga]

I haven't heard of any "cloud" provider offering a network filesystem, which would enable files larger-than-RAM. NFS is not really secure enough to offer over the 'net.

OTOH, the fuse module that uses your gmail space might work like this. It's not quite in the same class as dropbox though.

[gerald_clark]

Jungledisk uses a fuse filesystem.  I don't know anyone using it with TC though.

[Ulysses_]

It's not quite in the same class as dropbox though.

Can you explain a little what is going on with fuse using gmail space (or jungledisk's fuse?) that makes it substantially different from NFS as far as the user is concerned?

[tinypoodle]

Any user account with a shell provider could be locally mounted with sshfs-fuse (as long as there is a remote sftp server running).

[gerald_clark]

The difference is how Jungledisk stores and archives the files.  The Jungledisk website would be a better place to read about its advantagaes and limitations.

[tinypoodle]

This appears like it might be worth looking into:

http://code.google.com/p/boxfs/

[tinypoodle]

Based on subject of this thread I was led to do some research about available remote storage options.

Now the one thing I did most certainly not expect to happen was this:

[gerald_clark]

I tried boxfs.  It has not been updated in over 2 years, and reports:
Error while logging in to Box.net.

This looks like a dead end.

[tinypoodle]

In NFS the data is only stored in the server's hard drive. Not so with dropbox, where a duplicate of the data is stored in your local drive. If there is no local drive, dropbox is not an option, right? At least not with files larger than available ram.

Is there any other cloud storage solution that does not keep a duplicate of the data locally?

Here is the result of putting sshfs to the test:

Code: [Select]

tc@box:~$ ssh user@shell.org
Last login: Fri Apr  5 10:07:58 2013 from 72.208.234.114
[user@shell ~]$ du -sh
296M    .
[user@shell ~]$ exit
logout
Connection to shell.org closed.
tc@box:~$ mkdir -p mnt/sshfsmount
tc@box:~$ sshfs user@shell.org: mnt/sshfsmount/ -C
tc@box:~$ df mnt/sshfsmount/
Filesystem                Size      Used Available Use% Mounted on
user@shell.org:      7.0T    875.1G      5.6T  13% /home/tc/sshfsmount
tc@box:~$ du -sh mnt/sshfsmount/
298.4M  mnt/sshfsmount/
tc@box:~$ df /
Filesystem                Size      Used Available Use% Mounted on
rootfs                   452.0M     27.0M    425.1M   6% /
tc@box:~$ cd mnt/sshfsmount/
tc@box:~/mnt/sshfsmount$ mkdir dload
tc@box:~/mnt/sshfsmount$ cd dload/
tc@box:~/mnt/sshfsmount/dload$ wget -c http://archive.org/download/utopia/Utopia_512kb.mp4
Connecting to archive.org (207.241.224.2:80)
Connecting to ia600209.us.archive.org (207.241.227.239:80)
Utopia_512kb.mp4     100% |*******************************|   563M 00:00:00 ETA
tc@box:~/mnt/sshfsmount/dload$ cd
tc@box:~$ ssh user@shell.org
Last login: Fri Apr  6 08:12:23 2013 from 72.208.234.114
[user@shell ~]$ du -sh
860M
[user@shell ~]$ exit
logout
Connection to shell.org closed.
tc@box:~$ df -h /
Filesystem                Size      Used Available Use% Mounted on
rootfs                  452.0M     27.2M    424.9M   6% /
tc@box:~$ mplayer sshfsmount/dload/Utopia_512kb.mp4
MPlayer SVN-r33811-snapshot-4.4.3 (C) 2000-2011 MPlayer Team

Playing sshfsmount/dload/Utopia_512kb.mp4.
libavformat file format detected.
[....]

tc@box:~$ df /                   
Filesystem                Size      Used Available Use% Mounted on
rootfs                   452.0M     27.2M    424.9M   6% /
tc@box:~$ mount|grep sshfsmount 
user@shell.org: on /home/tc/mnt/sshfsmount type fuse.sshfs (rw,nosuid,nodev,user_id=1001,group_id=50,max_read=65536)

QED

[Ulysses_]

That's cool. So you have paid for a shell account at shell.org? Or is it a free service, and if yes what is the password?

Also how does shell.org compare with common cloud storage like amazon's, might amazon's be faster?

[tinypoodle]

Err... "shell.org" is a fictional replacement to obfuscate my personal credentials, perhaps I should have made that "shell.example.com" to be clear beyond a doubt 
If you search the web you could find dozens of shell providers at a vast range of various terms and conditions, from in-mommas-basement until enterprise.
No idea about amazon, but sure net speed can differ between shell providers and perhaps even between types of accounts of same provider.

[Ulysses_]

What happens if you put a truecrypt container into the sshfs folder and you then mount the truecrypt container, and you then echo hello >> truecrypt_folder/test.txt? Does the entire container file get transfered?

I know truecrypt only changes a small part of the container file, but if that file is under sshfs, does the entire file get transfered?

[Ulysses_]

Can software raid 1 be done between two sshfs volumes in different servers?

One server would be my pc, the other server would be "in the cloud" - with sshfs set up of course in both servers.

[Ulysses_]

Can software raid 1 be done between two sshfs volumes in different servers?

For redundancy one could go for something like GFS2, probably interfaced to DRDB, and using free VPS's from different providers. Add client-side encryption like truecrypt or its more up-to-date cousins.

That would be a cool live CD. Anyone see problems with this?