BitTorrent Sync – Making Your Own Cloud

I’ve been using DropBox for probably about 3 years now, and it’s rather good.  Overall, I’ve been quite impressed.  It’s reliable, fairly quick (bandwidth permitting), and fantastic for delivering work to clients and models.

Delivery of final work to clients and models I’ve shot with has been pretty much my sole reason for using DropBox, and it will probably remain that way, at least for the foreseeable future.

I don’t need masses and masses of space for that, but I’ve still never really been all that keen on the idea of keeping things online in “the cloud” (which is basically just a fancy new term for the same old servers we’ve always had) as a form of file backup.

After recently having had a pretty major crash on one of my machines (motherboard died, taking the processor, RAM and a 2TB hard drive along with it), I’ve rebuilt and started looking into other potential backup solutions.

While the internal drives (including the one that died) were backed up to external USB drives, I’d still like to find better alternatives to my current way of doing things (which is using Microsoft SyncToy).

DropBox was never going to be a solution, nor was any other online backup solution.  I have far too many files here to be practical for online storage.

Even if I could get everything uploaded as I create it over the course of however months or years there are between now and my next crash (it will eventually happen, don’t kid yourselves!), it will take days, if not weeks, to download it all again on a fresh system – and my ISP would probably be screaming at me for it.

I needed something a bit more “local”, but just as reliable.  My search eventually led me over to the BitTorrent Sync website, and what appeared to be a too-good-to-be-true piece of software to create your own “cloud” over either a local network or the internet.

See, that’s one of the problems with DropBox.  EVERYTHING has to go up to their servers.  You can’t just have 2 or 3 machines on your LAN syncing up between them, and bypassing the DropBox servers.

This is where BitTorrent Sync steps in.  Your own computers all basically become peer-to-peer “servers” (even if that is an oxymoron, you get the point).

Basically, here’s how it works.  You make a folder on Computer A, and set it up as a BTSync shared folder.  You make a folder on Computer B, enter the code you were given on Computer A, and now you can see the contents of that folder locally (in either a read-only or read/write capacity), as it gets copied across.

btsync_qrcodeWant to share it on a mobile or tablet?  No problem, just enter the code as you did before, or scan the very handy QR Code presented to you by BitTorrent Sync’s web interface running on one of your computers (or through the web interface).

BitTorrent Sync actually has an advantage over DropBox when it comes to mobile devices (at least on iOS, I haven’t tried it with Android yet).

With DropBox, there’s no way to automatically sync an entire folder between your computer and your mobile device.  All you can do is “Favourite” individual files or images, and they will automatically sync if they are updated, but you still have to go into each one individually, and if they’re images that never change, you might as well just download them and not bother “favouriting” them.

BitTorrent Sync, on the other hand, allows you to select a folder and choose to sync it locally on your mobile device, automatically.  Now, I can make a “Portfolio” folder on my PC, create subfolders within it, add & remove images, and have it back up to both of my iPhones and my iPad automatically, and be accessible even when they don’t have internet access.  Bonus!

It also works across the Internet, too.  You’re not just restricted to using it on your LAN.  If I’m out and about with just my iPhone and I need to snag a file from a PC at home, I can just fire up BTSync, browse to where I need (assuming it’s in a shared folder, of course), and grab that one file over 3G, or whatever free WiFi may happen to be available at the location.

You also don’t have to share everything with each system.  If I install the DropBox client on 3 Windows PCs, all three machines will have an identical folder (a mirror of whatever happens to be on the DropBox server).

With BTSync, I can pick and choose exactly which folders I share with which other machines.

With clients available for Windows, Mac, iOS, Android, FreeBSD (i386 or x64) and Linux (i386, x64, PPC or ARM), it’s a pretty versatile and robust system.

Right now I have it installed on 2 Windows machines, 2 iPhones, an iPad, a Raspberry Pi running Raspbmc and a CubieBoard running Cubian.

btsync_desktop_interface

Raspbmc & Cubian are sharing a folder containing Linux downloads on my Windows machine.  It’s a lot easier to browse the web, FTP sites, and wherever else on my PC than it is through SSH.

Downloading a .tar.gz file on Windows and putting it in that folder copies it across my LAN to both the Raspberry Pi and CubieBoard in a predesignated folder, where I can SSH in, extract the files, configure, compile and install.

My iPhones and iPad are sharing the “Portfolio” folder I mentioned above, as well as another couple of folders containing “moodboard” type stuff, for when I’m out shooting with models and clients to give them an idea of the kind of images I’m going for.  These folders are automatically synced up so they’re always with me.

My Windows machines, are sharing a total of about 4TB of files with each other (essentially all my photos, PSD files and final TIF/JPGs from the last year or so).  Having it on two separate machines means that if one dies completely, the other will still be up and running with all files intact until I can get the dead one back up and running.

I still have the option of remote backup, à la Dropbox, just setup a machine at another location, with BTSync installed, and sharing the folder the same way you do locally.

This way, if your house burns down, destroying all local backup copies, you still have everything.  Of course, if something like this ever happened, you’d likely have bigger things to worry about, but at least you can ensure your work is not lost.  Remember, in a fire, the photo albums are often the first things people grab to try and save!

At the moment, my off-site backup is done weekly, manually copying the entire contents of the work drives to USB drives that I can then store in another location.

Sure, it’s far from ideal, but it works, for now.  Eventually I will setup a machine at another location that will do all this automatically across the Internet using BTSync.

With as little power as something like the Raspberry Pi draws, setting up a dedicated machine off-site with a couple of USB drives isn’t as difficult or as expensive (or as loud) as setting up a full blown PC running 24/7.

The other final great thing about BTSync, as I mentioned speed up at the top, is that BTSync is generally faster than DropBox.

When you sync up a folder with DropBox, you’re downloading from one of DropBox’s servers.  The files come down one at a time, and you’re done whenever you’re done.

With BitTorrent Sync, when you add a new client, it will attempt to download from all BTSync systems sharing that folder.  So, if you have 3 BTSync machines already up and running sharing a folder, and wish to add a 4th, it will pull from all of the other 3 machines simultaneously to populate the folder.

How it does this exactly, I’m not sure.  The way I can see it, it works one of two ways.  Either…

  1. It downloads complete files sequentially.  File 1 is pulled from PC1, File 2 is pulled from PC2, File 3 is pulled from PC3, and whichever machine frees up first serves File 4, and so on.
  2. It splits each file up into chunks (the way the regular BitTorrent client works), and downloads different chunks of the same file from each peer all at once.  So, all three of your existing systems could be sending different parts of the same file until your new machine has received it all.

I’m hoping once I post this article, I can have one of the lovely folks at BTSync confirm this for me.

Either way, it’s potentially a much faster download than DropBox,


What would I like to see added to BTSync to make me move away from DropBox completely?

Primarily, a beefier web interface that allows browsing and downloading of files stored on that machine (in the directories that are shared, anyway), offering both individual file downloads, or the entire contents of a given folder in a ZIP file.

btsync_browser_interface

If the folder contains images, a new gallery type interface to flick through all the images would be lovely.  A slideshow option would be amazing, and something DropBox doesn’t currently offer, either.

Seeing a gallery type image view come to fruition even just through a browser interface would be great, but especially fantastic on the iPhone/iPad app, as I can then use the HDMI or VGA output adapter to hook my iOS device up to a screen at a client’s location and just let it play through.

A browser based file & image viewer is the one thing that’s keeping me on DropBox for client and model file delivery, as I can just send them a URL that doesn’t actually require they install and setup a client software to download or view any files contained within a folder, and the folder gets updated automatically as I make changes to files, or add new files to it.

Of course, if you’re running BTSync on a Linux system (or Windows either, I suppose), there’s nothing to really stop you installing Apache, and whipping up some PHP code to create an interface for this type of functionality yourself, with some MariaDB interaction to configure settings, deal with user authentication, etc).

That’s something I’ll have to have a play with at some point if the BitTorrent folks don’t add this functionality anyway while I’m procrastinating.

Still, even without that, it’s definitely worth the price of entry, so why not give it a try?