How to synchronise our data in our local drive to our server disk?

Hello all,

This article discusses about my own strategy in synchronising my data from the server's staff home directory to my local machine drive. First, let understand what is my motivation to write this article. But of course you may have tons of other reasons why you need to synchronise your disks.

Even though Linux offers great capabilities and advantages as mentioned in the previous article, we sometimes cannot really escape from the MS. Windows (Windows) domination. The fact that MS. Windows has dominated operating system (OS) markets since few decades ago leads to abundant users have used and become familiar with this OS. Big companies and some funding agencies often use MS. Offices for their business documentations as well as their publications.  If we work collaboratively with them, either in applying a consortium funding or writing a deliverable to a sponsor company, we may not always be able to avoid writing the reports using MS. Word.

Although personally I think Linux offers more benefits to scientists, but we cannot escape from windows world, since most funding agencies and sponsor companies typically use MS. Word template for grant proposal or other documentations.   

In order to compromise this, at Helsinki University, I have used a platform, called Virtual Windows (VW). As I mentioned in another article, here we are provided a server disk for every staff to save their work. The disk is typically used as a virtual hard-drive in VW environment. The size of the disk is not too large (that is only 20 GB), but it is enough to store some document files.  The disk can be mounted easily to our local machine with whatever OS that we are using. In my case (Linux), the disk can be easily mounted to the local machine, using this command:

# sshfs zaidanma@melkinpaasi.cs.helsinki.fi:/home/ad/home/z/zaidanma adhome

"adhome" is the folder in my local machine, in which the disk is mounted.

Although we think our data is safe there, but the data back up is still crucial for this disk because the data is stored in only one location. One way to do it is by using rsync command as discuss before or by using unison command, which was also discussed briefly.

For this drive, I would like to use unison to synchronise between my local drive and this drive (although, I can mount it in the local machine). Let see what I did here. First we can login to the remote disk by ssh to the server by typing this command:

# ssh zaidanma@melkinpaasi.cs.helsinki.fi

or the old disk can be called:

# ssh pangolin.it.helsinki.fi

Then, it will ask us to insert our password. Once the password is correct, we are in the remote disk. To access my home directory, I can navigate to (windows):

# cd /home/ad/home/z/zaidanma/

or Linux home directory (i.e. Desktop is shared with AD Windows) by typing:

# cd /home/ad/lxhome/z/zaidanma/Linux/ 

So, we can see the files that are there and so on. In my case, I choose to use Document folder to store all my works via VW. Then, we can go back to our local machine by simply typing:

# exit

We comeback now to our local machine and we should create a back-up folder, here I make:

# mkdir adhome_Documents

Since my remote disk already contains the data, to be more careful, I do not use unison command yet on the first place, because it may erase the data in the remote disk since my new "adhome_Documents" folder is also empty. To be in safe side, let use rsync command first.

# rsync -av zaidanma@melkinpaasi.cs.helsinki.fi:/home/ad/home/z/zaidanma/Documents/ adhome_Documents/

or the below can be used to synchronise and remove the deleted files too:

# rsync -av --delete-after zaidanma@melkinpaasi.cs.helsinki.fi:/home/ad/home/z/zaidanma/Documents/ adhome_Documents/

This synchronises all files that are in "Documents" folder in my remote disk to the folder of "adhome_Documents" in my local machine. Since "adhome_Documents" is empty, it just simply copies all files there from "Document" folder in the remote disk. Now, both directories have the same files. In the future, I just want to synchronise these two directories by simply typing (remember here we have to use Linux home directory because unison command will not work on Windows direcoty):

# unison adhome_Documents/ ssh://melkinpaasi.cs.helsinki.fi////home/ad/lxhome/z/zaidanma/Linux/Documents/

If we want to keep using rsync, this is also powerful:

# rsync -av --delete-after zaidanma@melkinpaasi.cs.helsinki.fi:/home/ad/lxhome/z/zaidanma/Linux/Documents/ adhome_Documents/

This means that our local folder just receives the updates from the remote folder.

By:
Martha Arbayani Zaidan
Helsinki, 15 November 2017