Mounting disk from a computer cluster (remote disk) to our local machine
Hello all,
When we are working with a super computer cluster, traditionally we often do not have visualisation access there. If we want to navigate the folders or files, we may see them, using for example a "Linux terminal", by using command such as ls, cd and pwd.
If we run a program there and it then produces numerous results, sometimes we would like to visualise and analyse the results immediately in our local machine. Of course, we may visualise and analyse quickly the result data by using gnuplot [1] directly in the cluster or by copying it to our local machine by scp or rsync commands.
However, I found sometimes it is inconvenient to use gnuplot because we need to know the commands there. Copying the result to our local machine may also take time and effort
The above methods may be powerful if we know well the Unix commands for copying or be comfortable in using gnuplot. However, an alternative solution may be to mounted the cluster disk to our local machine directly. This way we can visualise the files and run other software locally to analyse the results.
We can use sshfs command to mount a remote disk and command fusermount to unmount the remote disk. Since I use Taito supercluster, I follow the instruction from the CSC website [2], In my case, I create a mounted disk folder in my home directory:
# mkdir MountedDisks
Then we go to that directory by:
# cd MountedDisks
There we created two new folders, one is for home directory and another one is for work directory.
# mkdir CSC_home
# mkdir CSC_work
After that we can simply mount the remote disks by:
# sshfs zaidanma@taito.csc.fi: ./CSC_home
# sshfs zaidanma@taito.csc.fi: ./CSC_work
Now the remote disks have been disappeared from our local machine.
References:
[1] http://www.gnuplot.info/ (Accessed on 16 November 2017)
[2] https://research.csc.fi/csc-guide-remote-disk-mounts (Accessed on 16 November 2017)
When we are working with a super computer cluster, traditionally we often do not have visualisation access there. If we want to navigate the folders or files, we may see them, using for example a "Linux terminal", by using command such as ls, cd and pwd.
If we run a program there and it then produces numerous results, sometimes we would like to visualise and analyse the results immediately in our local machine. Of course, we may visualise and analyse quickly the result data by using gnuplot [1] directly in the cluster or by copying it to our local machine by scp or rsync commands.
However, I found sometimes it is inconvenient to use gnuplot because we need to know the commands there. Copying the result to our local machine may also take time and effort
Example of gnuplot window, we can perform plotting function directly on Unix environment by gnuplot functions. |
We can use sshfs command to mount a remote disk and command fusermount to unmount the remote disk. Since I use Taito supercluster, I follow the instruction from the CSC website [2], In my case, I create a mounted disk folder in my home directory:
# mkdir MountedDisks
Then we go to that directory by:
# cd MountedDisks
There we created two new folders, one is for home directory and another one is for work directory.
# mkdir CSC_home
# mkdir CSC_work
After that we can simply mount the remote disks by:
# sshfs zaidanma@taito.csc.fi: ./CSC_home
# sshfs zaidanma@taito.csc.fi: ./CSC_work
After this, we will be asked to type our credential (password) to allow this process. These commands mount the remote disk in the Taito supercluster to our local machine. Now we can do whatever we want by having the remote disk's files in our local machine. For example, you can navigate the files by:
# cd CSC_home
# ls
You can see now all the files in the remote disk are already in the CSC:home folder. In order to unmount the remote disk, we can simply type:
# fusermount -u CSC_home
# fusermount -u CSC_work
Now the remote disks have been disappeared from our local machine.
By:
Martha Arbayani Zaidan
Helsinki, 16 November 2017
References:
[1] http://www.gnuplot.info/ (Accessed on 16 November 2017)
[2] https://research.csc.fi/csc-guide-remote-disk-mounts (Accessed on 16 November 2017)
Comments
Post a Comment