HPC Grid Tutorial: How to Setup and Use Google Drive

GSuite for Education's Google Drive Cloud Storage is available to Wayne State University (WSU) and WSU affiliated Faculty, Staff, and Students that have an active AccessID. Each user has unlimited storage which may be useful for archival storage, however users should be aware of the following information before utilizing Google Drive.

 

  • WSU administers all GSuite accounts
  • Google Drive is not suitable for either FERPA or HIPAA protected data
  • Google will process and analyze your data.
  • Data may be stored in countries outside of the United States.
  • Google limits the amount of data that may be transferred during a short period of time so it may take a long time to transfer large amounts of data into or out of Google Drive.
  • WSU does not pay Google for this service. Google may change their offering at any time and Wayne State is not guaranteeing this product should their offering change.
  • We recommend that Google Drive be used only as a duplicate or backup for your data.
  • Please review Google's documentation for terms of service and privacy:

 

The WSU HPC Services Team can make a best effort attempt to assist with the utilization of this offering but cannot guarantee results based on Google's services. Use at your own risk.

 

Please follow these steps to transfer files from your WSU Grid home directory to your Google Drive account:

 

1. Make sure your session allows for X11 forwarding. Follow the X11 tutorial if needed here

 

2. Configure rclone. Enter the following command and follow the onscreen instructions: rclone config

3. Type n to make a new remote. 

You will be asked to name your drive. Be sure to remember the name because you will need it later.

4. Next, it will ask what type of storage you wish to configure. Google Drive is listed as number 10. Type: 10 

5. The next steps ask for a 'Google Application Client Id' and 'Google Application Client Secret'. These can be left blank as the prompt states, press Enter to continue. 

The same applies for the 'Service Account Credentials JSON file path'. Press enter to continue. 

6. Continue through the last prompts. 

A browser window should open that will ask you to log into your Google Account and authorize access for rclone. After doing so it will give you a verification code to enter. 

You have now successfully configured your Google Drive!

7. To copy a local directory to a drive directory called backup type: rclone copy /home/sourceDriveName:backup

3. To list all folders in the top level of your drive simply type: rclone lsd DriveName:

 

For additional rclone commands see: https://rclone.org/drive