Google Drive as a Source

Google Drive is a cloud storage service developed by Google, which allows users to store files in the cloud. This page contains the setup guide and reference information for the Google Drive source connector.

Pre-requisites

• Google Cloud Console Account: Make sure you have a Google account and are able to access the Google Cloud Console

• Google Folder URL: URL for the folder you want to sync

• OAuth Credentials

  1. Client ID: Client ID for the Google Drive API

  2. Client Secret: Client Secret for the Google Drive API

  3. Refresh Token: Refresh Token for the Google Drive API

Note

To obtain the Google OAuth credentials for your source, please follow the setup guide below. If you have them already you can skip the setup guide and create your source

Setup guide

Step 1: Oauth Consent screen

1. Go to the Google Cloud Console, you’ll have to create a new project if you don’t have one.

2. Navigate to APIs & Services > OAuth consent screen.

3. Complete the app registration form, then click Save and Continue.

Step 2: Create OAuth2 Credentials

Next, you’ll need to create OAuth2 credentials from the Google Cloud Console. These credentials provide authorization to access files from your Google Drive. Learn more

1. Go to the Google Cloud Console.

2. Navigate to APIs & Services > Credentials > Create credentials > OAuth Client IDs.

3. Choose Desktop as the application type.

4. Input the name and click on CREATE.

5. Download the JSON file containing your OAuth2 credentials.

Step 3: Generate Token file

Now execute the following code to generate a token.json file

Installing Dependencies

pip install google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client

Generating tokens

from google_auth_oauthlib.flow import InstalledAppFlow

SCOPES = ['https://www.googleapis.com/auth/drive']

creds_file_path = "<path_to_credentials_file.json>"
flow = InstalledAppFlow.from_client_secrets_file(
    creds_file_path,
    SCOPES
)

creds = flow.run_local_server(port=0)

with open('token.json', 'wb') as token:
    pickle.dump(creds, token)

You will be redirected to a page on your browser where you can autheticate your account. After authentication a token.json will be created in the same directory which will have all the required creedentials.

Step 4: Setting Up Google Drive Source on TIR

1. Go to TIR

2. Go to Data Syncer > Sources > Create Source.

3. Choose Google Drive.

4. Add your Folder URL from where you want your data to be ingested.

5. Add the Client ID, Client Secret and Refresh Token from the token.json that was created in the step above.

6. Click CREATE.