Skip to main content

Google Drive as a Source

Google Drive is a cloud storage service developed by Google, which allows users to store files in the cloud. This page contains the setup guide and reference information for the Google Drive source connector.

Pre-requisites​

  • Google Cloud Console Account: Make sure you have a Google account and can access the Google Cloud Console.
  • Google Folder URL: URL for the folder you want to sync.
  • OAuth Credentials:
    • Client ID: Client ID for the Google Drive API.
    • Client Secret: Client Secret for the Google Drive API.
    • Refresh Token: Refresh Token for the Google Drive API.
Note: To obtain the Google OAuth credentials for your source, please follow the setup guide below. If you already have them, you can skip the setup guide and create your source.

Setup Guide​

Step 1: Enable Google Drive API​

  1. Go to the API Console/Library page.
  2. Make sure you have selected the correct project from the top.
  3. Find and select the Google Drive API.
  4. Click ENABLE.
  1. Go to the Google Cloud Console. Create a new project if you don't have one.
  2. Navigate to APIs & Services > OAuth consent screen.
  3. Complete the app registration form, then click Save and Continue.

Step 3: Create OAuth2 Credentials​

Next, you’ll need to create OAuth2 credentials from the Google Cloud Console. These credentials provide authorization to access files from your Google Drive.

  1. Go to the Google Cloud Console.
  2. Navigate to APIs & Services > Credentials > Create credentials > OAuth Client IDs.
  3. Choose Desktop as the application type.
  4. Input the name and click on CREATE.
  5. Download the JSON file containing your OAuth2 credentials.

Step 4: Generate Token File​

Now, execute the following code to generate a token.json file.

Installing Dependencies​

pip install google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client

Generating Tokens​

from google_auth_oauthlib.flow import InstalledAppFlow
from google.oauth2.credentials import Credentials
import json

SCOPES = ['https://www.googleapis.com/auth/drive']

creds_file_path = "<path_to_credentials_file.json>"
flow = InstalledAppFlow.from_client_secrets_file(creds_file_path, SCOPES)
creds = flow.run_local_server(port=0)

with open('token.json', 'w') as token_file:
    json.dump(json.loads(creds.to_json()), token_file, indent=2)

print("Access token saved to token.json")

You will be redirected to a page on your browser where you can autheticate your account. After authentication a token.json will be created in the same directory which will have all the required creedentials.

Step 5: Setting Up Google Drive Source on TIR​

  1. Go to TIR
  2. Go to Data Syncer > Sources > Create Source.
  3. Choose Google Drive.
  4. Add your Folder URL from where you want your data to be ingested.
  5. Add the Client ID, Client Secret and Refresh Token from the token.json that was created in the step above.
  6. Click CREATE.