How to Upload File to Google Drive using Python Script?

Learn How to Upload a File to Google Drive using Python.

Objective For ‘How To Upload File To Google Drive Using Python Script’

This step-by-step recipe will show you how to upload files to Google Drive Python.

Why Do Data Engineers Extract From/Upload To Google Drive Python?

When working on big data projects, extracting data from Google Drive into orchestration workflows to initially store the data in data lakes, followed by a series of operations like data validation, cleansing, and transformation, is widely used to gather business insights from the data. In addition to extracting data from Google Drive, data engineers also upload file to Drive Python scripts. This is useful for storing data in a central location where other users can easily access it. Python can also be used to automate the process of uploading data to Google Drive, saving data engineers a lot of time.

Steps For Uploading Files on Google Drive Using Python

In the following Python Sample Code, we will upload files to Google Drive using Python and use them in data flow orchestration processes.

Pre-Requisites For Google Drive Upload Python

  • Install the pydrive python module as follows: pip install pydrive

  • The below codes can be run in Jupyter notebook or any python console

Get Closer To Your Dream of Becoming a Data Scientist with 70+ Solved End-to-End ML Projects

Step 1: Import the Libraries

from pydrive.auth import GoogleAuth from pydrive.drive import GoogleDrive

Step 2: OAuth Made Easy

Follow the steps to Get Authentication for Google Service API in the below link: Get Authentication for Google Service API

Download client_secrets.json from Google API Console and OAuth2.0 is done in two lines. You can customize the behavior of OAuth2 in one settings file settings.yaml

gauth = GoogleAuth() drive = GoogleDrive(gauth)

Above steps together as follows :

from pydrive.auth import GoogleAuth from pydrive.drive import GoogleDrive gauth = GoogleAuth() drive = GoogleDrive(gauth)

Step 3: Upload files to your Google Drive

upload_file_list = ['1.jpg', '2.jpg'] for upload_file in upload_file_list: gfile = drive.CreateFile({'parents': [{'id': '1pzschX3uMbxU0lB5WZ6IlEEeAUE8MZ-t'}]}) # Read file and set it as the content of this instance. gfile.SetContentFile(upload_file) gfile.Upload() # Upload the file.

The output of the above code:

Upload files to your Google Drive

  • The above code uploads my two local files, 1.jpg and 2.jpg, to my Google Drive folder test/. To do that, the pydrive library will create two files in Google Drive and then read and upload the two files to the corresponding folder.

  • Note that we need to provide the id of the corresponding Google Drive folder. In this example, the test folder's ID is 1pzschX3uMbxU0lB5WZ6IlEEeAUE8MZ-t. You can get the Google Drive folder ID from the browser.

  • For example: when we open the test folder in my Google Drive, the browser shows the address as https://drive.google.com/drive/folders/1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi. Then the corresponding ID for the test folder is the part after the last \ symbol, which is 1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi.

Step 4: List out files from Google Drive

We can also list all files from the specific folder in google drive as follows :

file_list = drive.ListFile({'q': "'{}' in parents and trashed=false".format('1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi')}).GetList() for file in file_list: print('title: %s, id: %s' % (file['title'], file['id']))

The output of the above code:

List out files from Google Drive

Step 5: Download the files from Google Drive

We can also download the files from Google Drive as follows. Note - after listing the files only we can download the file.

for i, file in enumerate(sorted(file_list, key = lambda x: x['title']), start=1): print('Downloading {} file from GDrive ({}/{})'.format(file['title'], i, len(file_list))) file.GetContentFile(file['title'])

The output of the above code:

Download the files from Google Drive

In the above snapshot files are downloaded from the specific folder, Note here files will download where the code will be executed.

Step 6: Create the Text files in Google Drive

We can also write files directly to Google Drive using the following code:

# Create a GoogleDriveFile instance with title 'test.txt'. file1 = drive.CreateFile({'parents': [{'id': '1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi'}],'title': 'test.txt'}) # Set content of the file from the given string. file1.SetContentString('Hello World!') file1.Upload()

The output of the above code: test.txt file is created in google drive.

Create the Text files in Google Drive

Explore More Data Science and Machine Learning Projects for Practice. Fast-Track Your Career Transition with ProjectPro

Step 7: Read the content of the text file directly from Google Drive

Also, we can read the file directly from Google Drive using the below code:

file2 = drive.CreateFile({'id': file1['id']}) file2.GetContentString('test.txt')

The output of the above code:

Read the file directly from Google Drive

In the above snapshot reading, the file's content is "Hello world."

How Does Python write to Google Drive?

The first step is to install the pydrive library using the following command:

pip install pydrive

 

Once the pydrive library is installed, you can use it to write to Google Drive. The following code shows how to do this:

 

import pydrive

# Create a PyDrive client

client = pydrive.auth.GoogleAuth()

client.authenticate()

 

# Get the file to write to

file_path = '/path/to/file.txt'

 

# Create a Google Drive file object

file = pydrive.DriveFile(file_path)

 

# Open the file in write mode

with open(file_path, 'w') as f:

  f.write('This is a test.')

 

# Save the file to Google Drive

file.update()

How Does Python Send File to Google Drive?

Once you have successfully installed the pydrive library, you can use it to send a file to Google Drive, as shown in the following code:

 

import pydrive

 

# Create a PyDrive client

client = pydrive.auth.GoogleAuth()

client.authenticate()

 

# Get the file to upload

file_path = '/path/to/file.txt'

 

# Create a Google Drive file object

file = pydrive.DriveFile(file_path)

 

# Upload the file to Google Drive

file.upload()

How Does Python Save to Google Drive?

You can save a file to Python Google Drive using the google-cloud-storage library. The google-cloud-storage library is a Python library that provides a high-level API for Google Cloud Storage. To use the google-cloud-storage library, you first need to install it using the following command:

pip install google-cloud-storage

 

Once the google-cloud-storage library is installed, you can use it to save a file to Google Drive with the following code:

 

from google.cloud import storage

# Create a Google Cloud Storage client

client = storage.Client()

 

# Get the file to save

file_path = '/path/to/file.txt'

 

# Create a bucket

bucket = client.bucket('my-bucket')

 

# Create a blob

blob = bucket.blob('my_file.txt')

 

# Open the file in binary mode

with open(file_path, 'rb') as f:

  blob.upload_from_file(f)



What Users are saying..

profile image

Jingwei Li

Graduate Research assistance at Stony Brook University
linkedin profile url

ProjectPro is an awesome platform that helps me learn much hands-on industrial experience with a step-by-step walkthrough of projects. There are two primary paths to learn: Data Science and Big Data.... Read More

Relevant Projects

Hands-On Approach to Causal Inference in Machine Learning
In this Machine Learning Project, you will learn to implement various causal inference techniques in Python to determine, how effective the sprinkler is in making the grass wet.

Mastering A/B Testing: A Practical Guide for Production
In this A/B Testing for Machine Learning Project, you will gain hands-on experience in conducting A/B tests, analyzing statistical significance, and understanding the challenges of building a solution for A/B testing in a production environment.

AWS Project to Build and Deploy LSTM Model with Sagemaker
In this AWS Sagemaker Project, you will learn to build a LSTM model on Sagemaker for sales forecasting while analyzing the impact of weather conditions on Sales.

Detectron2 Object Detection and Segmentation Example Python
Object Detection using Detectron2 - Build a Dectectron2 model to detect the zones and inhibitions in antibiogram images.

Deploy Transformer-BART Model on Paperspace Cloud
In this MLOps Project you will learn how to deploy a Tranaformer BART Model for Abstractive Text Summarization on Paperspace Private Cloud

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

GCP MLOps Project to Deploy ARIMA Model using uWSGI Flask
Build an end-to-end MLOps Pipeline to deploy a Time Series ARIMA Model on GCP using uWSGI and Flask

Deploying Machine Learning Models with Flask for Beginners
In this MLOps on GCP project you will learn to deploy a sales forecasting ML Model using Flask.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Learn to Build a Siamese Neural Network for Image Similarity
In this Deep Learning Project, you will learn how to build a siamese neural network with Keras and Tensorflow for Image Similarity.