How to use google collab

Google Colab 101 Tutorial with Python — Tips, Tricks, and FAQ

Last Updated on October 21, 2021 by Editorial Team

An in-depth tutorial on how to use Google Colab with Python, along with Colab’s tips, tricks, and FAQ

Table of Contents

Introduction

Google Colab is a project from Google Research, a free, Jupyter based environment that allows us to create Jupyter [programming] notebooks to write and execute Python [1](and other Python-based third-party tools and machine learning frameworks such as Pandas, PyTorch, Tensorflow, Keras, Monk, OpenCV, and others) in a web browser.

A programming notebook is a type of a shell or kernel in the form of a word processor, where we can write and execute code. The data required for processing in Google Colab can be mounted into Google Drive or imported from any source on the internet. Project Jupyter is an open-source software organization that develops and supports Jupyter notebooks for interactive computing [4].

Google Colab requires no configuration to get started and provides free access to GPUs. One of the main functionalities of Google Colab is that it allows anyone to share live code, mathematical equations, data visualizations, data processing (cleaning and transformation), numerical simulations, machine learning models, and many other projects with others.

Why We Use Google Colab?

Google Colab has unique and critical features:

Start Google Colab

Python code can be executed directly on the web browser by using Colab. We can launch it with the URL below:

The launch window opens with a popup offering many features:

Figure 1: Screenshot of Google Colab’s start page.

It provides options to create a notebook as well as to upload and select from different sources such as:

Uploading a Notebook from GitHub

Python code can be directly uploaded from Github by using its project’s URL or by searching the organization or user. The steps below highlight how to upload a project using a Github URL:

Similarly, the code can be uploaded directly from Google Drive by filtering saved notebooks by name, date, owner, or modified date.

Figure 5: Screenshot showing how to upload a notebook directly from Google Drive to Google Colab.

Uploading Data from Kaggle

Data from Kaggle can be uploaded directly into Colab for processing. An API token from Kaggle is required to accomplish the data import.

Steps to generate API token from Kaggle

Steps to upload data from Kaggle

Save the “kaggle.json” file on your local computer.

Install the Kaggle package

Upload the local file “kaggle.json”

Figure 8: Screenshot of the output of the uploaded file “kaggle.json”

Check if the Colab notebook connects with Kaggle correctly.

Figure 9: Screenshot showing a dataset list from Kaggle.

Download any competition data from Kaggle (i.e., competition name — predict future sales)

Figure 10: Screenshot showing the output of data downloads from Kaggle.

Data from Kaggle will be downloaded and uploaded in Colab, like:

Figure 11: Screenshot of Google Colab showing the uploaded data from Kaggle.

Read Files from Google Drive

Google Colab provides functionality to read data from google drive too.

Mount Google Drive

This will ask Google to drive authorization code.

Figure 12: Screenshot showing Google Colab asking you to insert Google Drive’s authorization code.

Input box for the authorization code

Click on the link and generate the authorization code.

Read a CSV file from the drive.

Figure 13: Output from the test CSV file on Google Drive.

Setting up Hardware Accelerator GPU for Runtime

Google Colab provides a free cloud service with a GPU hardware accelerator. High configurations GPU machines are very costly and required in machine learning and deep learning to simultaneously process multiple computations.

Why are GPUs required in Machine Learning or Deep Learning?

Nowadays, GPUs are dominant in machine learning and deep learning due to the optimized capability of more compute-intensive workloads and streaming memory models.

GPUs give outstanding performance through parallelism and can launch millions of threads in one call. They function unusually better than CPUs even though GPUs may have a lower clock speed and the absence of many-core management features compared to a CPU.

Setup Hardware Accelerator GPU in Colab

Steps to setup GPU:

Checking details about the GPU in Colab.

Import important packages

Check the GPU accelerator

Figure 15: Screenshot of GPU’s accelerator in Google Colab.

Check the hardware used for the GPU.

Figure 16: Screenshot showcasing the details about the GPU in our session.

Code Example Using a GPU

Check the number of available GPUs without selecting GPU in Runtime. Keep it set to “ None.”

Figure 17: Hardware accelerator set to none. Figure 18: Hardware accelerator is None, so the GPU’s value is 0.

Select Hardware accelerator in runtime to GPU.

Figure 19: Screenshot of hardware accelerator set to GPU. Figure 20: Hardware accelerator is GPU so the value of GPU is 1.

Multiply Tensors on GPU:

Figure 21: Tensor multiplication result.

Clone a GitHub Repository to Google Drive

GitHub repository code can be cloned and stored into local Google Drive.

Steps to clone the GitHub repository to Google Drive:

Colab Magic

There are some interestingly amazing tricks which Colab offers. It provides multiple commands that provide quick operations in short. These commands are used with a % prefix.

List All Magic Commands

List Local Directories

Figure 27: List of local directories.

Get Notebook History

Figure 28: CPU and wall time.

How long has the system been running?

Figure 29: Displaying system uptime.

Display available and used memory

Figure 30: Display available and used memory.

Display the CPU specification

Figure 31: Display CPU specification.

List all running VM processes.

Figure 32: Display all VM running processes.

Embed HTML Text

Design HTML Form

Figure 33: Generating a form in Google Colab. Figure 34: Cell execution output.

Plotting

Google Colab can be used for data visualization, as well. The following code and graph show a plot containing more than one polynomial, Y = X³+X²+X [3].

Figure 35: A graph showing more than one polynomial.

The following code and map are used to graph a heat map.

Figure 36: Heatmap.

TPU (Tensor Processing Unit) in Google Colab

We use Tensor Processing Units (TPUs) for acceleration on a Tensorflow graph. They are an AI accelerator application-specification integrated circuit (ASIC) specially designed for the neural network machines. Google has developed this processing unit.

TPU has a wonder configuration of teraflops, floating-point performance, and others. Each TPU packs up to 180 teraflops of floating-point performance and 64 GB of high-bandwidth memory on a single board. A teraflop is the measurement of a computer’s speed. Its speed can be a trillion floating-point operations per second.

Setup TPU in Colab

Steps to setup a TPU in Google Colab:

Check Running on TPU Hardware Accelerator

It requires the TensorFlow package. Below code, implementations check whether Colab has set TPU accelerator or not?

Figure 38: Running on TPU hardware accelerator.

If the TPU is not set, then this code will encounter an error.

Conclusion

Google Colab or Colaboratory, from Google Research, is a Jupyter notebook environment to execute python-based code to build a machine learning or deep learning model.

It is completely free (unless you would like to go pro) and provides GPU and TPU hardware accelerators. It is effortless to use and share due to the zero-configuration features requirement.

It allows combining executable code and rich text in a single document and images, HTML, LaTex, and others. It has a vital machine learning library like TensorFlow already installed, so it is perfect for machine learning and deep learning model building. Colab is outstanding for developing neural networks.

We can achieve parallelism and execution of multiple threads by using CPU based hardware accelerator. We can share Google Colab notebooks publicly as a tutorial notebook. The insertion of HTML tags and the text’s styling builds an attractive and meaningful notebook for tutorials, and the insertion of text with code is remarkably helpful for explaining code flow and logic.

Data scientists and machine learners can harness Python libraries’ full power to analyze and visualize data, and Google Colab can import data directly from Kaggle and upload code from GitHub.

DISCLAIMER: The views expressed in this article are those of the author(s) and do not represent the views of Carnegie Mellon University nor other companies (directly or indirectly) associated with the author(s). These writings do not intend to be final products, yet rather a reflection of current thinking and being a catalyst for discussion and improvement.

All images are from the author(s) unless stated otherwise.

WiktorJ/google-colab-tutorial

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

Basic tutorial on how to (relatively) conveniently use Google Colab with Google Drive and Github

It’s a Jupyter notebook environment that requires no setup to use. Moreover it gives you free GPU access to run your code.

In simple words Colab is a free hosted Jupyter notebook with access to GPU.

Make sure you trust the authors of any notebook before executing it.

With a local connection, the code you execute can read, write, and delete files on your computer. Connecting to a Jupyter notebook server running on your local machine can provide many benefits. With these benefits come serious potential risks. By connecting to a local runtime, you are allowing the Colaboratory frontend to execute code in the notebook using the local resources on your machine. This means that the notebook could:

Before attempting to connect to a local runtime, make sure you trust the authors of the notebook and ensure you understand what code is being executed. For more information on the Jupyter notebook server’s security model, consult Jupyter’s documentation.

Colab stores the notebooks in your Google Drive. (Although limited integration with github is also possible, more on this in Quick Start)

All the files uploaded to Colab are ephemeral, they are gone after reloading Colab Runtime.

The recommended way is to upload your data to Google Drive then mount the drive in the Runtime (instructions in Quick Start)

Colab provides integration with Github. It’s possible to load notebooks directly from repositories and commit changes made in Colab. However, there are some limitations (more in Quick Start)

This tutorial shows just one way of using Colab, everyone should adjust the flow accordingly.

Create Github repository and push Jupyter Notebook (can be empty) into it (if you already have a notebook that you want to run in colab in some other repository you can skip this step.)

Go to https://colab.research.google.com/, pick GITHUB source and put the repository details

Click on NEW PYTHON 3 NOTEBOOK and that’s (almost) it! Now you can write and run your code in Colab using Google GPUs

Accessing data on Google Drive

Code below mounts your Google Drive to /content/drive/My Drive directory.

In case google.cloab is not found it assumes that the notebook runs somewhere else and it doesn’t mount the drive.

After that your data are available locally: f = gzip.open(«<>/train-images-idx3-ubyte.gz».format(data_prefix), ‘r’)

Integration with Github

It is more convenient to work on your code locally (e.g. Using JupyterLab or Pycharm) and use Colab only to execute the code on GPU and do some small changes (e.g. Hyper parameters tuning).

Warning This might create conflicts!

Warning This will reload your runtime!

About

Basic tutorial on how to conveniently use Google Colab with Google Drive and Github

Google Colab — The Beginner’s Guide

Whether you are a student interested in exploring Machine Learning but struggling to conduct simulations on enormous datasets, or an expert playing with ML desperate for extra computational power, Google Colab is the perfect solution for you. Google Colab or “the Colaboratory” is a free cloud service hosted by Google to encourage Machine Learning and Artificial Intelligence research, where often the barrier to learning and success is the requirement of tremendous computational power.

Benefits of Colab

Besides being easy to use (which I’ll describe later), the Colab is fairly flexible in its configuration and does much of the heavy lifting for you.

If you prefer to read more before getting started, I recommend the Google Colab FAQ, Google Colab Documentation and Code Snippets, and advice from the helpful community of users on Stack Overflow.

Let’s Begin!

Create a Colab Notebook

Setting GPU Accelerator

The default hardware of Google Colab is CPU or it can be GPU.

Bash commands can be run by prefixing the command with ‘!’.

This command outputs the folders /content and /drive (if it has been mounted). Run the following snippet to change the current folder.

Although most of the commonly used Python libraries are pre-installed, new libraries can be installed using the below packages:

Upload local files

Select the files for upload

For multiple files, the individual key names can be obtained by looping through the uploaded files.

Mounting Google Drive

Run the following code.

Click on the link and enter the api key.

Your drive is now mounted. You can use any files and folders in your drive by using the path as follows

/content is the root folder of Google Colab and has to be appended to all paths used in the notebook.

To import any module

To run a script

Check CPU and RAM specifications

Check GPU specifications

Colab provides the Tesla K80 GPU.

This should get you started with Google Colab. Feel free to ask questions!

How to use Google Colab

If you want to create a machine learning model but say you don’t have a computer that can take the workload, Google Colab is the platform for you. Even if you have a GPU or a good computer creating a local environment with anaconda and installing packages and resolving installation issues are a hassle.
Colaboratory is a free Jupyter notebook environment provided by Google where you can use free GPUs and TPUs which can solve all these issues.

Getting Started

To start working with Colab you first need to log in to your google account, then go to this link https://colab.research.google.com.

Opening Jupyter Notebook:
On opening the website you will see a pop-up containing following tabs –

EXAMPLES: Contain a number of Jupyter notebooks of various examples.
RECENT: Jupyter notebook you have recently worked with.
GOOGLE DRIVE: Jupyter notebook in your google drive.
GITHUB: You can add Jupyter notebook from your GitHub but you first need to connect Colab with GitHub.
UPLOAD: Upload from your local directory.

Else you can create a new Jupyter notebook by clicking New Python3 Notebook or New Python2 Notebook at the bottom right corner.

Notebook’s Description:

On creating a new notebook, it will create a Jupyter notebook with Untitled0.ipynb and save it to your google drive in a folder named Colab Notebooks. Now as it is essentially a Jupyter notebook, all commands of Jupyter notebooks will work here. Though, you can refer the details in Getting started with Jupyter Notebook.

Let’s talk about what different here.

Change Runtime Environment:
Click the “Runtime” dropdown menu. Select “Change runtime type”. Select python2 or 3 from “Runtime type” dropdown menu.

Use GPU and TPU:
Click the “Runtime” dropdown menu. Select “Change runtime type”. Now select anything(GPU, CPU, None) you want in the “Hardware accelerator” dropdown menu.

Getting Started With Google Colab

A Simple Tutorial for the Frustrated and Confused

You know it’s out there. You know there’s free GPU somewhere, hanging like a fat, juicy, ripe blackberry on a branch just slightly out of reach.

Beautiful lightning-fast speed waiting just for you.

Wondering how on earth to get it to work? You’re in the right place!

For anyone who doesn’t already know, Google has done the coolest thing ever by providing a free cloud service based on Jupyter Notebooks that supports free GPU. Not only is this a great tool for improving your coding skills, but it also allows absolutely anyone to develop deep learning applications using popular libraries such as PyTorch, TensorFlow, Keras, and OpenCV.

Colab provides GPU and it’s totally free. Seriously!

There are, of course, limits. (Nitty gritty details are available on their faq page, of course.) It supports Python 2.7 and 3.6, but not R or Scala yet. There is a limit to your sessions and size, but you can definitely get around that if you’re creative and don’t mind occasionally re-uploading your files…

Colab is ideal for everything from improving your Python coding skills to working with deep learning libraries, like PyTorch, Keras, TensorFlow, and OpenCV. You can create notebooks in Colab, upload notebooks, store notebooks, share notebooks, mount your Google Drive and use whatever you’ve got stored in there, import most of your favorite directories, upload your personal Jupyter Notebooks, upload notebooks directly from GitHub, upload Kaggle files, download your notebooks, and do just about everything else that you might want to be able to do.

Working in Google Colab for the first time has been totally phenomenal and pretty shockingly easy, but it hasn’t been without a couple of small challenges! If you know Jupyter Notebooks at all, you’re pretty much good to go in Google Colab, but there are just a few little differences that can make the difference between flying off to freedom on the wings of free GPU and sitting at your computer, banging your head against the wall…

This article is for anyone out there who is confused, frustrated, and just wants this thing to work!

Setting up your drive

Create a folder for your notebooks

(Technically speaking, this step isn’t totally necessary if you want to just start working in Colab. However, since Colab is working off of your drive, it’s not a bad idea to specify the folder where you want to work. You can do that by going to your Google Drive and clicking “New” and then creating a new folder. I only mention this because my Google Drive is embarrassingly littered with what looks like a million scattered Colab notebooks and now I’m going to have to deal with that.)

If you want, while you’re already in your Google Drive you can create a new Colab notebook. Just click “New” and drop the menu down to “More” and then select “Colaboratory.”

Otherwise, you can always go directly to Google Colab.

Game on!

You can rename your notebook by clicking on the name of the notebook and changing it or by dropping the “File” menu down to “Rename.”

Set up your free GPU

Want to use GPU? It’s as simple as going to the “runtime” dropdown menu, selecting “change runtime type” and selecting GPU in the hardware accelerator drop-down menu!

Get coding!

You can easily start running code now if you want! You are good to go!

Make it better

Want to mount your Google Drive? Use:

Then you’ll see a link, click on that, allow access, copy the code that pops up, paste it in the box, hit enter, and you’re good to go! If you don’t see your drive in the side box on the left, just hit “refresh” and it should show up.

(Run the cell, click the link, copy the code on the page, paste it in the box, hit enter, and you’ll see this when you’ve successfully mounted your drive):

Now you can see your drive right there on the left-hand side of the screen! (You may need to hit “refresh.”) Plus, you can reach your drive any time with

If you’d rather download a shared zip file link, you can use:

That will give you Udacity’s flower data set in seconds!

If you’re uploading small files, you can just upload them directly with some simple code. However, if you want to, you can also just go to the left side of the screen and click “upload files” if you don’t feel like running some simple code to grab a local file.

Google Colab is incredibly easy to use on pretty much every level, especially if you’re at all familiar with Jupyter Notebooks. However, grabbing some large files and getting a couple of specific directories to work did trip me up for a minute or two.

I covered getting started with Kaggle in Google Colab in a separate article, so if that’s what interests you, please check that out!

Importing libraries

Imports are pretty standard, with a few exceptions.

For the most part, you can import your libraries by running import like you do in any other notebook.

PyTorch is different! Before you run any other Torch imports, you’ll want to run

*** UPDATE! (01/29)*** Colab now supports native PyTorch. You shouldn’t need to run the code below, but I’m leaving it up just in case anyone is having any issues!

Then you can continue with your imports. If you try to simply run import torch you’ll get an error message. I really recommend clicking on the extremely helpful links that pop up. If you do, you’ll get that code right away and you can just click on “INSTALL TORCH” to import it into your notebook. The code will pop up on the left-hand side of your screen, and then hit “INSERT.”

Not able to simply import something else that you want with an import statement? Try a pip install! Just be aware that Google Colab wants an exclamation point before most commands.

I did find that Pillow can be sort of buggy, but you can solve that by running

If you get anything below 5.3, go to the “runtime” dropdown menu, restart the runtime, and run the cell again. You should be good to go!

It’s easy to create a new notebook by dropping “File” down to “New Python 3 Notebook.” If you want to open something specific, drop the “File” menu down to “Open Notebook…”

Then you’ll see a screen that looks like this:

As you can see, you can open a recent file, files from your Google Drive, GitHub files, and you can upload a notebook right there as well.

The GitHub option is great! You can easily search by an organization or user to find files. If you don’t see what you’re looking for, try checking the repository drop-down menu!

Always be saving

That should be enough to at least get you up and running on Colab and taking advantage of that sweet, sweet free GPU! Please let me know if you run into any other newbie problems that I might be able to help you with. I’d love to help you if I can!

If you want to reach out or find more cool articles, please come and join me at Content Simplicity!

If you’re new to data science, machine learning, and artificial intelligence, you might want to check out the ultimate beginner’s guide to NumPy!