| layout | title | permalink |
|---|---|---|
page |
Google Cloud Tutorial |
/gce-tutorial/ |
(We know you won't read until the very bottom once your assignment is running, so we are printing this at the top too since it is super important)
Don't forget to stop your instance when you are done (by clicking on the stop button at the top of the page showing your instances), otherwise you will run out of credits and that will be very sad. :(
If you follow our instructions below correctly, you should be able to restart your instance and the downloaded software will still be available.
For the class project and assignments, we offer an option to use Google Compute Engine for developing and testing your implementations. This tutorial lists the necessary steps of working on the assignments using Google Cloud. We expect this tutorial to take about an hour. Don't get intimidated by the steps, we tried to make the tutorial detailed so that you are less likely to get stuck on a particular step. Please tag all questions related to Google Cloud with google_cloud on Piazza.
This tutorial goes through how to set up your own Google Compute Engine (GCE) instance to work on the assignments. Each student will have $100 in credit throughout the quarter. When you sign up for the first time, you also receive $300 credits from Google by default. Please try to use the resources judiciously. But if $100 ends up not being enough, we will try to adjust this number as the quarter goes on. Note: for assignment 1, we are only supporting python version 3.5.3.
First, if you don't have a Google Cloud account already, create one by going to the Google Cloud homepage and clicking on Compute. When you get to the next page, click on the blue TRY IT FREE button. If you are not logged into gmail, you will see a page that looks like the one below. Sign into your gmail account or create a new one if you do not already have an account.
If you already have a gmail account, it will direct you to a signup page which looks like the following.
Click the appropriate yes or no button for the first option, and check yes for the latter two options after you have read the required agreements. Press the blue Agree and continue button to continue to the next page to enter the requested information (your name, billing address and credit card information). Once you have entered the required information, press the blue Start my free trial button. You will be greeted by a page like this:
To change the name of your project, click on Manage project settings on the Project info button and save your changes.
To launch a virtual instance, go to the Compute Engine menu on the left column of your dashboard and click on VM instances. Then click on the blue CREATE button on the next page. This will take you to a page that looks like the screenshot below. (NOTE: the values that you see in the screenshot are the default values, please keep reading below for the values you will need to fill in.)
Make sure that the Zone is set to be us-west1-b (especially for assignments where you need to use GPU instances). Under Machine type pick the 8 vCPUs option. Click on the customize button under Machine type and make sure that the number of cores is set to 8 and the number of GPUs is set to None (we will not be using GPUs in assignment 1 and this tutorial will be updated with instructions for GPU usage). Click on the Change button under Boot disk, choose OS images, check Ubuntu 16.04 LTS and click on the blue select button. Check Allow HTTP traffic and Allow HTTPS traffic. Click on disk and then Disks and uncheck Delete boot disk when instance is deleted (Note that the "Disks" option may be hiding under an expandable URL at the bottom of that webform). Click on the blue Create button at the bottom of the page. You should have now successfully created a Google Compute Instance, it might take a few minutes to start running. Your screen should look something like the one below. When you want to stop running the instance, click on the blue stop button above.
Take note of your <YOUR-INSTANCE-NAME>, in this case, my instance name is instance-2.
Now that you have created your virtual GCE, you want to be able to connect to it from your computer. The rest of this tutorial goes over how to do that using the command line. First, download the Google Cloud SDK that is appropriate for your platform from here and follow their installation instructions. NOTE: this tutorial assumes that you have performed step #4 on the website which they list as optional. When prompted, make sure you select us-west1-b as the time zone. The easiest way to connect is using the gcloud compute command below. The tool takes care of authentication for you. On OS X, run:
./<DIRECTORY-WHERE-GOOGLE-CLOUD-IS-INSTALLED>/bin/gcloud compute ssh --zone=us-west1-b <YOUR-INSTANCE-NAME>
See this page for more detailed instructions. You are now ready to work on the assignments on Google Cloud.
Run the following command to download the current assignment onto your GCE:
wget http://cs231n.stanford.edu/assignments/2017/spring1617_assignment1.zip
where X is the assignment number (1, 2 or 3). Run:
sudo apt-get install unzip
and
unzip spring1617_assignment1.zip
to get the contents. You should now see a folder titled assignmentX. To install the necessary dependencies, cd into the assignment directory and run the provided shell script: (Note: you will need to hit the [enter] key at all the "[Y/n]" prompts)
cd assignment1
./setup_googlecloud.sh
You will be prompted to enter Y/N at various times during the download. Press enter for every prompt. You should now have all the software you need for assignmentX. If you had no errors, you can proceed to work with your virtualenv as normal.
I.e. run
source .env/bin/activate
in your assignment directory to load the venv, and run
deactivate
to exit the venv. See assignment handout for details.
Many of the assignments will involve using Jupyter Notebook. Below, we discuss how to run Jupyter Notebook from your GCE instance and use it on your local browser.
Change the Extenal IP address of your GCE instance to be static (see screenshot below).
To Do this, click on the 3 line icon next to the Google Cloud Platform button on the top left corner of your screen, go to Networking and External IP addresses (see screenshot below).
To have a static IP address, change Type from Ephemeral to Static. Enter your preffered name for your static IP, mine is assignment-1 (see screenshot below). And click on Reserve. Remember to release the static IP address when you are done because according to this page Google charges a small fee for unused static IPs. Type should now be set to Static.
Take note of your Static IP address (circled on the screenshot below). I used 104.196.224.11 for this tutorial.
One last thing you have to do is adding a new firewall rule allowing TCP acess to a particular <PORT-NUMBER>. I usually use 7000 or 8000 for <PORT-NUMBER>. Click on the 3 line icon at the top of the page next to Google Cloud Platform. On the menu that pops up on the left column, go to Networking and Firewall rules (see the screenshot below).
Click on the blue CREATE FIREWALL RULE button. Enter whatever name you want: I used assignment1-rules. Enter 0.0.0.0/0 for Source IP ranges and tcp:<PORT-NUMBER> for Allowed protocols and ports where <PORT-NUMBER> is the number you used above. Click on the blue Create button. See the screen shot below.
The following instructions are excerpts from this page that has more detailed instructions.
On your GCE instance check where the Jupyter configuration file is located:
ls ~/.jupyter/jupyter_notebook_config.py
Mine was in /home/timnitgebru/.jupyter/jupyter_notebook_config.py
If it doesn’t exist, create one:
# Remember to activate your virtualenv ('source .env/bin/activate') so you can actually run jupyter :)
jupyter notebook --generate-config
Using your favorite editor (vim, emacs etc...) add the following lines to the config file, (e.g.: /home/timnitgebru/.jupyter/jupyter_notebook_config.py):
c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.port = <PORT-NUMBER>
Where <PORT-NUMBER> is the same number you used in the prior section. Save your changes and close the file.
The instructions below assume that you have SSH'd into your GCE instance using the prior instructions, have already downloaded and unzipped the current assignment folder into assignmentX (where X is the assignment number), and have successfully configured Jupyter Notebook.
If you are not already in the assignment directory, cd into it by running the following command:
cd assignment1
If you haven't already done so, activate your virtualenv by running:
source .env/bin/activate
Launch Jupyter notebook using:
jupyter-notebook --no-browser --port=<PORT-NUMBER>
Where <PORT-NUMBER> is what you wrote in the prior section.
On your local browser, if you go to http://<YOUR-EXTERNAL-IP-ADDRESS>:<PORT-NUMBER>, you should see something like the screen below. My value for <YOUR-EXTERNAL-IP-ADDRESS> was 104.196.224.11 as mentioned above. You should now be able to start working on your assignments.
Don't forget to stop your instance when you are done (by clicking on the stop button at the top of the page showing your instances). You can restart your instance and the downloaded software will still be available.