CS6963 Distributed Systems

Lab Setup

This walk-through should help you setup the tools you'll need for this class, and it should familiarize you with the work flow you'll use for development and hand-in.

You'll need to complete these steps, which are detailed below:

  1. Create a CADE account.
  2. Get go working.
  3. Create a CS6963 gitlab account (be sure to use your UnID (uXXXXXXX) as your username.)
  4. Get added to the cs6963 group on gitlab.
  5. Fork the cs6963/labs repo to a private repo under your own gitlab user.
  6. Create an ssh public/private key pair on CADE.
  7. Configure git on CADE.
  8. Clone your private repo to a local repo on CADE.
  9. Take a look at Lab 1 and see if you can make a local commit and push to your private gitlab repo.
  10. Check the lab submission status after an hour or so and see if your commit shows up.
  11. Perhaps start learning a bit of Go.

Canvas

Please try these steps, if you get stuck please post to the Canvas discussion boards. We'll try to revise and update this document with all of the needed operational details of how to install and use the tools needed to complete the class projects and labs.

Create and setup a CADE Account

CADE manages two clusters that you can use to do your development and testing for all of the class projects. You are free to use other machines and environments, but all grading will be done on these machines. Please test your solutions on these machines.

Check with CADE if you need to setup an account.

CADE machines all share your home directory, so you needn't log in to the same machine each time to continue working.

Setting up and installing software

Shell to a CADE machine

After you have an account choose a machine at random from the lab status page from the lab1- set of machines (that is, lab1-1.eng.utah.edu through lab1-40.eng.utah.edu).

ssh lab1-10.eng.utah.edu

Run bash

CADE user accounts have tcsh set as their default shell. Each time you login first run bash before anything else. All instructions, examples, and scripts from this class assume you are using bash as your shell. People who use tcsh instead of bash are bad and should feel bad.

You'll need to do this each time unless you reset your default shell (which I'd recommend). Perhaps, savvy users can provide slick setups for your PS1 and more on Canvas.

Install go in your home directory

go is the language we'll be using in our labs. The compiler and toolchain comes in a pre-compiled tarball. It just needs to be downloaded and untared and it's pretty much ready to go. Since CADE machines all share home directories, installing it once in your home will work across all the lab machines.

$ wget https://storage.googleapis.com/golang/go1.6.3.linux-amd64.tar.gz
$ tar zxvf go1.6.3.linux-amd64.tar.gz
$ echo 'export GOROOT=$HOME/go' >> ~/.bashrc
$ echo 'export PATH=$PATH:$GOROOT/bin' >> ~/.bashrc
$ source ~/.bashrc

You may also like to read more on how to install go wherever you like.

Test go

Use your favorite editor (emacs, vim, etc.) to create test.go with the following contents:


package main

import "fmt"

func main() {
  fmt.Printf("hello, world\n")
}

Then run

$ go run test.go
hello, world

If you see "hello, world", then you're ready to get setup with the lab code.

Gitlab

Our class has an instance of gitlab running on a VM that will be the homebase for all lab and project development. It will be where you get the lab skeletons from, it will be where updates to the skeleton will be posted, and it will host development and lab solutions.

Create an account

Next create an account at gitlab. Use your UnID (i.e. u0982239), your real name with proper casing (i.e. Ryan Stutsman), your CS or umail address, and a safe password.

Request Access to the Gitlab group

Next send an email exactly as follows to stutsman@cs.utah.edu, except with your information filled in just as you created the account with.

Subject: CS6963 Account Info

u0982239
Ryan Stutsman
stutsman@cs.utah.edu

I will reply back to let you know when you've been added to the cs6963 group, and at that point you can continue with the next steps.

Fork the labs repo into a private Gitlab repo

Once logged in, go to the main labs repo and click the 'Fork' button. It will ask you to choose a user or a group. Choose your own username (u0xxxxxx). This will create a private copy of the repo under your username. Make sure you keep the 'labs' name for the repo - this will be important for grading scripts.

Do not make the code you write for this class public in any form, except in the ways explicitly communicated in class. This means your forked repo must be 'private' and you must not give additional users access to your repo (with two exceptions below). I've tried to make it hard to do on accident, but I want to be clear. Publishing your code publicly will be considered cheating.

Grading scripts will automatically be run at the project deadlines that will grab the contents of the branch with the correct name. For example, at the deadline for lab1, my scripts will fetch the lab1 branch from the private repo of each student in the class. lab2, lab3, and lab4 will all be similar.

You should check your lab submission status. This page is updated hourly and indicates which of your commits the grading script has found and will use for grading.

Cloning your private repo

Finally, to start development you'll have to do some further setup on the CADE machines. Shell back into one of the CADE machines.

ssh lab1-10.eng.utah.edu

Set up a public/private key pair

If you've already got a public/private key pair you use for SSH that you'd like to keep using you can skip this step. If that doesn't mean much to you then you'll need to take a few more steps.

Run the following filling in your email address instead of mine:

$ ssh-keygen -t rsa -C "stutsman@cs.utah.edu"

This should create two files: ~/.ssh/id_rsa, which you should never share with anyone and ~/.ssh/id_rsa.pub, which you can add to gitlab to give you access to your repos.

cat ~/.ssh/id_rsa.pub

Copy the contents of that file. In gitlab go to your keys management page and click 'Add SSH Key', give the key a name, and paste the contents you copied into the Key box. Once you've done this git on CADE should able to access your repos on gitlab.

Setup git

Tell git about the email address you used on gitlab and your name - of course, replace with your info as appropriate.

$ git config --global user.email stutsman@cs.utah.edu
$ git config --global user.name 'Ryan Stutsman'

Now you'll want to 'clone' your private repo from gitlab (make sure to sub in your UnID here for mine).

$ mkdir ~/cs6963
$ cd cs6963
$ git clone git@cs6963.utah.systems:u9802239/labs.git

Cloning into 'labs'...
remote: Counting objects: 397, done.
remote: Compressing objects: 100% (148/148), done.
remote: Total 397 (delta 247), reused 397 (delta 247)
Receiving objects: 100% (397/397), 1.48 MiB | 634.00 KiB/s, done.
Resolving deltas: 100% (247/247), done.
Checking connectivity... done.

$ ls
labs

If that all works you should now have a new directory called labs with all of the labs' skeleton code inside.

Workflow

Version Control

So far, in we've seen three git repos of code. The first two were on gitlab with a third in our home directory on CADE. For most students this is probably starting to seem complicated so let's take a step back and think about what they are each for and how each can/should be used.

The first, is the cs6963/labs repo on gitlab. This repo contains the basis for all the projects. You'll never have to modify this repo or 'push' anything to it - it just supplies the basis for the class. There may be times when this gets updated to deal with bugs in the lab code; in these cases instructions will be sent out about how to merge those changed into your lab work with as little pain as possible.

The second repo was the private gitlab repo you created under your own user. This serves a few purposes. It's a place where you can permanently store or 'push' code that you'd like to make sure stays safe. It also provides a place where you can push your code changes for me to pick up for grading.

Finally, the third repo you created was a clone of the second into your home directory on CADE. This is the repo where you'll do your development and make your commits. Once you're happy with how you've set things up there, or if you want to make sure your data it safely stored off of the CADE machine then you can 'push' the commits you've made to your local CADE repo over to your private gitlab repo.

Remember, my grading scripts will look at your private gitlab repo to find your lab submissions and it will look for a particular branch name for each lab. For example, if my username is u9802239 and I've just finished lab1, I need to make sure that all of the commits that comprise my solution are pushed to the gitlab repo at u0982239/labs under branch name lab1. Don't get confused and leave your commits only stored in your CADE repo, which I won't have access to.

Check the lab submission status page if you're in doubt.

Developing Go

You'll want to walk through the Go Tour after you've got a working installation. The tour is interactive - so it'll give you an easy way to play with the language and get a sense of how it differs from C, C++, Java, etc.

Effective Go is an optional but interesting read that will undoubtedly improve your Go code - it details nearly all of the more nuanced aspects of the language.

It may be helpful to read How to Write Go Code, which outlines the details of the go tool, though we'll only use a subset of its capabilities for the labs. Go has a stylized way to run, compile, and test code - this document gives the full details of how it works.