CS6450 Distributed Systems

CS6450 Fall 2018 Distributed Systems

Tuesdays & Thursdays
15:40 to 17:00 WEB L122

Instructor Ryan Stutsman stutsman@cs.utah.edu
Office Hours Tuesday 14:00 to 15:30 in MEB 3436

TA Zirak Zaheer
Office Hours Monday 11:00 to 12:30 in MEB 3161

TA Mazhar Naqvi
Office Hours Wednesday 10:30 to 12:00 in MEB 3445

Course Calendar
Lab Setup
Lab 1
Lab 2
Lab 3
Final Project
Lab Submission Status
Gitlab
Discussions
Grades

Course Overview

3 credit hours

Prerequisites: CS 5460, CS 4480, or similar systems experience with a heavy programming component.

In the last decade, large-scale distributed systems have revolutionized the way we live and work. They form the basis for our communications platforms. They provide our only means of finding, aggregating, storing, and analyzing the massive data we collect each day. These systems will only need to grow larger, faster, and more reliable as more and more of our lives and devices are online.

Building distributed systems, especially those that operate at large-scale with high performance, presents special challenges. For example, scale introduces faults and the need for redundancy. Redundancy complicates consistency.

This class introduces many of the key aspects of designing and building distributed systems such RPC, naming, routing, replication, consistency, fault tolerance, transactions, and time.

The first half of the class class is driven in large part by lectures often extracted from real systems described in papers from the systems research community. Students are expected to attend lectures and take short comprehension quizzes. Later lectures will fold in current research topics like kernel-bypass networking, RDMA, and highly concurrent storage and transactional systems.

However, the bulk of the work of the class involves a progressive series of labs where students develop and debug a fault-tolerance consensus-replicated key-value store. Students will also complete a final project based on a small set of (about 4) academic papers.

Objectives

  • To get an overview of practical modern distributed systems, with an eye toward data center systems in particular.
  • To learn, use, and implement standard distributed systems techniques and infrastructure (RPC, consensus).
  • To learn how to approach, discuss, and communicate about difficult and technical subject matter.

Topic List/Syllabus

The calendar includes a preliminary list of topics (at the bottom) that will be covered in class.

Structure

CS6450 is a special topics class that straddles upper-level undergrad, early grad level with lectures and several programming labs, and a final reading/project.

Lectures

Most class meetings will be lectures, some will be part paper discussion. You must read all required readings before coming to class and be prepared to discuss them. You can find out what paper to read for each meeting on the calendar. Required readings are marked "REQUIRED"; be prepared to answer quiz questions for REQUIRED readings at the start of that day's class BEFORE the lecture is presented.

See the misconduct policy below for details on collaboration.

Programming Labs

3 labs with several sub-steps will be due every few weeks throughout the course. See the misconduct policy below for details on collaboration.

Quizzes

Very short quizzes based on (typically immediately) prior material or the assigned required reading will be given to complete before class or in class at the start of the lecture period. When a particular day on the calendar has a reading marked REQUIRED the pre-lecture quiz may cover content and objectives from the assigned reading.

The two lowest (percentage) scores will be discarded from the final grade. Each quiz will have equal weight, totaling to 33% of the final grade weight. The two scores that are discarded are, in part, intended cover absences, so plan accordingly -- there will be no make up quizzes.

Quizzes are closed-book, closed-notes if they are given in class; quizzes that are to be completed before lecture are open-book, open-note, and the usual collaboration policy applies.

Final Project/Reading Thread

In lieu of a final or exams, students will work in groups of 2 students to read a an academic paper, summarizing their understanding of the paper, drawing their own conclusions, and summarizing what directions the sum of the ideas suggests today. Final reports covering the readings and the project (about 3 pages) are required along with an in-class presentation. The intent is to help acclimatize students to reading about distributed systems, extracting ideas from the literature, and applying the ideas in new ways.

Both students from each group are expected to read the chosen paper, to contribute to the discussion about them, and to contribute to the final report and presentation.

Grading

Final course grades will be based on:

  • 33% for programming assignments.
    • About equal weight for each of labs 1, 2a, 2b, 2c, and 3a.
  • 33% for quizzes.
    • About 12 total quizzes with the lowest two scores thrown out.
  • 34% for final project.

Late Submissions

You can hand in your lab solutions late with a few conditions:

  • You can only use late days for labs.
  • You have 3 late days to use at your discretion subject to the other constraints.
  • You can only use late days in whole integer units.
  • You must notify the TA that you are turning it in late and how many late days you'd like to use. You can do this up to 3 days beyond the due date.
  • You must turn in all work before the last day of normal class or it will be an E.You must turn in all work before Dec 15 or it will be an E.
  • If you fail to meet these requirements and a lab is late, we'll grade it and scale the score by 0.9 if it is up to one day late, by 0.8 if it is up to two days late, by 0.7 if it is up to three days late. by 0.6 if it is any later.

If you want an exception to these rules, in any way you must get a letter from the appropriate office at the University.

Appeals

Students who wish to appeal a grade must do so within one week of receiving the grade.

Textbook

The course has no official textbook, but there are several texts that can be of help:

  • Distributed Systems, van Steen and Tanenbaum (Suggested Optional Readings will most frequently be drawn from this book).
  • Guide to Reliable Distributed Systems, Birman.
  • Distributed Systems: Concepts and Design, Coulouris et al.

Behavior in Class

Students are expected to maintain professional behavior in class according to the University of Utah Student Code, which is available here. Students should read the Code carefully and know what their responsibilities are. According to Faculty Rules and Regulations, it is the faculty responsibility to enforce responsible classroom behavior, beginning with verbal warnings and progressing to dismissal from class and a failing grade. Students have the right to appeal such action to the Student Behavior Committee.

College of Engineering Guidelines

Information on withdrawing from courses, appealing grades, and more, see the College of Engineering Academic Affairs website.

School of Computing Guidelines

This class is also subject to the School of Computing Guidelines and Policies.

Students with disabilities

The University of Utah seeks to provide equal access to its programs, services and activities for people with disabilities. If you will need accommodations in the class, reasonable prior notice needs to be given to the Center for Disability Services, 162 Union Building, 581-5020 (V/TDD). CDS will work with you and the instructor to make arrangements for accommodations.

Collaboration and Cheating

Collaboration and Cheating

  • Everyone needs to read the SoC Policy on Academic Misconduct.
  • Working with others on assignments is a good way to learn the material and we encourage it. However, there are limits to the degree of cooperation that we will permit.
  • When working on programming assignments, you must work only with others whose understanding of the material is approximately equal to yours. In this situation, working together to find a good approach for solving a programming problem is cooperation; listening while someone dictates a solution is cheating. You must limit collaboration to a high-level discussion of solution strategies, and stop short of actually writing down a group answer. Anything that you hand in, whether it is a written problem or a computer program, must be written in your own words. If you base your solution on any other written solution, you are cheating.
  • Never look at another student's code or share your code with any other student.
  • You must not make your code public (on github or by any other means).
  • We do not distinguish between cheaters who copy other's work and cheaters who allow their work to be copied. If you cheat, you will be given an E in the course and referred to the University Student Behavior Committee. If you have any questions about what constitutes cheating, please ask.
  • Clearly, any attempt to subvert the ordinary grading process constitutes cheating.

Acknowledgements

The entire structure and much of the content was blatantly lifted from MIT's 6.824. Special thanks to Robert Morris, Franz Kaashoek, and Nickolai Zeldovich at MIT and Mike Freedman and Kyle Jamieson at Princeton for all of the incredible content on which this course is built.