Fall 2019: CS 7960 Special Topics: Neuromorphic Architectures


General Information:


Course Description:

The course will cover hardware approaches for implementing neural-inspired algorithms. In recent years, machine learning and AI have re-emerged as effective solutions to a number of difficult and economically relevant problems. They will likely enable autonomous vehicles, healthcare solutions, assistive technologies, etc. These solutions will be deployed in datacenters, mobile phones, self-driving cars, and sensors. The course will start with a brief primer on why machine learning has made significant strides in the past decade. We will then move to discussing specialized processors (accelerators) that can efficiently execute a large family of machine learning algorithms (for both inference and training). We will focus our discussions on accelerators for artificial/spiking neural networks, and convolutional neural networks -- areas that have dominated recent architecture conferences. We will end the course by discussing how the learned concepts can apply to other relevant application domains, e.g., genomic analysis.

The course does not have any formal pre-requisites, but is intended primarily for graduate students with some familiarity in architecture and/or machine learning. The lectures will be self-contained, i.e., I will provide sufficient background in architecture and machine learning to make the material accessible. Most class lectures will be based on recent research papers (see tentative schedule below). Students will also work in groups on semester-long projects -- the projects will compare the implementations of various cognitive tasks with different algorithms and hardware approaches.


University Support:

College of Engineering Policies (Disability, Add, Drop, Appeals, Safety, etc.): Guidelines from the college.

School of Computing Policies

Class rosters are provided to the instructor with the student's legal name as well as "Preferred first name" (if previously entered by you in the Student Profile section of your CIS account). While CIS refers to this as merely a preference, I will honor you by referring to you with the name and pronoun that feels best for you in class, on papers, exams, group projects, etc. Please advise me of any name or pronoun changes (and please update CIS) so I can help create a learning environment in which you, your name, and your pronoun will be respected.

The University of Utah values the safety of all campus community members. To report suspicious activity or to request a courtesy escort, call campus police at 801-585-COPS (801-585-2677). You will receive important emergency alerts and safety messages regarding campus safety via text message. For more information regarding safety and to view available training resources, including helpful videos, visit SAFEU.


Grading:

The following is a tentative guideline and may undergo changes. The class project accounts for 50% of the final grade. 40% will be based on two take-home exams. 10% will be based on class participation and class presentations.


Tentative Class Schedule

Dates Lecture Topic
Tue Aug 20 Overview, landscape, history of neural-based hardware
Thu Aug 22 Intro to Deep Learning Algorithms
Tue Aug 27 Custom SIMD Architectures: DianNao
Thu Aug 29 The DaDianNao Architecture
Tue Sep 3 Deep Compression
Thu Sep 5 Deep Compression Architectures
Tue Sep 10 Systolic architectures: Eyeriss
Thu Sep 12 Commercial architectures: Google TPU, Tesla FSD
Tue Sep 17 Commercial architectures: NVIDIA Volta, Graphcore
Thu Sep 19 Architectures for Training: intro, vDNN
Tue Sep 24 More Training Innovations: ScaleDeep, HyPar, GIST
Thu Sep 26 Analog Accelerators: ISAAC
Tue Oct 1 Improving Analog Acceleration
Thu Oct 3 Spiking Neuron Intro
Tue/Thu Oct 8/10 Fall Break, Take-Home Midterm Exam
Tue Oct 15 TrueNorth Architecture
Thu Oct 17 Comparing SNNs and ANNs
Tue Oct 22 Self Driving Car Pipeline
Thu Oct 24 Project discussions
Tue Oct 29 Exploiting Variable Precision
Thu Oct 31 Exploiting Variable Precision
Tue Nov 5 In-Memory Processing
Thu Nov 7 Recent Papers
Tue Nov 12 Recent Papers
Thu Nov 14 Systolic Arrays I -- sort, matrix mult
Tue Nov 19 Systolic Arrays II -- add, mult, eqn solver
Thu Nov 21 Sequence Alignment
Tue Nov 26 Molecular Dynamics Accelerators
Thu Nov 28 Thanksgiving
Tue Dec 3 Project presentations
Thu Dec 5 Project presentations
due Dec 15 Take-Home Final Exam