Fall 2019: CS 7960 Special Topics: Neuromorphic Architectures


General Information:


Course Description:

The course will cover hardware approaches for implementing neural-inspired algorithms. In recent years, machine learning and AI have re-emerged as effective solutions to a number of difficult and economically relevant problems. They will likely enable autonomous vehicles, healthcare solutions, assistive technologies, etc. These solutions will be deployed in datacenters, mobile phones, self-driving cars, and sensors. The course will start with a brief primer on why machine learning has made significant strides in the past decade. We will then move to discussing specialized processors (accelerators) that can efficiently execute a large family of machine learning algorithms (for both inference and training). We will focus our discussions on accelerators for artificial/spiking neural networks, and convolutional neural networks -- areas that have dominated recent architecture conferences. We will end the course by discussing how the learned concepts can apply to other relevant application domains, e.g., genomic analysis.

The course does not have any formal pre-requisites, but is intended primarily for graduate students with some familiarity in architecture and/or machine learning. The lectures will be self-contained, i.e., I will provide sufficient background in architecture and machine learning to make the material accessible. Most class lectures will be based on recent research papers (see tentative schedule below). Students will also work in groups on semester-long projects -- the projects will compare the implementations of various cognitive tasks with different algorithms and hardware approaches.


University Support:

College of Engineering Policies (Disability, Add, Drop, Appeals, Safety, etc.): Guidelines from the college.

School of Computing Policies

Class rosters are provided to the instructor with the student's legal name as well as "Preferred first name" (if previously entered by you in the Student Profile section of your CIS account). While CIS refers to this as merely a preference, I will honor you by referring to you with the name and pronoun that feels best for you in class, on papers, exams, group projects, etc. Please advise me of any name or pronoun changes (and please update CIS) so I can help create a learning environment in which you, your name, and your pronoun will be respected.

The University of Utah values the safety of all campus community members. To report suspicious activity or to request a courtesy escort, call campus police at 801-585-COPS (801-585-2677). You will receive important emergency alerts and safety messages regarding campus safety via text message. For more information regarding safety and to view available training resources, including helpful videos, visit SAFEU.


Grading:

The following is a tentative guideline and may undergo changes. The class project accounts for 50% of the final grade. 40% will be based on two take-home exams. 10% will be based on class participation and class presentations.


Tentative Class Schedule

Dates Lecture Topic Slides Annotated Slides
Tue Aug 20 Overview, landscape, history of neural-based hardware ppt pdf ppt pdf
Thu Aug 22 Intro to Deep Learning Algorithms ppt pdf ppt pdf
Tue Aug 27 Custom SIMD Architectures: DianNao ppt pdf ppt pdf
Thu Aug 29 The DaDianNao Architecture ppt pdf ppt pdf
Tue Sep 3 Deep Compression ppt pdf ppt pdf
Thu Sep 5 Deep Compression Architectures ppt pdf ppt pdf
Tue Sep 10 Systolic architectures: Eyeriss ppt pdf ppt pdf
Thu Sep 12 Commercial architectures: Google TPU, Tesla FSD ppt pdf ppt pdf
Tue Sep 17 Commercial architectures: NVIDIA Volta, Graphcore, Intel NNP. Training intro. ppt pdf ppt pdf
Thu Sep 19 Training Innovations: vDNN, ScaleDeep. ppt pdf ppt pdf
Tue Sep 24 No Class    
Thu Sep 26 Analog Accelerators: ISAAC ppt pdf ppt pdf YouTube videos: Part 1 and Part 2
Tue Oct 1 Spiking Neuron Intro ppt pdf ppt pdf
Thu Oct 3 TrueNorth, Projects discussion ppt pdf ppt pdf
Tue/Thu Oct 8/10 Fall Break, Take-Home Midterm Exam    
Tue Oct 15 Comparing SNNs and ANNs ppt pdf ppt pdf
Thu Oct 17 Self Driving Car Pipeline ppt pdf ppt pdf
Tue Oct 22 Exploiting Variable Precision ppt pdf ppt pdf
Thu Oct 24 Simba ppt pdf  
Tue Oct 29 Projects Discussion Notes  
Thu Oct 31 Ineffectuals ppt pdf  
Tue Nov 5 In-Memory Processing and Projects Discussion ppt pdf  
Thu Nov 7 Gradient Overheads, 3D CNN Architectures ppt pdf  
Tue Nov 12 Sequence Alignment Basics ppt pdf  
Thu Nov 14 Molecular Dynamics Accelerators ppt pdf  
Tue Nov 19 Sequence Alignment Accelerators ppt pdf  
Thu Nov 21 Systolic Arrays I -- sort, matrix mult ppt pdf  
Tue Nov 26 Systolic Arrays II -- add, mult, eqn solver, graphs ppt pdf  
Thu Nov 28 Thanksgiving    
Tue Dec 3 Project presentations    
Thu Dec 5 Project presentations    
due Dec 15 Take-Home Final Exam