Loading Events

« All Events

  • This event has passed.

IEEE CIR: Understanding Neural Collapse in Deep Learning

June 9 @ 5:30 PM - 7:00 PM MDT

In the past decade, the revival of deep neural networks has led to dramatic success in numerous applications ranging from computer vision, to natural language processing, to scientific discovery and beyond. Nevertheless, the practice of deep networks has been shrouded with mystery as our theoretical understanding for the success of deep learning remains elusive. In this talk, we will focus on the representations learned by deep neural networks. For example, Neural collapse is an intriguing empirical phenomenon that persists across different neural network architectures and a variety of standard datasets. This phenomenon implies that (i) the class means and the last-layer classifiers all collapse to the vertices of a Simplex Equiangular Tight Frame (ETF) up to scaling, and (ii) cross-example within-class variability of last-layer activations collapses to zero. We will provide a geometric analysis for understanding why this happens on a simplified unconstrained feature model. We will also exploit these findings to improve training efficiency: we can set the feature dimension equal to the number of classes and fix the last-layer classifier to be a Simplex ETF for network training, reducing memory cost by over 20% on ResNet18 without sacrificing the generalization performance. Co-sponsored by: Christopher Reardon, Eric Ericson Speaker(s): Dr. Zhihui Zhu , 2155 East Wesley Avenue, Denver, Colorado, United States, 80208, Virtual: https://events.vtools.ieee.org/m/295539

Sep 26

2022 IEEE IAS Petroleum and Chemical Industry Technical Conference (PCIC)

September 26 @ 8:00 AM - September 29 @ 5:00 PM MDT
Apr 19

IEEE Green Technologies Conference 2023

April 19, 2023 @ 9:00 AM - April 21, 2023 @ 5:00 PM MDT
Apr 21

2023 Region 5 Annual Meeting, Student Competitions, and Awards Banquet

April 21, 2023 @ 5:00 PM - April 23, 2023 @ 5:00 PM MDT
Apr 21

Region 5 Student Competitions and Annual Meeting 2023

April 21, 2023 @ 7:00 PM - April 23, 2023 @ 1:00 PM MDT