A Dynamic GCN with Cross-Representation Distillation for Event-Based Learning
DOI:
https://doi.org/10.1609/aaai.v38i2.27914Keywords:
CV: Vision for Robotics & Autonomous Driving, CV: Video Understanding & Activity Analysis, CV: Representation Learning for Vision, CV: 3D Computer VisionAbstract
Recent advances in event-based research prioritize sparsity and temporal precision. Approaches learning sparse point-based representations through graph CNNs (GCN) become more popular. Yet, these graph techniques hold lower performance than their frame-based counterpart due to two issues: (i) Biased graph structures that don't properly incorporate varied attributes (such as semantics, and spatial and temporal signals) for each vertex, resulting in inaccurate graph representations. (ii) A shortage of robust pretrained models. Here we solve the first problem by proposing a new event-based GCN (EDGCN), with a dynamic aggregation module to integrate all attributes of vertices adaptively. To address the second problem, we introduce a novel learning framework called cross-representation distillation (CRD), which leverages the dense representation of events as a cross-representation auxiliary to provide additional supervision and prior knowledge for the event graph. This frame-to-graph distillation allows us to benefit from the large-scale priors provided by CNNs while still retaining the advantages of graph-based models. Extensive experiments show our model and learning framework are effective and generalize well across multiple vision tasks.
Downloads
Published
2024-03-24
How to Cite
Deng, Y., Chen, H., & Li, Y. (2024). A Dynamic GCN with Cross-Representation Distillation for Event-Based Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(2), 1492-1500. https://doi.org/10.1609/aaai.v38i2.27914
Issue
Section
AAAI Technical Track on Computer Vision I
