This paper studies how to keep a vision backbone effective while removing token mixers in
its basic building blocks. Token mixers, as self-attention for vision transformers (ViTs), are …

Kullback-Leiber divergence has been widely used in Knowledge Distillation (KD) to compress
Large Language Models (LLMs). Contrary to prior assertions that reverse Kullback-Leibler …

Recently, the success of pre-training in text domain has been fully extended to vision, audio,
and cross-modal scenarios. The proposed pre-training models of different modalities are …

This work examines whether decoder-only Transformers such as LLaMA, which were originally
designed for large language models (LLMs), can be adapted to the computer vision field. …

Zero-shot entity retrieval, aiming to link mentions to candidate entities under the zero-shot
setting, is vital for many tasks in Natural Language Processing. Most existing methods …

Aspect-based Sentiment Analysis (ABSA) is a sentiment analysis task at fine-grained level.
Recently, generative frameworks have attracted increasing attention in ABSA due to their …

Distilling high-accuracy Graph Neural Networks~(GNNs) to low-latency multilayer
perceptrons~(MLPs) on graph tasks has become a hot research topic. However, MLPs rely …

Knowledge Distillation (KD) is a predominant approach for BERT compression. Previous KD-based
methods focus on designing extra alignment losses for the student model to mimic …

Honesty is a fundamental principle for aligning large language models (LLMs) with human
values, requiring these models to recognize what they know and don't know and be able to …

This paper studies how to keep a vision backbone effective while removing token mixers in
its basic building blocks. Token mixers, as self-attention for vision transformers (ViTs), are …