<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jiao-etal-2024-spin">
    <titleInfo>
        <title>SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Difan</namePart>
        <namePart type="family">Jiao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yilun</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhenwei</namePart>
        <namePart type="family">Tang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Daniel</namePart>
        <namePart type="family">Matter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jürgen</namePart>
        <namePart type="family">Pfeffer</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ashton</namePart>
        <namePart type="family">Anderson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Among the many tasks that Large Language Models (LLMs) have revolutionized is text classification. Current text classification paradigms, however, rely solely on the output of the final layer in the LLM, with the rich information contained in internal neurons largely untapped. In this study, we present SPIN: a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of LLMs for text classification. Specifically, SPIN sparsifies internal neurons by linear probing-based salient neuron selection layer by layer, avoiding noise from unrelated neurons and ensuring efficiency. The cross-layer salient neurons are then integrated to serve as multi-layered features for the classification head. Extensive experimental results show our proposed SPIN significantly improves text classification accuracy, efficiency, and interpretability.</abstract>
    <identifier type="citekey">jiao-etal-2024-spin</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-acl.277</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.277</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>4666</start>
            <end>4682</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification
%A Jiao, Difan
%A Liu, Yilun
%A Tang, Zhenwei
%A Matter, Daniel
%A Pfeffer, Jürgen
%A Anderson, Ashton
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics: ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F jiao-etal-2024-spin
%X Among the many tasks that Large Language Models (LLMs) have revolutionized is text classification. Current text classification paradigms, however, rely solely on the output of the final layer in the LLM, with the rich information contained in internal neurons largely untapped. In this study, we present SPIN: a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of LLMs for text classification. Specifically, SPIN sparsifies internal neurons by linear probing-based salient neuron selection layer by layer, avoiding noise from unrelated neurons and ensuring efficiency. The cross-layer salient neurons are then integrated to serve as multi-layered features for the classification head. Extensive experimental results show our proposed SPIN significantly improves text classification accuracy, efficiency, and interpretability.
%R 10.18653/v1/2024.findings-acl.277
%U https://aclanthology.org/2024.findings-acl.277
%U https://doi.org/10.18653/v1/2024.findings-acl.277
%P 4666-4682

Markdown (Informal)

[SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification](https://aclanthology.org/2024.findings-acl.277) (Jiao et al., Findings 2024)

• SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification (Jiao et al., Findings 2024)

ACL

• Difan Jiao, Yilun Liu, Zhenwei Tang, Daniel Matter, Jürgen Pfeffer, and Ashton Anderson. 2024. SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4666–4682, Bangkok, Thailand. Association for Computational Linguistics.