Reading list for research topics in Sound AI

Audio Captioning datasets for PyTorch.

Using pretrained encoder and language models to generate captions from multimedia inputs.

Python code for handling the Clotho dataset.

Source code for "MusCaps: Generating Captions for Music Audio" (IJCNN 2021)

Song Describer is a data collection platform for annotating music with textual descriptions.

Code base for WaveTransformer: A novel architecture for automated audio captioning

Audio captioning baseline system for DCASE 2020 challenge.

Metrics for evaluating Automated Audio Captioning systems, designed for PyTorch.

Tracking states of the arts and recent results (bibliography) on sound tasks.

PyTorch implementation of the ICASSP-24 paper: "Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation"

Official Implementation of "Prefix tuning for Automated Audio Captioning(ICASSP 2023)"

2nd place solution for 2020 DCASE challenge task 6 audio captioning. http://dcase.community/challenge2020/task-automatic-audio-captioning-results#wuyusong2020_t6

Fluency ENhanced Sentence-bert Evaluation (FENSE), metric for audio caption evaluation. And Benchmark dataset AudioCaps-Eval, Clotho-Eval.

Tools for the evaluation of audio captioning.

[NeurIPS 2023 - ML for Audio Workshop (Oral)] Zero-shot audio captioning with audio-language model guidance and audio context keywords

CoNeTTE: An efficient Audio Captioning system leveraging multiple datasets with Task Embedding

Code for ICASSP 2024 Paper: RECAP: Retrieval-Augmented Audio Captioning

Workshop on Detection and Classification of Acoustic Scenes and Events

text-only training or language-free training for multimodal tasks (image/audio/video caption, retrieval, text2image)