HackerPapers

arXiv:2311.05609cs.SD

What Do I Hear? Generating Sounds for Visuals with ChatGPT

David Chuan-En Lin, Nikolas Martelaro

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

This short paper introduces a workflow for generating realistic soundscapes for visual media. In contrast to prior work, which primarily focus on matching sounds for on-screen visuals, our approach extends to suggesting sounds that may not be immediately visible but are essential to crafting a convincing and immersive auditory environment. Our key insight is leveraging the reasoning capabilities of language models, such as ChatGPT. In this paper, we describe our workflow, which includes creating a scene context, brainstorming sounds, and generating the sounds.

arXiv comments
Demo: http://soundify.cc
Journal reference
DOI
Subjects
cs.SDSound cs.CVComputer Vision and Pattern Recognition cs.MMMultimedia eess.ASAudio and Speech Processing

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Nov 2023 to Nov 2023.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. [dead] 1 comment

    • What Do I Hear? Papers: - Generating Sounds for Visuals with ChatGPT: https://arxiv.org/pdf/2311.05609.pdf - Soundify: Matching Sound Effects to Video: https://arxiv.org/pdf/2112.09726.pdf

      helperfunctionView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.