arXiv:1810.04826eess.AS
VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking
Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.
Abstract
In this paper, we present a novel system that separates the voice of a target speaker from multi-speaker signals, by making use of a reference signal from the target speaker. We achieve this by training two separate neural networks: (1) A speaker recognition network that produces speaker-discriminative embeddings; (2) A spectrogram masking network that takes both noisy spectrogram and speaker embedding as input, and produces a mask. Our system significantly reduces the speech recognition WER on multi-speaker signals, with minimal WER degradation on single-speaker signals.
Now imagine letting lose SfM and tracking algorithms on this video dataset and reconstructing full temporal 3D model of the event ala Enemy of the State (1998). Add in some of [1] and we could reconstruct full Cyberpunk 2077 Braindance. [1] https://arxiv.org/pdf/1810.04826.pdf…
raszView on HNParent context