HN submission. Found in a submission link or body. Submitters do not count as commenters in the main ranking; a submission is not an endorsement.
Abstract
This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec.
arXiv comments
37th Conference on Neural Information Processing Systems (NeurIPS 2023)
Happy Friday, Really happy to share that the code and model for RETVec our new SOTA robust text tokenizer for classification is available on Github: https://github.com/google-research/retvec/ and the NeurIPS paper on Arxiv: https://arxiv.org/abs/2302.09207 Beside its compactness and robustness one of the RETVec strong point is that it greatly simplify the creation of on-device models: RETVec work natively on TFlite and can be use in web deployement via a TFJS. We hope you will find it useful…
Also discussed alongside
Connected through shared HN threads, not inferred similarity or a recommendation.