Radford Alec ; Kim Jong Wook ; Xu Tao ; Brockman Greg ; McLeavey Christine ; Sutskever Ilya

Abstract

We study the capabilities of speech processing systems trained simply topredict large amounts of transcripts of audio on the internet. When scaled to680,000 hours of multilingual and multitask supervision, the resulting modelsgeneralize well to standard benchmarks and are often competitive with priorfully supervised results but in a zero-shot transfer setting without the needfor any fine-tuning. When compared to humans, the models approach theiraccuracy and robustness. We are releasing models and inference code to serve asa foundation for further work on robust speech processing.