Meta AI releases “data2vec”, a self-supervised algorithm that works for speech, vision, and text
They have applied it separately to speech, text and images where it outperformed the previous best single-purpose algorithms for computer vision and speech.
BY SREEJANI BHATTACHARYYA
Meta AI has released data2vec, calling it “the first high-performance self-supervised algorithm that works for multiple modalities.” They have applied it separately to speech, text and images, where it outperformed the previous best single-purpose algorithms for computer vision and speech. It came out as competitive on NLP tasks.
Meta added that data2vec does not rely on contrastive learning or reconstructing the input example. The tech giant has also released open source code and pretrained models.
