DeepMind's RETRO Transformer Model скачать в хорошем качестве

DeepMind's RETRO Transformer Model

Retrieval-Enhanced Language Model cross-attends trillions of tokens for SoTA on Wikitext103 and The Pile with 25x fewer parameters. Notes & Links: https://vaclavkosar.com/ml/DeepMinds-... Retrieval-Enhanced Transformer (RETRO) is autoregressive language model from DeepMind’s Improving Language Models by Retrieving from Trillions of Tokens (2021), Deep Mind Blog conditions on retrieved 2-chunks retrieves based on Bert-similarity with preceding chunk SoTA on Wikitext103 and the Pile Competitive on QA same perf GPT-3 with 25x less params model performs even when low train-test overlap retrieval reduces hallucinations and increases interpretability Newsletter: https://vaclavkosar.com/ Twitter:   / vackosar   LinkedIn:   / vaclav-kosar   Facebook:   / vackosar