Efficient Attention for Long Documents
A. Researcher and B. Coauthor
3 Results
Sparse attention reduces memory use by 41% on long documents [4], while accuracy stays within 0.3 points of dense attention across every benchmark we tried.
Citation added from DOI. Child et al., 2019. Written to references.bib.