While retrieval has become a de-facto method of augmenting traditional, text-only language models, significantly less work has been done on multi-modal retrieval and retrieval with very long-context models. This talk will cover why retrieval is needed from first principles, state-of-the-art uses of retrieval apart from their traditional settings, specifically in multi-modal settings, and our long-term hypothesis of where retrieval as a technique will evolve too as context lengths continue to increase.