Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Simple item page
dc.contributor.authorRaffel, Colin
dc.contributor.authorShazeer, Noam
dc.contributor.authorRoberts, Adam
dc.contributor.authorLee, Katherine
dc.contributor.authorNarang, Sharan
dc.contributor.authorMatena, Michae
dc.contributor.authorZhou, Yanqi
dc.contributor.authorLi, Wei
dc.contributor.authorLiu, Peter J.
dc.date.accessioned2025-06-02T13:10:32Z
dc.date.available2025-06-02T13:07:37Z
dc.date.available2025-06-02T13:10:32Z
dc.date.issued2019-10-19
dc.description이 논문은 모든 NLP 태스크를 text-to-text 문제로 재구성하는 T5 프레임워크를 제안하며, 전이학습의 효과를 대규모 데이터셋에서 실험적으로 검증합니다. ©2019 Google Research
dc.description.abstractTransfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts all text-based language problems into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled data sets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new ``Colossal Clean Crawled Corpus'', we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our data set, pre-trained models, and code.
dc.description.sponsorshipGoogle Research
dc.identifier.urihttps://arxiv.org/abs/1910.10683
dc.identifier.urihttp://data.inu.ac.kr/handle/123456789/1954.2
dc.language.isoen_US
dc.publisherarXiv
dc.subjectT5
dc.subjectTransfer Learning
dc.subjectUnified Framework
dc.subjectNLP
dc.subjectText-to-Text
dc.titleExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
dc.typeArticle

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
1910.10683v4.pdf
Size:
1.13 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
97 B
Format:
Item-specific license agreed to upon submission
Description:
Download

Collections

Natural Language Processing

Version History

Now showing 1 - 2 of 2
VersionDateSummary
2*
 2025-06-02 22:10:16
add subject
1
 2025-06-02 22:07:37
* Selected version