BART

Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

BART is a denoising autoencoder / sequence-to-sequence Transformer pretraining model family for natural language generation, translation, and comprehension. It corrupts text with a noising function and learns to reconstruct the original text, combining a bidirectional encoder with a left-to-right decoder.