DistilLED Large CNN 16384
distil-led-large-cnn-16384 was initialized from sshleifer/distilbart-cnn-12-6, in a fashion similar to allenai/led-large-16384.
To be able to process 16K tokens, sshleifer/distilbart-cnn-12-6's position embedding matrix was simply copied 16 times.
This checkpoint should be loaded into LEDForConditionalGeneration.from_pretrained. See the LED documentation for more information.
- Downloads last month
- 22
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.