task_categories:fill-mask size_categories:1B<n<10B language:he arxiv:2304.11077 region:us
---
task_categories:
- fill-mask
language:
- he
size_categories:
- 1B<n<10B
---

### Dataset Summary

A Hebrew Deduplicated and Cleaned Common Crawl Corpus.  A thoroughly cleaned and
approximately deduplicated dataset for unsupervised learning.

### Citing

If you use HeDC4 in your research, please cite [HeRo: RoBERTa and Longformer Hebrew Language Models](http://arxiv.org/abs/2304.11077).
```
@article{shalumov2023hero,
      title={HeRo: RoBERTa and Longformer Hebrew Language Models}, 
      author={Vitaly Shalumov and Harel Haskey},
      year={2023},
      journal={arXiv:2304.11077},
}
```