task_categories:summarization language:tl language:en region:us
---
task_categories:
- summarization
language:
- tl
- en
---
# 📝 BUOD Article Scraper

Authors: [James Esguerra](https://huggingface.co/jamesesguerra), [Julia Avila](), [Hazielle Bugayong](https://huggingface.co/0xhaz)
- Article Scraper for the KAMI-3000 dataset used in the BUOD [distilBART](https://huggingface.co/ateneoscsl/BUOD_distilBART_TM) and [bert2bert](https://huggingface.co/ateneoscsl/BUOD_bert2bert_TM) Transformer Models. This was also used for the text summarization tasks in the Filipino Language.

### Setup
1. Clone the repository.
```sh
# https
git clone https://github.com/avila-bugayong-esguerra/article-scraper.git

# or 

# ssh
git clone git@github.com:avila-bugayong-esguerra/article-scraper.git
```

2. Change directory into project folder.
```sh
cd article_scraper
```

3. Create a virtual environment.
```sh
python -m venv venv
```

4. Activate the virtual environment.
```sh
# windows
\venv\Scripts\activate

# unix
source venv/bin/activate
```

5. Install the dependencies.
```sh
pip install -r article_scraper/requirements.txt
```

6. Change directory into the Scrapy project.
```sh
cd article_scraper
```