Arts Automobile Bank Crime Diaspora Education Entertainment Health Lifestyle Literature Market Politics Sports Technology Tourism World
---
license: apache-2.0

language: ne

tags:
    - multiclass-classification
    - newsgroup
    - nepali
---


# Nepali 16 News Group Classification

This model is suitable for classifying news categories in Nepali language into 16 different groups. It is fine-tuned on a pretrained DistilBERT model with Sequence Classification head on 16 newsgroup dataset for Nepali language.

## Acknowledgements

### Pretrained DistilBERT model

This model is fine-tuned on a text classification problem using a [pretrained model](https://huggingface.co/Sakonii/distilbert-base-nepali) available at HuggingFace.

## Dataset

This dataset consists of News documents in Nepali language which are equally categorized into 16 different categories. It is primarily designed for the purpose of multiclass text classification tasks. Each category consists of 1000 different news articles scraped from online Nepali news portals namely Ekantipur, Nagarik, Gorkhapatra, Online Khabar and many more. In addition to the article text, it also contains news headings, source from where the news is taken from and brief summary of what news is about. However, summaries are only available for news from certain sources.

The dataset is Label Encoded, i.e. it consists of 'labels' column that denote the numerical representation of news categories.

## Model Fine-tuning
The model fine-tuning was carried out in Google Colab with Tesla T4 GPU environment using HuggingFace's Trainer API. The training took approx. 42 minutes and was trained for 4 epochs until it reached the validation accuracy threshold.

**Dataset Splits**

| Split      | Size | No. of samples |
|------------|------|----------------|
| train      | 0.7  | 11200          |
| validation | 0.15 | 2400           |
| test       | 0.15 | 2400           |

**DistilBERT Tokenizer parameters**
```
padding = True
truncation = True
max_len = 512
```

**Model Trainer arguments (For Trainer API)**
```
epochs = 5
batch_size = 16
learning_rate = 5e-05
save_steps = 500
eval_steps = 500
```
## Training Results

| Step | Training Loss | Validation Loss | Accuracy | Balanced Accuracy | Precision | Recall   | F1       |
|------|---------------|-----------------|----------|-------------------|-----------|----------|----------|
|  500 |      0.718600 |        0.407946 | 0.878750 |          0.878750 |  0.882715 | 0.878750 | 0.877678 |
| 1000 |      0.252300 |        0.372410 | 0.897083 |          0.897083 |  0.903329 | 0.897083 | 0.897369 |
| 1500 |      0.175000 |        0.323519 | 0.916250 |          0.916250 |  0.917955 | 0.916250 | 0.916297 |
| 2000 |      0.099400 |        0.339903 | 0.916667 |          0.916667 |  0.919054 | 0.916667 | 0.916141 |
| 2500 |      0.058900 |        0.354112 | 0.921250 |          0.921250 |  0.922036 | 0.921250 | 0.920899 |
| 3000 |      0.023300 |        0.360163 | 0.922500 |          0.922500 |  0.922767 | 0.922500 | 0.922219 |

**Validation Loss:** 0.3235

**Validation Accuracy:** 92.625%

## Testing Results

| category      | precision | recall | f1-score | support |
|---------------|-----------|--------|----------|---------|
| Arts          | 0.94      | 0.97   | 0.95     | 150     |
| Diaspora      | 0.97      | 0.93   | 0.95     | 150     |
| Bank          | 0.97      | 0.86   | 0.91     | 150     |
| Technology    | 0.98      | 0.99   | 0.99     | 150     |
| Literature    | 0.92      | 0.88   | 0.90     | 150     |
| Automobile    | 0.93      | 0.97   | 0.95     | 150     |
| World         | 0.90      | 0.93   | 0.92     | 150     |
| Market        | 0.93      | 0.98   | 0.95     | 150     |
| Lifestyle     | 0.99      | 0.96   | 0.97     | 150     |
| Sports        | 0.90      | 0.86   | 0.88     | 150     |
| Health        | 0.86      | 0.89   | 0.87     | 150     |
| Entertainment | 0.98      | 0.97   | 0.97     | 150     |
| Politics      | 0.97      | 0.99   | 0.98     | 150     |
| Tourism       | 0.82      | 0.96   | 0.88     | 150     |
| Crime         | 0.97      | 0.96   | 0.97     | 150     |
| Education     | 0.96      | 0.84   | 0.90     | 150     |
|               |           |        |          |         |
| accuracy      |           |        | 0.93     | 2400    |
| macro avg     | 0.94      | 0.93   | 0.93     | 2400    |
| weighted avg  | 0.94      | 0.93   | 0.93     | 2400    |


## Sample Predictions

### Sample Text (Sports)
``` 
काठमाडौँ — त्रिभुवन आर्मी क्लबले ६ स्वर्ण, २ रजत र ६ कांस्य पदक जित्दै प्रथम वीर गणेशमान सिंह राष्ट्रिय फेन्सिङ प्रतियोगितामा टिम च्याम्पियन ट्रफी जितेको छ ।

दोस्रो भएको एपीएफले ३ स्वर्ण, ५ रजत र ८ कांस्य जित्यो । वाग्मती प्रदेशले ३ स्वर्ण, ५ रजत र ३ कांस्य जित्दै तेस्रो स्थान हात पार्‍यो । 
वीर गणेशमान सिंह स्पोर्ट्स कमिटी र नेपाल फेन्सिङ संघको संयुक्त आयोजनामा भएको प्रतियोगिताको महिला फोइलतर्फ एपीएफकी मन्दिरा थापाले स्वर्ण जितिन् । उनले फाइनलमा चिरप्रतिद्वन्द्वी सेनाकी रमा सिंहलाई १५–१२ ले हराइन् । आर्मीकी मनीषा राई र वाग्मतीकी अञ्जु तामाङ तेस्रो भए ।

पुरुषको टिम फोइलतर्फ आर्मीले स्वर्ण जित्यो । आर्मीले वाग्मतीलाई ४५–२९ स्कोरले हरायो । गण्डकी र एपीएफले कांस्य जिते । 
टिम महिला सावरमा आर्मीले स्वर्ण जित्यो । फाइनलमा आर्मीले एपीएफलाई ४५–३६ स्कोरले हराएर स्वर्ण जितेको हो । वाग्मती र गण्डकीले कांस्य जिते ।

महिला टिम फोइलतर्फ एपीएफले वाग्मती प्रदेशलाई ४५–३६ अंकले हरायो । आर्मी र प्रदेश १ तेस्रो भए ।

पुरुष इपी टिमतर्फ आर्मीले एपीएफलाई ४५–४० अंकले पराजित गर्दै स्वर्ण हात जित्यो ।
```

**Predicted Outputs**
```
***** Running Prediction *****
  Num examples = 1
  Batch size = 8
Predicted Category:  Sports
```

### Sample Text (RU-UKR issue)
```
रूसी आक्रमणका कारण शरणार्थी जीवन बिताउन बाध्य युक्रेनीलाई 
छिमेकी देशहरुले खाने, बस्नेलगायतका आधारभूत आवश्यकता उपलब्ध गराइरहेका छन्

जेनेभा — युक्रेनमा रुसले आक्रमण सुरु गरेयता २० लाख सर्वसाधारणले देश छाडेका छन् । शरणार्थीसम्बन्धी संयुक्त राष्ट्रसंघीय निकायका अुनसार विस्थापितहरू पोल्यान्ड, हंगेरी, स्लोभाकिया, मोल्दोभा, रोमानिया पुगेका छन् । 
कम्तीमा १२ लाख ४० हजार जना छिमेकी देश पोल्यान्ड पुगेको जनाइएको छ । 

त्यसैगरी, १ लाख ९१ हजार जना हंगेरी पुगेका छन् । १ लाख ४१ हजार स्लोभाकिया, ८३ हजार मोल्दोभा र ८२ हजार रोमानिया पुगेका छन् । 
त्यस्तै, रुस जानेको संख्या ९९ हजार ३ सय पुगेको छ ।
```

**Predicted Outputs**
```
***** Running Prediction *****
  Num examples = 1
  Batch size = 8
Predicted Category:  World
```