wisalkhanmv commited on
Commit
78887c2
1 Parent(s): 67b2869

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +104 -0
README.md CHANGED
@@ -13,3 +13,107 @@ tags:
13
  - rating
14
  - lowerated
15
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  - rating
14
  - lowerated
15
  ---
16
+
17
+ # Lowerated/deberta-v3-lm6
18
+
19
+ ## Model Details
20
+ **Model Name:** Lowerated/deberta-v3-lm6
21
+ **Model Type:** Text Classification (Aspect-Based Sentiment Analysis)
22
+ **Language:** English
23
+ **Framework:** PyTorch
24
+ **License:** Apache 2.0
25
+
26
+ ## Model Description
27
+ Lowerated/deberta-v3-lm6 is a DeBERTa-v3-based model fine-tuned for aspect-based sentiment analysis on IMDb movie reviews. The model is designed to classify sentiments across seven key aspects of filmmaking: Cinematography, Direction, Story, Characters, Production Design, Unique Concept, and Emotions.
28
+
29
+ ## Dataset
30
+ **Dataset Name:** Lowerated/imdb-reviews-rated
31
+ **Dataset URL:** [IMDb Reviews Rated](https://huggingface.co/datasets/LOWERATED/imdb-reviews-rated)
32
+ **Dataset Description:** The dataset contains IMDb movie reviews with sentiment scores for seven aspects of filmmaking. Each review is labeled with sentiment scores for Cinematography, Direction, Story, Characters, Production Design, Unique Concept, and Emotions.
33
+
34
+ ## Performance
35
+ **Evaluation Metric:** Mean Squared Error (MSE)
36
+ **MSE:** 0.08594679832458496
37
+
38
+ ### Detailed Results
39
+
40
+ **Cinematography:**
41
+ - **Precision:** 0.96
42
+ - **Recall:** 0.97
43
+ - **F1-score:** 0.96
44
+ - **Accuracy:** 0.95
45
+
46
+ **Confusion Matrix:**
47
+ \[
48
+ \begin{bmatrix}
49
+ 68 & 0 & 0 \\
50
+ 1 & 377 & 37 \\
51
+ 0 & 0 & 310 \\
52
+ \end{bmatrix}
53
+ \]
54
+
55
+ **Direction:**
56
+ - **Precision:** 0.93
57
+ - **Recall:** 0.97
58
+ - **F1-score:** 0.94
59
+ - **Accuracy:** 0.95
60
+
61
+ **Story:**
62
+ - **Precision:** 0.85
63
+ - **Recall:** 0.88
64
+ - **F1-score:** 0.85
65
+ - **Accuracy:** 0.85
66
+
67
+
68
+ **Characters:**
69
+ - **Precision:** 0.89
70
+ - **Recall:** 0.89
71
+ - **F1-score:** 0.89
72
+ - **Accuracy:** 0.90
73
+
74
+ **Production Design:**
75
+ - **Precision:** 0.95
76
+ - **Recall:** 0.98
77
+ - **F1-score:** 0.96
78
+ - **Accuracy:** 0.96
79
+
80
+ **Unique Concept:**
81
+ - **Precision:** 0.83
82
+ - **Recall:** 1.00
83
+ - **F1-score:** 0.89
84
+ - **Accuracy:** 1.00
85
+
86
+ **Emotions:**
87
+ - **Precision:** 0.76
88
+ - **Recall:** 0.87
89
+ - **F1-score:** 0.78
90
+ - **Accuracy:** 0.82
91
+
92
+ **Test Results:**
93
+ - **Eval Loss:** 0.08594681322574615
94
+ - **Eval Model Preparation Time:** 0.0011
95
+ - **Eval MSE:** 0.08594679832458496
96
+ - **Eval Runtime:** 23.1411
97
+ - **Eval Samples per Second:** 34.268
98
+ - **Eval Steps per Second:** 8.599
99
+
100
+ ## Intended Use
101
+ This model is intended for rating of movies across seven aspects of filmmaking. It can be used to provide a more nuanced understanding of viewer opinions and improve movie rating systems.
102
+
103
+ ## Limitations
104
+ While the model performs well on the evaluation dataset, its performance may vary on different datasets. Continuous monitoring and retraining with diverse data are recommended to maintain and improve its accuracy.
105
+
106
+ ## Future Work
107
+ Future improvements could focus on exploring alternative methods for handling neutral values, investigating advanced techniques for addressing missing ratings, enhancing sentiment analysis methods, and expanding the range of aspects analyzed.
108
+
109
+ ## Citation
110
+ If you use this model in your research, please cite it as follows:
111
+
112
+ ```bibtex
113
+ @model{lowerated_deberta-v3-lm6,
114
+ author = {LOWERATED},
115
+ title = {deberta-v3-lm6},
116
+ year = {2024},
117
+ url = {https://huggingface.co/Lowerated/deberta-v3-lm6},
118
+ }
119
+ ```