pratyushmaini
commited on
Commit
•
b124912
1
Parent(s):
9f50e53
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -10,6 +10,8 @@ license: mit
|
|
10 |
|
11 |
# Phi-1.5 TOFU Unlearning Model
|
12 |
|
|
|
|
|
13 |
This model is a variant of the Phi-1.5 model, fine-tuned on the TOFU (Task of Fictitious Unlearning) dataset and then subjected to various unlearning algorithms.
|
14 |
|
15 |
## Model Details
|
@@ -27,22 +29,24 @@ This model uses the `KL_1e-05` unlearning algorithm with the following parameter
|
|
27 |
|
28 |
## Revisions
|
29 |
|
30 |
-
The model is organized into multiple revisions, each representing a checkpoint during the unlearning process. The revision names follow the pattern `checkpoint-X`, where X is the checkpoint number.
|
31 |
|
32 |
## Loading the Model
|
33 |
|
34 |
-
To load a specific revision of this model, you
|
35 |
|
36 |
```python
|
37 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
38 |
|
39 |
-
# Replace 'checkpoint-X' with the desired revision (e.g., 'checkpoint-12')
|
40 |
revision = "checkpoint-X"
|
41 |
|
42 |
model = AutoModelForCausalLM.from_pretrained("locuslab/{model_name}", revision=revision)
|
43 |
tokenizer = AutoTokenizer.from_pretrained("locuslab/{model_name}", revision=revision)
|
44 |
```
|
45 |
|
|
|
|
|
46 |
## TOFU Dataset
|
47 |
|
48 |
TOFU (Task of Fictitious Unlearning) is a dataset designed for training and evaluating unlearning algorithms in language models. It simulates scenarios where certain information needs to be "forgotten" or removed from the model's knowledge.
|
@@ -51,7 +55,7 @@ TOFU (Task of Fictitious Unlearning) is a dataset designed for training and eval
|
|
51 |
|
52 |
1. The base Phi-1.5 model was first fine-tuned on the TOFU dataset (checkpoint-625).
|
53 |
2. Various unlearning algorithms were then applied to this fine-tuned model to selectively "forget" certain information.
|
54 |
-
3. The results of these unlearning processes are captured in the different revisions of this model.
|
55 |
|
56 |
## Usage and Limitations
|
57 |
|
|
|
10 |
|
11 |
# Phi-1.5 TOFU Unlearning Model
|
12 |
|
13 |
+
**IMPORTANT: This model's checkpoints are stored in separate branches. You MUST specify a revision when loading the model to access a specific checkpoint.**
|
14 |
+
|
15 |
This model is a variant of the Phi-1.5 model, fine-tuned on the TOFU (Task of Fictitious Unlearning) dataset and then subjected to various unlearning algorithms.
|
16 |
|
17 |
## Model Details
|
|
|
29 |
|
30 |
## Revisions
|
31 |
|
32 |
+
The model is organized into multiple revisions, each representing a checkpoint during the unlearning process. The revision names follow the pattern `checkpoint-X`, where X is the checkpoint number. Each revision is stored in a separate branch.
|
33 |
|
34 |
## Loading the Model
|
35 |
|
36 |
+
To load a specific revision of this model, you MUST specify the revision parameter. Use the following code:
|
37 |
|
38 |
```python
|
39 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
40 |
|
41 |
+
# The 'revision' parameter is REQUIRED. Replace 'checkpoint-X' with the desired revision (e.g., 'checkpoint-12')
|
42 |
revision = "checkpoint-X"
|
43 |
|
44 |
model = AutoModelForCausalLM.from_pretrained("locuslab/{model_name}", revision=revision)
|
45 |
tokenizer = AutoTokenizer.from_pretrained("locuslab/{model_name}", revision=revision)
|
46 |
```
|
47 |
|
48 |
+
**Note: If you don't specify a revision, you will not be able to load the model correctly.**
|
49 |
+
|
50 |
## TOFU Dataset
|
51 |
|
52 |
TOFU (Task of Fictitious Unlearning) is a dataset designed for training and evaluating unlearning algorithms in language models. It simulates scenarios where certain information needs to be "forgotten" or removed from the model's knowledge.
|
|
|
55 |
|
56 |
1. The base Phi-1.5 model was first fine-tuned on the TOFU dataset (checkpoint-625).
|
57 |
2. Various unlearning algorithms were then applied to this fine-tuned model to selectively "forget" certain information.
|
58 |
+
3. The results of these unlearning processes are captured in the different revisions (branches) of this model.
|
59 |
|
60 |
## Usage and Limitations
|
61 |
|