# Model Card for IDMGSP-Galactica-TRAIN-CG

A fine-tuned Galactica model to detect machine-generated scientific papers based on their abstract, introduction, and conclusion. 

This model is trained on the `train-cg` dataset found in https://huggingface.co/datasets/tum-nlp/IDMGSP.

# this model card is WIP, please check the repository, the dataset card and the paper for more details.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Developed by:** Technical University of Munich (TUM)
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** English
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** Galactica

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/qwenzo/-IDMGSP
- **Paper:** [More Information Needed]

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

```python
from transformers import AutoTokenizer, OPTForSequenceClassification, pipeline

model = OPTForSequenceClassification.from_pretrained("tum-nlp/IDMGSP-Galactica-TRAIN-CG")
tokenizer = AutoTokenizer.from_pretrained("tum-nlp/IDMGSP-Galactica-TRAIN-CG")
reader = pipeline("text-classification", model=model, tokenizer = tokenizer)
reader(
'''
Abstract:
....

Introduction:
....

Conclusion:
...'''
)
```

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

## Training Details

### Training Data

The training dataset comprises scientific papers generated by the Galactica, GPT-2, and SCIgen models, as well as papers extracted from the arXiv database.

The provided table displays the sample counts from each source utilized in constructing the training dataset. 
The dataset could be found in https://huggingface.co/datasets/tum-nlp/IDMGSP.

| Dataset                      | arXiv (real) | ChatGPT (fake) | GPT-2 (fake) | SCIgen (fake) | Galactica (fake) | GPT-3 (fake) |
|------------------------------|--------------|----------------|--------------|----------------|------------------|--------------|
| TRAIN without ChatGPT (TRAIN-CG) | 8k         | -              | 2k           | 2k             | 2k               | -            |

### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

[More Information Needed]


#### Training Hyperparameters

[More Information Needed]

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Data Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed]

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information [optional]

[More Information Needed]

## Model Card Authors [optional]

[More Information Needed]

## Model Card Contact

[More Information Needed]



{"language": ["en"], "tags": ["scientific paper", "fake papers", "science", "scientific text"], "datasets": ["tum-nlp/IDMGSP"], "widget": [{"text": "Abstract:\n\nThe Hartree-Fock (HF) method is a widely used method for approximating the electronic structure of many-electron systems. In this work, we study the properties of HF solutions of the three-dimensional electron gas (3DEG), a model system consisting of a uniform, non-interacting electron gas in three dimensions. We find that the HF solutions accurately reproduce the known analytic results for the ground state energy and the static structure factor of the 3DEG. However, we also find that the HF solutions fail to accurately describe the excitation spectrum of the 3DEG, particularly at high energies.\n\nIntroduction:\n\nThe HF method is a self-consistent method for approximating the electronic structure of many-electron systems. It is based on the assumption that the electrons in a system can be described as non-interacting quasiparticles, each with its own effective potential. The HF method is commonly used to study the ground state properties of systems, such as the energy and the density distribution, but it can also be used to study excited states.\n\nThe 3DEG is a model system that has been widely studied as a test case for electronic structure methods. It consists of a uniform, non-interacting electron gas in three dimensions, with a finite density and a periodic boundary condition. The 3DEG has a number of known analytic results for its ground state properties, such as the ground state energy and the static structure factor, which can be used to test the accuracy of approximate methods.\n\nConclusion:\n\nIn this work, we have studied the properties of HF solutions of the 3DEG. We find that the HF solutions accurately reproduce the known analytic results for the ground state energy and the static structure factor of the 3DEG. However, we also find that the HF solutions fail to accurately describe the excitation spectrum of the 3DEG, particularly at high energies. This suggests that the HF method may not be suitable for accurately describing the excited states of the 3DEG. Further work is needed to understand the limitations of the HF method and to develop improved methods for studying the electronic structure of many-electron systems.\n", "example_title": "Example ChatGPT fake"}, {"text": "Abstract:\n\nRecent calculations have pointed to a 2.8 $\\sigma$ tension between data on $\\epsilon^{\\prime}_K / \\epsilon_K$ and the standard-model (SM) prediction. Several new physics (NP) models can explain this discrepancy, and such NP models are likely to predict deviations of $\\mathcal{B}(K\\to \\pi \\nu \\overline{\\nu})$ from the SM predictions, which can be probed precisely in the near future by NA62 and KOTO experiments. We present correlations between $\\epsilon^{\\prime}_K / \\epsilon_K$ and $\\mathcal{B}(K\\to \\pi \\nu \\overline{\\nu})$ in two types of NP scenarios: a box dominated scenario and a $Z$-penguin dominated one. It is shown that different correlations are predicted and the future precision measurements of $K \\to \\pi \\nu \\overline{\\nu}$ can distinguish both scenarios.\n\nIntroduction:\n\nCP violating \ufb02avor-changing neutral current decays of K mesons are extremely sensitive to new physics (NP) and can probe virtual e\ufb00ects of particles with masses far above the reach of the Large Hadron Collider. Prime examples of such observables are \u03f5\u2032 K measuring direct CP violation in K \u2192 \u03c0\u03c0 decays and B(KL \u2192 \u03c00\u03bd\u03bd). Until recently, large theoretical uncertainties precluded reliable predictions for \u03f5\u2032 K. Although standard-model (SM) predictions of \u03f5\u2032 K using chiral perturbation theory are consistent with the experimental value, their theoretical uncertainties are large. In contrast, calculation by the dual QCD approach 1 \ufb01nds the SM value much below the experimental one. A major breakthrough has been the recent lattice-QCD calculation of the hadronic matrix elements by RBC-UKQCD collaboration 2, which gives support to the latter result. The SM value at the next-to-leading order divided by the indirect CP violating measure \u03f5K is 3 which is consistent with (\u03f5\u2032 K/\u03f5K)SM = (1.9\u00b14.5)\u00d710\u22124 given by Buras et al 4.a Both results are based on the lattice numbers, and further use CP-conserving K \u2192 \u03c0\u03c0 data to constrain some of the hadronic matrix elements involved. Compared to the world average of the experimental results 6, Re (\u03f5\u2032 K/\u03f5K)exp = (16.6 \u00b1 2.3) \u00d7 10\u22124, (2) the SM prediction lies below the experimental value by 2.8 \u03c3. Several NP models including supersymmetry (SUSY) can explain this discrepancy. It is known that such NP models are likely to predict deviations of the kaon rare decay branching ratios from the SM predictions, especially B(K \u2192 \u03c0\u03bd\u03bd) which can be probed precisely in the near future by NA62 and KOTO experiments.b In this contribution, we present correlations between \u03f5\u2032 K/\u03f5K and B(K \u2192 \u03c0\u03bd\u03bd) in two types of NP scenarios: a box dominated scenario and a Z-penguin dominated one. Presented at the 52th Rencontres de Moriond electroweak interactions and uni\ufb01ed theories, La Thuile, Italy, 18-25 March, 2017. aOther estimations of the SM value are listed in Kitahara et al 5. b The correlations between \u03f5\u2032 K/\u03f5K, B(K \u2192 \u03c0\u03bd\u03bd) and \u03f5K through the CKM components in the SM are discussed in Ref. 7.\n\nConclusion:\n\nWe have presented the correlations between \u03f5\u2032 K/\u03f5K, B(KL \u2192 \u03c00\u03bd\u03bd), and B(K+ \u2192 \u03c0+\u03bd\u03bd) in the box dominated scenario and the Z-penguin dominated one. It is shown that the constraint from \u03f5K produces di\ufb00erent correlations between two NP scenarios. In the future, measurements of B(K \u2192 \u03c0\u03bd\u03bd) will be signi\ufb01cantly improved. The NA62 experiment at CERN measuring B(K+ \u2192 \u03c0+\u03bd\u03bd) is aiming to reach a precision of 10 % compared to the SM value already in 2018. In order to achieve 5% accuracy more time is needed. Concerning KL \u2192 \u03c00\u03bd\u03bd, the KOTO experiment at J-PARC aims in a \ufb01rst step at measuring B(KL \u2192 \u03c00\u03bd\u03bd) around the SM sensitivity. Furthermore, the KOTO-step2 experiment will aim at 100 events for the SM branching ratio, implying a precision of 10 % of this measurement. Therefore, we conclude that when the \u03f5\u2032 K/\u03f5K discrepancy is explained by the NP contribution, NA62 experiment could probe whether a modi\ufb01ed Z-coupling scenario is realized or not, and KOTO-step2 experiment can distinguish the box dominated scenario and the simpli\ufb01ed modi\ufb01ed Z-coupling scenario.\n", "example_title": "Example real"}]}
transformers pytorch opt text-classification scientific paper fake papers science scientific text en dataset:tum-nlp/IDMGSP autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #opt #text-classification #scientific paper #fake papers #science #scientific text #en #dataset-tum-nlp/IDMGSP #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

Model Card for IDMGSP-Galactica-TRAIN-CG
========================================


A fine-tuned Galactica model to detect machine-generated scientific papers based on their abstract, introduction, and conclusion.


This model is trained on the 'train-cg' dataset found in URL


this model card is WIP, please check the repository, the dataset card and the paper for more details.
=====================================================================================================


Model Details
-------------


### Model Description


* Developed by: Technical University of Munich (TUM)
* Model type:
* Language(s) (NLP): English
* License:
* Finetuned from model [optional]: Galactica


### Model Sources


* Repository: URL
* Paper:


Uses
----


### Direct Use


### Downstream Use [optional]


### Out-of-Scope Use


Bias, Risks, and Limitations
----------------------------


Training Details
----------------


### Training Data


The training dataset comprises scientific papers generated by the Galactica, GPT-2, and SCIgen models, as well as papers extracted from the arXiv database.


The provided table displays the sample counts from each source utilized in constructing the training dataset.
The dataset could be found in URL



### Training Procedure


#### Preprocessing [optional]


#### Training Hyperparameters


#### Speeds, Sizes, Times [optional]


Evaluation
----------


### Testing Data, Factors & Metrics


#### Testing Data


#### Factors


#### Metrics


### Results


#### Summary


Model Examination [optional]
----------------------------


Environmental Impact
--------------------


* Hardware Type:
* Hours used:
* Cloud Provider:
* Compute Region:
* Carbon Emitted:


Technical Specifications [optional]
-----------------------------------


### Model Architecture and Objective


### Compute Infrastructure


#### Hardware


#### Software


[optional]


BibTeX:


APA:


Glossary [optional]
-------------------


More Information [optional]
---------------------------


Model Card Authors [optional]
-----------------------------


Model Card Contact
------------------



### Model Description


* Developed by: Technical University of Munich (TUM)
* Model type:
* Language(s) (NLP): English
* License:
* Finetuned from model [optional]: Galactica ### Model Sources


* Repository: URL
* Paper:


Uses
---- ### Direct Use ### Downstream Use [optional] ### Out-of-Scope Use


Bias, Risks, and Limitations
----------------------------


Training Details
---------------- ### Training Data


The training dataset comprises scientific papers generated by the Galactica, GPT-2, and SCIgen models, as well as papers extracted from the arXiv database.


The provided table displays the sample counts from each source utilized in constructing the training dataset.
The dataset could be found in URL ### Training Procedure #### Preprocessing [optional] #### Training Hyperparameters #### Speeds, Sizes, Times [optional]


Evaluation
---------- ### Testing Data, Factors & Metrics #### Testing Data #### Factors #### Metrics ### Results #### Summary


Model Examination [optional]
----------------------------


Environmental Impact
--------------------


* Hardware Type:
* Hours used:
* Cloud Provider:
* Compute Region:
* Carbon Emitted:


Technical Specifications [optional]
----------------------------------- ### Model Architecture and Objective ### Compute Infrastructure #### Hardware #### Software


[optional]


BibTeX:


APA:


Glossary [optional]
-------------------


More Information [optional]
---------------------------


Model Card Authors [optional]
-----------------------------


Model Card Contact
------------------
TAGS
#transformers #pytorch #opt #text-classification #scientific paper #fake papers #science #scientific text #en #dataset-tum-nlp/IDMGSP #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ### Model Description


* Developed by: Technical University of Munich (TUM)
* Model type:
* Language(s) (NLP): English
* License:
* Finetuned from model [optional]: Galactica ### Model Sources


* Repository: URL
* Paper:


Uses
---- ### Direct Use ### Downstream Use [optional] ### Out-of-Scope Use


Bias, Risks, and Limitations
----------------------------


Training Details
---------------- ### Training Data


The training dataset comprises scientific papers generated by the Galactica, GPT-2, and SCIgen models, as well as papers extracted from the arXiv database.


The provided table displays the sample counts from each source utilized in constructing the training dataset.
The dataset could be found in URL ### Training Procedure #### Preprocessing [optional] #### Training Hyperparameters #### Speeds, Sizes, Times [optional]


Evaluation
---------- ### Testing Data, Factors & Metrics #### Testing Data #### Factors #### Metrics ### Results #### Summary


Model Examination [optional]
----------------------------


Environmental Impact
--------------------


* Hardware Type:
* Hours used:
* Cloud Provider:
* Compute Region:
* Carbon Emitted:


Technical Specifications [optional]
----------------------------------- ### Model Architecture and Objective ### Compute Infrastructure #### Hardware #### Software


[optional]


BibTeX:


APA:


Glossary [optional]
-------------------


More Information [optional]
---------------------------


Model Card Authors [optional]
-----------------------------


Model Card Contact
------------------