
# Model Card for Model hogru/MolReactGen-USPTO50K-Reaction-Templates

<!-- Provide a quick summary of what the model is/does. -->

MolReactGen is a model that generates reaction templates in SMARTS format (this model) and [molecules in SMILES format](https://huggingface.co/hogru/MolReactGen-GuacaMol-Molecules).

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

MolReactGen is based on the the GPT-2 transformer decoder architecture and has been trained on a [pre-processed version of the USPTO-50K dataset](https://github.com/ml-jku/mhn-react/raw/de0fda32f76f866835aa65a6ff857964302b2178/data/). More information can be found in these [introductory slides](https://github.com/hogru/MolReactGen/blob/main/presentations/Slides%20(A4%20size).pdf).

- **Developed by:** Stephan Holzgruber
- **Model type:** Transformer decoder
- **License:** MIT

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/hogru/MolReactGen
- **Presentation:** https://github.com/hogru/MolReactGen/blob/main/presentations/Slides%20(A4%20size).pdf
- **Poster:** https://github.com/hogru/MolReactGen/blob/main/presentations/Poster%20(A0%20size).pdf

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

The main use of this model is to pass the master's examination of the author ;-)

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

The model can be used in a Hugging Face text generation pipeline. For the intended use case a wrapper around the raw text generation pipeline is needed. This is the [`generate.py` from the repository](https://github.com/hogru/MolReactGen/blob/main/src/molreactgen/generate.py).
The model has a default `GenerationConfig()` (`generation_config.json`) which can be overwritten. Depending on the number of molecules to be generated (`num_return_sequences` in the `JSON` file) this might take a while. The generation code above shows a progress bar during generation.


## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

The model generates reaction templates that are similar to the USPTO-50K training data. Any checks of the reaction templates, e.g. chemical feasiblitly, must be adressed by the user of the model.

## Training Details

### Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[Pre-processed version of the USPTO-50K dataset](https://github.com/ml-jku/mhn-react/raw/de0fda32f76f866835aa65a6ff857964302b2178/data/), originally introduced by [Schneider et al.](https://pubs.acs.org/doi/10.1021/acs.jcim.6b00564).

### Training Procedure 

The default Hugging Face `Trainer()` has been used, with an `EarlyStoppingCallback()`.

### Preprocessing

The training data was pre-processed with a `PreTrainedTokenizerFast()` trained on the training data with a bespoke `RegEx` pre-tokenizer which "understands" the SMARTS syntax.

### Training Hyperparameters

- **Batch size:** 8
- **Gradient accumulation steps:** 4
- **Mixed precision:** fp16, native amp
- **Learning rate:** 0.0005
- **Learning rate scheduler:** Cosine
- **Learning rate scheduler warmup:** 0.1
- **Optimizer:** AdamW with betas=(0.9,0.95) and epsilon=1e-08
- **Number of epochs:** 43 (early stopping)

More configuration (options) can be found in the [`conf`](https://github.com/hogru/MolReactGen/tree/main/src/molreactgen/conf) directory of the repository.

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Please see the slides / the poster mentioned above.

### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Please see the slides / the poster mentioned above.

### Results

Please see the slides / the poster mentioned above.


## Technical Specifications

### Framework versions

- Transformers 4.27.1
- Pytorch 1.13.1
- Datasets 2.10.1
- Tokenizers 0.13.2

### Hardware

- Local PC running Ubuntu 22.04
- NVIDIA GEFORCE RTX 3080Ti (12GB)
{"license": "mit", "tags": ["chemistry", "smarts"], "widget": [{"text": "^", "example_title": "Sample reaction template | SMARTS"}]}
transformers pytorch gpt2 text-generation chemistry smarts license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-generation #chemistry #smarts #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


# Model Card for Model hogru/MolReactGen-USPTO50K-Reaction-Templates



MolReactGen is a model that generates reaction templates in SMARTS format (this model) and molecules in SMILES format.

## Model Details

### Model Description



MolReactGen is based on the the GPT-2 transformer decoder architecture and has been trained on a pre-processed version of the USPTO-50K dataset. More information can be found in these introductory URL).

- Developed by: Stephan Holzgruber
- Model type: Transformer decoder
- License: MIT

### Model Sources



- Repository: URL
- Presentation: URL
- Poster: URL

## Uses



The main use of this model is to pass the master's examination of the author ;-)

### Direct Use



The model can be used in a Hugging Face text generation pipeline. For the intended use case a wrapper around the raw text generation pipeline is needed. This is the 'URL' from the repository.
The model has a default 'GenerationConfig()' ('generation_config.json') which can be overwritten. Depending on the number of molecules to be generated ('num_return_sequences' in the 'JSON' file) this might take a while. The generation code above shows a progress bar during generation.


## Bias, Risks, and Limitations



The model generates reaction templates that are similar to the USPTO-50K training data. Any checks of the reaction templates, e.g. chemical feasiblitly, must be adressed by the user of the model.

## Training Details

### Training Data



Pre-processed version of the USPTO-50K dataset, originally introduced by Schneider et al..

### Training Procedure 

The default Hugging Face 'Trainer()' has been used, with an 'EarlyStoppingCallback()'.

### Preprocessing

The training data was pre-processed with a 'PreTrainedTokenizerFast()' trained on the training data with a bespoke 'RegEx' pre-tokenizer which "understands" the SMARTS syntax.

### Training Hyperparameters

- Batch size: 8
- Gradient accumulation steps: 4
- Mixed precision: fp16, native amp
- Learning rate: 0.0005
- Learning rate scheduler: Cosine
- Learning rate scheduler warmup: 0.1
- Optimizer: AdamW with betas=(0.9,0.95) and epsilon=1e-08
- Number of epochs: 43 (early stopping)

More configuration (options) can be found in the 'conf' directory of the repository.

## Evaluation



Please see the slides / the poster mentioned above.

### Metrics



Please see the slides / the poster mentioned above.

### Results

Please see the slides / the poster mentioned above.


## Technical Specifications

### Framework versions

- Transformers 4.27.1
- Pytorch 1.13.1
- Datasets 2.10.1
- Tokenizers 0.13.2

### Hardware

- Local PC running Ubuntu 22.04
- NVIDIA GEFORCE RTX 3080Ti (12GB)
# Model Card for Model hogru/MolReactGen-USPTO50K-Reaction-Templates



MolReactGen is a model that generates reaction templates in SMARTS format (this model) and molecules in SMILES format. ## Model Details ### Model Description



MolReactGen is based on the the GPT-2 transformer decoder architecture and has been trained on a pre-processed version of the USPTO-50K dataset. More information can be found in these introductory URL).

- Developed by: Stephan Holzgruber
- Model type: Transformer decoder
- License: MIT ### Model Sources



- Repository: URL
- Presentation: URL
- Poster: URL ## Uses



The main use of this model is to pass the master's examination of the author ;-) ### Direct Use



The model can be used in a Hugging Face text generation pipeline. For the intended use case a wrapper around the raw text generation pipeline is needed. This is the 'URL' from the repository.
The model has a default 'GenerationConfig()' ('generation_config.json') which can be overwritten. Depending on the number of molecules to be generated ('num_return_sequences' in the 'JSON' file) this might take a while. The generation code above shows a progress bar during generation. ## Bias, Risks, and Limitations



The model generates reaction templates that are similar to the USPTO-50K training data. Any checks of the reaction templates, e.g. chemical feasiblitly, must be adressed by the user of the model. ## Training Details ### Training Data



Pre-processed version of the USPTO-50K dataset, originally introduced by Schneider et al.. ### Training Procedure 

The default Hugging Face 'Trainer()' has been used, with an 'EarlyStoppingCallback()'. ### Preprocessing

The training data was pre-processed with a 'PreTrainedTokenizerFast()' trained on the training data with a bespoke 'RegEx' pre-tokenizer which "understands" the SMARTS syntax. ### Training Hyperparameters

- Batch size: 8
- Gradient accumulation steps: 4
- Mixed precision: fp16, native amp
- Learning rate: 0.0005
- Learning rate scheduler: Cosine
- Learning rate scheduler warmup: 0.1
- Optimizer: AdamW with betas=(0.9,0.95) and epsilon=1e-08
- Number of epochs: 43 (early stopping)

More configuration (options) can be found in the 'conf' directory of the repository. ## Evaluation



Please see the slides / the poster mentioned above. ### Metrics



Please see the slides / the poster mentioned above. ### Results

Please see the slides / the poster mentioned above. ## Technical Specifications ### Framework versions

- Transformers 4.27.1
- Pytorch 1.13.1
- Datasets 2.10.1
- Tokenizers 0.13.2 ### Hardware

- Local PC running Ubuntu 22.04
- NVIDIA GEFORCE RTX 3080Ti (12GB)
TAGS
#transformers #pytorch #gpt2 #text-generation #chemistry #smarts #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # Model Card for Model hogru/MolReactGen-USPTO50K-Reaction-Templates



MolReactGen is a model that generates reaction templates in SMARTS format (this model) and molecules in SMILES format. ## Model Details ### Model Description



MolReactGen is based on the the GPT-2 transformer decoder architecture and has been trained on a pre-processed version of the USPTO-50K dataset. More information can be found in these introductory URL).

- Developed by: Stephan Holzgruber
- Model type: Transformer decoder
- License: MIT ### Model Sources



- Repository: URL
- Presentation: URL
- Poster: URL ## Uses



The main use of this model is to pass the master's examination of the author ;-) ### Direct Use



The model can be used in a Hugging Face text generation pipeline. For the intended use case a wrapper around the raw text generation pipeline is needed. This is the 'URL' from the repository.
The model has a default 'GenerationConfig()' ('generation_config.json') which can be overwritten. Depending on the number of molecules to be generated ('num_return_sequences' in the 'JSON' file) this might take a while. The generation code above shows a progress bar during generation. ## Bias, Risks, and Limitations



The model generates reaction templates that are similar to the USPTO-50K training data. Any checks of the reaction templates, e.g. chemical feasiblitly, must be adressed by the user of the model. ## Training Details ### Training Data



Pre-processed version of the USPTO-50K dataset, originally introduced by Schneider et al.. ### Training Procedure 

The default Hugging Face 'Trainer()' has been used, with an 'EarlyStoppingCallback()'. ### Preprocessing

The training data was pre-processed with a 'PreTrainedTokenizerFast()' trained on the training data with a bespoke 'RegEx' pre-tokenizer which "understands" the SMARTS syntax. ### Training Hyperparameters

- Batch size: 8
- Gradient accumulation steps: 4
- Mixed precision: fp16, native amp
- Learning rate: 0.0005
- Learning rate scheduler: Cosine
- Learning rate scheduler warmup: 0.1
- Optimizer: AdamW with betas=(0.9,0.95) and epsilon=1e-08
- Number of epochs: 43 (early stopping)

More configuration (options) can be found in the 'conf' directory of the repository. ## Evaluation



Please see the slides / the poster mentioned above. ### Metrics



Please see the slides / the poster mentioned above. ### Results

Please see the slides / the poster mentioned above. ## Technical Specifications ### Framework versions

- Transformers 4.27.1
- Pytorch 1.13.1
- Datasets 2.10.1
- Tokenizers 0.13.2 ### Hardware

- Local PC running Ubuntu 22.04
- NVIDIA GEFORCE RTX 3080Ti (12GB)