
<div style="text-align:center">
<img src="https://big-cheng.com/k2/k2.png" alt="k2-logo" width="200"/>
<h2>Delta Model for Large Language Model for Geoscience</h2>
</div>

## Introduction

We introduce **K2** (7B), an open-source language model trained by firstly further pretraining LLaMA on collected and cleaned geoscience literature, including geoscience open-access papers and Wikipedia pages, and secondly fine-tuning with knowledge-intensive instruction tuning data (GeoSignal). As for preliminary evaluation, we use GeoBenchmark (consisting of NPEE and AP Test on Geology, Geography, and Environmental Science) as the benchmark. K2 outperforms the baselines on objective and subjective tasks compared to several baseline models with similar parameters. 
We release K2 delta weights after further pretraining with the geoscience text corpus to comply with the LLaMA model license. 

***The following is the overview of training K2:***
![overview](https://big-cheng.com/k2/overview.png)

## How to Use

Please refer to [K2](https://github.com/davendw49/k2) Github repo for further usage.

{"language": ["en"], "license": "apache-2.0", "datasets": ["daven3/geosignal"]}
transformers pytorch llama text-generation en dataset:daven3/geosignal license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #llama #text-generation #en #dataset-daven3/geosignal #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


<div style="text-align:center">
<img src="URL alt="k2-logo" width="200"/>
<h2>Delta Model for Large Language Model for Geoscience</h2>
</div>

## Introduction

We introduce K2 (7B), an open-source language model trained by firstly further pretraining LLaMA on collected and cleaned geoscience literature, including geoscience open-access papers and Wikipedia pages, and secondly fine-tuning with knowledge-intensive instruction tuning data (GeoSignal). As for preliminary evaluation, we use GeoBenchmark (consisting of NPEE and AP Test on Geology, Geography, and Environmental Science) as the benchmark. K2 outperforms the baselines on objective and subjective tasks compared to several baseline models with similar parameters. 
We release K2 delta weights after further pretraining with the geoscience text corpus to comply with the LLaMA model license. 

*The following is the overview of training K2:*
!overview

## How to Use

Please refer to K2 Github repo for further usage.

## Introduction

We introduce K2 (7B), an open-source language model trained by firstly further pretraining LLaMA on collected and cleaned geoscience literature, including geoscience open-access papers and Wikipedia pages, and secondly fine-tuning with knowledge-intensive instruction tuning data (GeoSignal). As for preliminary evaluation, we use GeoBenchmark (consisting of NPEE and AP Test on Geology, Geography, and Environmental Science) as the benchmark. K2 outperforms the baselines on objective and subjective tasks compared to several baseline models with similar parameters. 
We release K2 delta weights after further pretraining with the geoscience text corpus to comply with the LLaMA model license. 

*The following is the overview of training K2:*
!overview ## How to Use

Please refer to K2 Github repo for further usage.
TAGS
#transformers #pytorch #llama #text-generation #en #dataset-daven3/geosignal #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ## Introduction

We introduce K2 (7B), an open-source language model trained by firstly further pretraining LLaMA on collected and cleaned geoscience literature, including geoscience open-access papers and Wikipedia pages, and secondly fine-tuning with knowledge-intensive instruction tuning data (GeoSignal). As for preliminary evaluation, we use GeoBenchmark (consisting of NPEE and AP Test on Geology, Geography, and Environmental Science) as the benchmark. K2 outperforms the baselines on objective and subjective tasks compared to several baseline models with similar parameters. 
We release K2 delta weights after further pretraining with the geoscience text corpus to comply with the LLaMA model license. 

*The following is the overview of training K2:*
!overview ## How to Use

Please refer to K2 Github repo for further usage.