Update README.md

README.md

@@ -1,43 +1,84 @@
 ---
 tags:
 - generated_from_trainer
+- finance
+- text-generation-inference
 model-index:
 - name: completed-model
   results: []
+language:
+- en
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
-# completed-model
-
-This model was trained from scratch on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.3186
-- Rewards/chosen: -0.6296
-- Rewards/rejected: -2.5591
-- Rewards/accuracies: 0.8571
-- Rewards/margins: 1.9295
-- Logps/rejected: -296.3221
-- Logps/chosen: -425.5087
-- Logits/rejected: -2.2481
-- Logits/chosen: -1.7413
+
+Albatross is a collection of domain-specific language models for finance applications developed by [Gradient AI](https://gradient.ai/).
+
+This is the repository for an initial, demonstration version, the `v-alpha-tross`.
+
 
 ## Model description
 
-More information needed
+The `v-alpha-tross` is based on [meta-llama/Llama-2-70b-hf](https://huggingface.co/meta-llama/Llama-2-70b-hf), with additional, finance specific, pre-training, fine-tuning and instruction tuning.
+
+This model substantially outperforms Llama2-70B models on H6 Average score, and GSM8K. It also reaches perfect performance in extracting information from tabular data like those found in SEC filings.
+
 
 ## Intended uses & limitations
 
-More information needed
+The `v-alpha-tross` is intended as a demonstration of Gradient AI's Albatross framework for developing large language models specific to the finance domain. We welcome additional research and development, but do not plan on continued internal development on this legacy model.
+
+To get the expected performance, follow formatting requirements of *Llama-2 chat*, including `INST` and `<<SYS>>` tags, and `<s>` tokens.
+
+## Training Strategy
+
+The Albatross framework overcomes deficiencies in general-purpose language models when faced with solving finance specific tasks.
+
+### Pre-Training
+
+A base Llama2-70B is further pre-trained on finance specific data since LLMs are poor at answering questions when their internal relevant document store is sparse [1].
+
+To curate enough quality training data with low operational overhead we use a novel data gathering approach: 
+1. Crawl public repositories of text data including [Red Pajamas](https://github.com/togethercomputer/RedPajama-Data) and [https://github.com/](https://github.com/).
+2. Programatically filter our crawled corpus to datasets not likely to be already in the base model's training, using adapted LiRA membership inference.[2]
+3. Human professionals review the (much smaller) filtered corpus to further remove low quality results. 
+
+[1] Kandpal, Nikhil, et al. "Large language models struggle to learn long-tail knowledge." International Conference on Machine Learning. PMLR, 2023.
+
+[2] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.
+
+### Fine-Tuning
+
+Supervised fine turning (SFT) and direct preference optimization (DPO)[3] further enhances performance in finance-related tasks. Specifically, we focus on improving English conversational communication, summarization, financial anchoring, mathematical reasoning, and tabular understanding.
+
+Fine tuning was performed with processed versions of following datasets:
 
-## Training and evaluation data
+- ultrachat_200k (SFT)
+- airoboros-3.1 (SFT)
+- SlimOrca-Dedup (SFT)
+- MetaMathQA (SFT)
+- GOAT - Arithmetic CoT (SFT)
+- table_instruct (SFT)*
+- flare-tatqa (SFT)
+- ECT_Summaries (SFT)
+- legal_summarization (SFT)
+- selfrag_train_data (SFT)
+- ragas-wikiqa (SFT)
+- rag_instruct_test_dataset2_financial_0.1 (SFT)
+- fingpt-ner (SFT)
+- fingpt-finred (SFT)
+- UltraFeedback (DPO)
+- orca_dpo_pairs (DPO)
+- fin_feedback (DPO)*
 
-More information needed
+(*) = Proprietary
+
+[3] Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C.D. and Finn, C., 2023. Direct preference optimization: Your language model is secretly a reward model. NeurIPS.
 
 ## Training procedure
 
+Our training pipeline includes [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl), build on top of huggingface [Accelerate](https://github.com/huggingface/accelerate), leveraging [Deepspeed](https://github.com/microsoft/DeepSpeed). We use a custom distributed training approach on Azure spot instances to be throughput optimized and memory efficient.
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
@@ -55,24 +96,40 @@ The following hyperparameters were used during training:
 - num_epochs: 1
 - dpo_beta: .1
 
-### Training results
-
-| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
-|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.4772        | 0.1   | 61   | 0.4037          | -0.3256        | -1.2099          | 0.8095             | 0.8843          | -282.8301      | -422.4691    | -2.1649         | -1.6678       |
-| 0.3859        | 0.2   | 122  | 0.3681          | -0.3816        | -1.7445          | 0.7143             | 1.3629          | -288.1762      | -423.0287    | -2.2536         | -1.7385       |
-| 0.3061        | 0.3   | 183  | 0.3546          | -0.4969        | -2.1025          | 0.8095             | 1.6056          | -291.7559      | -424.1818    | -2.1989         | -1.7108       |
-| 0.3765        | 0.4   | 244  | 0.3374          | -0.5153        | -2.1301          | 0.7619             | 1.6148          | -292.0326      | -424.3660    | -2.2182         | -1.7222       |
-| 0.2819        | 0.5   | 305  | 0.3303          | -0.4402        | -2.1809          | 0.8095             | 1.7407          | -292.5404      | -423.6147    | -2.1835         | -1.6998       |
-| 0.3009        | 0.6   | 366  | 0.3314          | -0.8026        | -2.7756          | 0.8571             | 1.9730          | -298.4871      | -427.2388    | -2.2430         | -1.7529       |
-| 0.3015        | 0.7   | 427  | 0.3228          | -0.6439        | -2.5710          | 0.9048             | 1.9271          | -296.4410      | -425.6519    | -2.2258         | -1.7303       |
-| 0.3407        | 0.8   | 488  | 0.3185          | -0.7270        | -2.7118          | 0.8571             | 1.9847          | -297.8488      | -426.4829    | -2.2530         | -1.7496       |
-| 0.3149        | 0.9   | 549  | 0.3186          | -0.6296        | -2.5591          | 0.8571             | 1.9295          | -296.3221      | -425.5087    | -2.2481         | -1.7413       |
-
-
 ### Framework versions
 
 - Transformers 4.35.1
 - Pytorch 2.0.1+cu118
 - Datasets 2.14.7
 - Tokenizers 0.14.1
+
+## Benchmarks
+
+From a Llama-2-70B base, we substantially improve H6 metrics, and in particular GSM8k (arithmetic reasoning). We also achieve performance for simple financial table understanding on par with GPT-4.
+
+| Model                                  | H6    | GSM8k | simple_tables
+|----------------------------------------|-------|------------|-------
+| **v-alpha-tross**                      | **72.81** | **61.79** | **100.0**
+| meta-llama/Llama-2-70B-hf              | 67.87 | 54.06    | 75.76 
+| meta-llama/Llama-2-70b-chat-hf         | 62.4 | 26.69    | 87.88
+| GPT-4                                  | N/A | 87.1 [4] | **100.0**
+| GPT-3.5                                | N/A | 57.1 [4] | **100.0**
+
+
+[4]
+([https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k](https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k)
+
+## License
+
+<span style="color:red">TODO</span>.
+
+## How to cite
+
+Whitepaper coming soon!
+
+
+## The Gradient AI Team
+Gradient is accelerating AI transformation across industries. For more information, see [https://gradient.ai/](https://gradient.ai/)
+
+## Contact Us
+Any questions and suggestions, please use the discussion tab. If you want to contact us directly, drop an email to <span style="color:red">TODO</span>