YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Falcon-7B-Hawkish-Lora
rev1-119
Hawkish vs BloombergGPT
| FinBenchmark | Hawkish-7B | BloombergGPT |
|---|---|---|
| Headline (SA)* | 71.29 | 82.20 |
| FiQA SA | 77.01 | 75.07 |
| FPB | 73.41 | 51.07 |
Hawkish vs CFA Level 1 Mock Exam
| CFA Level 1 Past Paper | # Questions | ChatGPT (%) | Falcon-Instruct (%) | Hawkish-7B (%) | Exam Weighting |
|---|---|---|---|---|---|
| Ethical and Professional | 18 | 66.66 | 33.3 | 50.0 | 0.15 |
| Quantitative Methods | 14 | 57.15 | 50.0 | 78.57 | 0.1 |
| Economics | 12 | 58.33 | 25.0 | 50.0 | 0.1 |
| Financial Reporting | 24 | 37.5 | 25.0 | 58.3 | 0.15 |
| Corporate Finance | 10 | 40.0 | 20.0 | 60.0 | 0.1 |
| Equity Investments | 12 | 58.3 | 33.3 | 66.6 | 0.11 |
| Fixed Income | 14 | 50.0 | 35.7 | 57.15 | 0.11 |
| Derivatives | 6 | 66.6 | 16.7 | 50.0 | 0.06 |
| Alternative Investments | 4 | 50.0 | 0.0 | 25.0 | 0.06 |
| Portfolio Management | 6 | 0.0 | 50.0 | 50.0 | 0.06 |
| Weighted Average | - | 50.1 | 29.8 | 56.2 | |
Disclaimer & Intended Uses
Inference Code
Use the provided instruction template for prompting, you can modify system prompt to "You are an AI assistant. You are a financial expert." for better performance on Finance and Economics instructions. Minimum 10GB VRAM and 15GB CPU RAM required.
import torch
from transformers import BitsAndBytesConfig, AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=getattr(torch, "bfloat16"),
bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b")
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
"tiiuae/falcon-7b", quantization_config=bnb_config, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, "mukaj/falcon-7b-hawkish-lora")
def PROMPT_TEMPLATE(system, human):
return f"### System: {system}\n### Human: {human}\n### Assistant:"
PROMPT = PROMPT_TEMPLATE("You are a helpful assistant.","Which bird is on the Albanian flag?")
inputs = tokenizer(PROMPT, return_tensors="pt").to(device)
outputs = model.generate(input_ids=inputs["input_ids"].to("cuda"),
attention_mask=inputs["attention_mask"],
max_new_tokens=10,
pad_token_id=tokenizer.eos_token_id,
do_sample=True, temperature=0.8, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
### The bird on the Albanian flag is the eagle.
Financial Domain Completion Examples
Equities
### System: You are an AI assistant. You are a financial expert and answer questions with respect to finance and economic domains.
### Human: Explain the types of standard ratios used in company analysis.
### Assistant: The types of standard ratios used in company analysis are:
1. Liquidity ratios: These ratios measure a company's ability to meet its short-term obligations.
2. Profitability ratios: These ratios measure a company's ability to generate profits.
3. Efficiency ratios: These ratios measure a company's ability to use its assets and liabilities effectively.
4. Market ratios: These ratios measure a company's ability to meet the expectations of its investors.
5. Financial leverage ratios: These ratios measure a company's ability to use debt to finance its assets.
6. Activity ratios: These ratios measure a company's ability to generate sales and profits.
7. Asset turnover ratios: These ratios measure a company's ability to generate sales and profits from its assets.
8. Investment ratios: These ratios measure a company's ability to generate profits from its investments.
9. Coverage ratios: These ratios measure a company's ability to meet its short-term obligations.
10. Du Pont ratios: These ratios measure a company's ability to generate profits from its assets and investments.
Fixed Income
### System: You are an AI assistant. You are a financial expert and answer questions with respect to finance and economic domains.
### Human: What is CMBS-level call protection, and is a Residual Tranche an example of one or is a Prepayment lockout an example? Or is that a individual loan level protection?
### Assistant: CMBS-level call protection is a level of protection that covers all individual loans in a CMBS transaction,
and ensures that any single prepayment on any class of assets covered by the loan level protection, will not result in a call of the remaining underlying assets.
A Residual Tranche is an example of CMBS-level call protection, which is more commonly employed in public deals, as opposed to private transactions.
Economics
### System: You are an AI assistant. You are a financial expert and answer questions with respect to finance and economic domains.
### Human: A new country is formed in Wakanda. They introduce a new currency. How does the international market value what it is worth?.
### Assistant: The value of such a novel economy could be difficult to determine. Assume a nominal return of 0.4%, constant to inflation based on the national nominal wage.
Assume nominal wages of both countries have a 200% price level (real price would be less). GDP of the novel sector will be about 5% (20% of the total GDP), assuming labor productivity is unchanged. National product will decrease.
With trade of a factor of the nominal price level, the novel GDP will be about 5%x20(50%). Nominal wage in the novel economy can be the same as for the main economy, resulting in the price level equal to (3/5)*(200/100) = 60/.
One of the challenges in valuing this asset will be deciding if the novel sector should be valued more as the capital market or the labor market. Assume a 30-year horizon.
Real wage of labor will be about 8% with national real interest rate at 5%. Assume that nominal returns and capital gains yield a 7/12 interest rate.
The present value of the factor-adjusted wage-based return would be: $1.4(7/12)×15(5).
It would be a risky asset yielding a return of about 4.6%. This is also equivalent to investing 0.4 + $1.4×(30) = $50 at risk in 30 years.
Of course, if the rate is higher, the value of the real wage is correspondingly higher. The return to holding labor is the wage rate, plus a risk premium equal to 0.5/4.4/3*(30).
The present value of expected cash flows and the intrinsic value of capital are assumed equal. In general, the expected discounted cash flows will be the market price, less expected inflation and real interest rates for those sectors.
The present value is therefore determined by multiplying the cash flows by an appropriate discount rate which can be estimated based by using historical inflation-adjusted returns,
and discounting cash flows at the domestic risk-adjusted risk premium and discounting future flows at the domestic real interest rate.
Real Estate
### System: You are an AI assistant. You are a financial expert and answer questions with respect to finance and economic domains.
### Human: Joshua Petersen is a real estate analyst that wants to make appropriate adjustments to a capitalization rate. He predicts that vacancy rates will increase and that the availability of credit will decrease. Based on these views, it is most likely that Petersen:
1) should increase the capitalization rate.
2) may need to either increase or decrease the capitalization rate because the two predictions have offsetting effects.
3) should decrease the capitalization rate.
### Assistant: B) To determine the impact of vacancy rate fluctuations and availability of credit the two views must be considered as opposed to simply adjusting due to increases in vacancies for vacant rates to go down
With these two views are the vacancy increases and decreases likely to result in a decline in available capital with credit getting more selective? To make a more robust conclusion, we'll use some of this input.
We will assume vacancy rates remain unchanged and assume available lines of credit are unchanged. In those two cases, vacancy increases can have a much greater effect leading to even higher and worse risks to asset value.
We're also assuming here that there are no issues with interest in rates, so higher vacancy rate and limited credit availability are really enough to put real assets assets out of consideration until conditions look better.
A decline in capital appreciation potential due to falling value of real estate, may require the investor to exit and switch to safer investments or wait out the conditions and have the potential of a huge capital gain when conditions improve.
From looking to real time statistics the conditions mentioned probably wouldn't be significant in most real contexts of what to capitalize on at real rates.
Real estate investment in certain large metros may see value drops, other investment categories will show some movement up while still some will look flat.
Here's where your analysis is missing, an important point not to skip that impacts whether all investors leave because of real estate volatility being high.
With vacancy rates and credit issues combined, it is true that assets would be viewed as poor assets to capitalize rates or real assets in general at that moment,
because so much money leaves market causing lack of market liquidity which makes investing an almost irrational thing at points.
You have a good understanding with this but need the market understanding and data side, then add it to a robust analysis of macro outlook.
Your conclusion is based on real market conditions and future conditions is uncertain so no hard conclusions should be drawn,
there can be multiple markets over same period, in your same industry, or across the whole industry that are going down for various reasons which you need to determine prior to assuming.
It gets more granular and looks deeper than numbers on a piece of paper for individual assets but instead it relates it with a market condition. Hope it helps.
Limitations & Further Work
Model is not fully evaluated, it is still work in progess, and is only trained on 0.2 Epochs.
- Repeat experiment with larger model.
- Filter data further.
- Data Mix Comparison
This is self explanatory, larger models have been shown to be better all-around, so a larger base model should help drastically.
Less is more, filter to highest quality samples that maintain highest diversity and compare results.
Different mixes of finance weightings in the data need to be tried to get best performance.
Compute
Hardware:
1 x RTX 4090
Training Software:
https://huggingface.co/docs/trl/main/en/sft_trainer
Max Seq Length: 2048
Learning Rate: 2e-4
Attributions & References
Base Model: https://huggingface.co/tiiuae/falcon-7b
Trained with QLoRA: https://arxiv.org/abs/2305.14314
OpenOrca Data: https://huggingface.co/datasets/Open-Orca/OpenOrca
Guanaco Data: https://huggingface.co/datasets/timdettmers/openassistant-guanaco
CodeParrot Self Instruct: https://huggingface.co/datasets/codeparrot/self-instruct-starcoder
BloombergGPT Paper: https://arxiv.org/pdf/2303.17564.pdf