metadata
library_name: peft
base_model: mistralai/Mistral-7B-Instruct-v0.2
Model Card for Model ID
This is a lora peft adapter for mistralai/Mistral-7B-Instruct-v0.2 fine tuned on a custom made synthetic dataset to give structured json response based on a given task and a pydantic json model in system prompt and further information in the user prompt.
Model Details
base_model: mistralai/Mistral-7B-Instruct-v0.2
Model Description
- Developed by: Aswanth
- Model type: Causal LM
- Finetuned from model [optional]:
mistralai/Mistral-7B-Instruct-v0.2
Downstream Use [optional]
Finetuned for generalised structured json output based on given pydantic object schema
Out-of-Scope Use
Not suitable for chat conversation
How to Get Started with the Model
Use the code below to get started with the model.
import json
import torch
from pprint import pprint
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
def generate_and_parse(model, tokenizer, system, user):
text = f"<s>[INST] <<SYS>>\n{system}\n<</SYS>>\n\n"
text += f"{user}[/INST]"
model_input = tokenizer(text, return_tensors="pt").to("cuda")
model.eval()
with torch.no_grad():
stop_token_id = tokenizer.convert_tokens_to_ids("</s>")
tokenizer.pad_token_id = 18610
tokenizer.padding_side = "right"
tokenizer.add_eos_token=False
tokenizer.add_bos_token=False
gen_config = model.generation_config
gen_config.max_length = 6220
gen_config.do_sample = False
gen_config.stop_token_id = stop_token_id
output = model.generate(**model_input, generation_config=gen_config)
decoded_output = [tokenizer.decode(token_id) for token_id in output]
response = decoded_output[0].split("[/INST]")[-1].strip()
response = response.replace("</s>", "").strip()
json_output = json.loads(response)
return(json_output)
###########################################
# Loading the base model and peft adapter #
###########################################
base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"
peft_model_name = "AswanthCManoj/azma-mistral-7b-v0.2-instruct-structured"
config = PeftConfig.from_pretrained(
peft_model_name
)
if load_in_4bit and not merge_model:
bnb_config = BitsAndBytesConfig(
load_in_4bit=True, # load model in 4-bit precision
bnb_4bit_quant_type="nf4", # pre-trained model should be quantized in 4-bit NF format
bnb_4bit_use_double_quant=True, # Using double quantization as mentioned in QLoRA paper
bnb_4bit_compute_dtype=torch.bfloat16, # During computation, pre-trained model should be loaded in BF16 format
)
model = AutoModelForCausalLM.from_pretrained(
base_model_name,
low_cpu_mem_usage=True,
quantization_config=bnb_config,
return_dict=True,
torch_dtype=torch.float16,
device_map="auto",
)
else:
model = AutoModelForCausalLM.from_pretrained(
base_model_name,
low_cpu_mem_usage=True,
return_dict=True,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(
model, peft_model_name
)
tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True, use_fast=False, add_eos_token=False)
tokenizer.pad_token_id = 18610
###################
# Model Inference #
###################
# EXAMPLE1:
system = """Task: Generate a suitable four line poem based on the provided conversation.
Output Schema:
{'properties': {'poem': {'description': 'The four line poem based on the given conversation', 'title': 'Poem', 'type': 'string'}}, 'required': ['poem'], 'title': 'ResponseModel', 'type': 'object'}
"""
user = """# Conversation:
---
User: You are an AI assistant. You will be given a task. You must generate a detailed and long answer.
@Jenni_starr i wish i could do the same! Describe the sentiment embodied by this tweet. Options: - negative - positive I think the answer is
Assistant: positive. This tweet represents a positive sentiment because the user, whose username is Jenni_starr, appears to have shared a personal accomplishment, thought, or experience that the person replying to the tweet wishes they could replicate or achieve as well. The exclamation mark at the end of the sentence also gives the statement an enthusiastic and lighthearted tone. The use of the word "wish" suggests a sense of admiration or desire to match whatever accomplishment, thought, or experience Jenni_starr has shared. By expressing the sentiment "I wish I could do the same!" the responder is acknowledging the admiration they have for Jenni_starr and implies that whatever was shared is a positive aspect in their standpoint. This admiration and acknowledgement help to create an empowering and supportive environment between the individuals involved in the tweet exchange. Moreover, this type of engagement on social media platforms such as Twitter is reflective of the positive relationships that can be built and nurtured online. People often share their achievements, ideas, and experiences to motivate and inspire others or to provide encouragement and support to their social circles. In conclusion, based on the statement "I wish I could do the same!", the sentiment embodied by this tweet can be described as positive. The responder expresses admiration for Jenni_starr's accomplishment, thought, or experience and conveys their desire to achieve something similar. This interaction demonstrates how social media platforms can foster supportive relationships and create a positive impact on individuals' emotions and personal experiences.
---"""
pprint(generate_and_parse(model, tokenizer, system, user))
'''{'poem': 'In tweets, we share our thoughts, so true,\n'
'A sentiment, positive or negative, to imbue,\n'
'"I wish I could do the same," we say,\n'
"Empathy and admiration, in a single tweet's array."}'''
# EXAMPLE-2 Resume Parsing:
system = """Task: Given a text content from resume, extract the required information along with analysis based on the specified schema.
Output Schema:
{'$defs': {'Address': {'properties': {'primary': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Primary address of the applicant', 'title': 'Primary'}, 'secondary': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Secondary address of the applicant', 'title': 'Secondary'}}, 'title': 'Address', 'type': 'object'}, 'Certification': {'properties': {'name': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Name of the certification', 'title': 'Name'}, 'date': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Date of certification', 'title': 'Date'}}, 'title': 'Certification', 'type': 'object'}, 'Education': {'properties': {'institution': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Name of the educational institution', 'title': 'Institution'}, 'course': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Name of the course', 'title': 'Course'}, 'year': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Year of completion or enrollment', 'title': 'Year'}}, 'title': 'Education', 'type': 'object'}, 'Email': {'properties': {'primary': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Primary email address of the applicant', 'title': 'Primary'}, 'secondary': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Secondary email address of the applicant', 'title': 'Secondary'}}, 'title': 'Email', 'type': 'object'}, 'Experience': {'properties': {'company': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Name of the company or organization', 'title': 'Company'}, 'year': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Year of employment or experience', 'title': 'Year'}}, 'title': 'Experience', 'type': 'object'}, 'Name': {'properties': {'full_name': {'description': 'Full name of the applicant', 'title': 'Full Name', 'type': 'string'}, 'first_name': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'First name of the applicant', 'title': 'First Name'}, 'middle_name': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Middle name of the applicant', 'title': 'Middle Name'}, 'last_name': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Last name of the applicant', 'title': 'Last Name'}}, 'required': ['full_name'], 'title': 'Name', 'type': 'object'}, 'PhoneNumber': {'properties': {'primary': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Primary phone number of the applicant', 'title': 'Primary'}, 'secondary': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Secondary phone number of the applicant', 'title': 'Secondary'}}, 'title': 'PhoneNumber', 'type': 'object'}, 'Reference': {'properties': {'name': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Name of the reference', 'title': 'Name'}, 'details': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Details or role of the reference', 'title': 'Details'}, 'phone_number': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Phone number of the reference', 'title': 'Phone Number'}, 'email': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None, 'description': 'Email address of the reference', 'title': 'Email'}}, 'title': 'Reference', 'type': 'object'}}, 'properties': {'name': {'allOf': [{'$ref': '#/$defs/Name'}], 'description': "Information about the applicant's name"}, 'email': {'anyOf': [{'$ref': '#/$defs/Email'}, {'type': 'null'}], 'default': None, 'description': "Information about the applicant's email addresses"}, 'number': {'anyOf': [{'$ref': '#/$defs/PhoneNumber'}, {'type': 'null'}], 'default': None, 'description': "Information about the applicant's phone numbers"}, 'address': {'anyOf': [{'$ref': '#/$defs/Address'}, {'type': 'null'}], 'default': None, 'description': "Information about the applicant's addresses"}, 'summary': {'description': 'An overall summary of the entire resume', 'title': 'Summary', 'type': 'string'}, 'job_post': {'description': 'Job post or desired position mentioned in the resume', 'title': 'Job Post', 'type': 'string'}, 'accomplishments': {'anyOf': [{'items': {'type': 'string'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Accomplishments mentioned in the resume, if any', 'title': 'Accomplishments'}, 'skills': {'anyOf': [{'items': {'type': 'string'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Skills mentioned in the resume', 'title': 'Skills'}, 'education': {'anyOf': [{'items': {'$ref': '#/$defs/Education'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Educational qualifications mentioned in the resume, if any', 'title': 'Education'}, 'experience': {'anyOf': [{'items': {'$ref': '#/$defs/Experience'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Work experience mentioned in the resume, if any', 'title': 'Experience'}, 'projects': {'anyOf': [{'items': {'type': 'string'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Projects mentioned in the resume, if any', 'title': 'Projects'}, 'achievements': {'anyOf': [{'items': {'type': 'string'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Achievements mentioned in the resume, if any', 'title': 'Achievements'}, 'certifications': {'anyOf': [{'items': {'$ref': '#/$defs/Certification'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'Certifications mentioned in the resume, if any', 'title': 'Certifications'}, 'references': {'anyOf': [{'items': {'$ref': '#/$defs/Reference'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'description': 'References mentioned in the resume, if any', 'title': 'References'}}, 'required': ['name', 'summary', 'job_post'], 'title': 'Resume', 'type': 'object'}
"""
user = """# Resume text content:
---
REFERENCE:
Jennifer Martinez Programmer, 6819 Fifth St, San Diego, Michigan
+628 769 101 1914 [email protected], Peter King Analyst, 4659 Washington St, Boston, Illinois
+38 652 346 1938 [email protected], Steve Anderson Technician, 6537 Third St, San Francisco, NV
+201 935 824 2502 [email protected],
DIRECTOR OF OPERATIONS Summary
A motivated professional seeking a position as an administrative assistant utilizing my 20 years of experience managing business operations and personnel to achieve organizational profitability. Dynamic personality with the interpersonal communication skills to engage and develop strong rapport with individuals at all levels. Excellent organizational and planning competencies utilized in providing full support to a department of 59 employees. Adapt worker, capable of efficiently maintaining departmental records in a multi-faceted organization to meet tight deadlines. Proficient in multiple computer applications with fast and accurate, typing skills and proven ability to plan, organize, direct, and execute while yielding optimum results. Well versed in Microsoft Office Suite.
Highlights
- Microsoft Office proficiency
- Excel spreadsheets
- Meticulous attention to detail
- Results-oriented
- Self-directed
- Time management
- Professional and mature
- Resourceful
- Strong problem solver
- Advanced MS Office Suite knowledge
- Strong interpersonal skills
- Dedicated team player
- Meeting planning
- Schedule management
Experience
Director of Operations
September 2007 to February 2012 Quantico Government Services ï¼ Quantico , Virginia USA
- Managed daily operations for 59 employees while managing a $2.5 million facility supporting 2.8K customers
- Oversaw execution, negotiation and implementation of $11.4 million annual contracts and $442K cook addendum contract
- Inventoried food stock levels and purchases which reduced operating costs by 18% in seven months
- Allocated essential core training for 32 personnel saving $61K in travel costs
- Balanced accounts receivables for 4 vendors and reconciled accounts payable
Fitness Operations Manager
July 2006 to September 2007 Aero Systems Inc. ï¼ Fairfax , VA
- Provided secretarial and administrative support to Services Director
- Allocated $90K in funding for new fitness equipment enhanced quality of life for 400K customers
- Accounted for $80K in resale funds, inventory, and daily bank deposits
- Developed plan of action and milestones for Operational Risk Management Program achieved goal of 90% in two weeks
- Networked with senior management, employees, and external agencies to correct 19 open work orders and 5 projects culminating in $2.8K in repairs and upgrades.
Administrative Assistant
February 2004 to July 2006 UAE Gold Trading DMCC ï¼ London , UK
- Provided administrative assistance to the Commander of the Air Logistics Center
- Prepared and maintained financial reports
- Received and maintained records of telephone calls, incoming and outgoing correspondence
- Liaised with protocol staff, external agencies, and visiting dignitary's support staff.
Training and Readiness Manager
August 2001 to February 2004 Horizon Aerospace Technologies ï¼ Rangoon , Yangon Region
- Developed training program and electronic reports of personnel to ensure proficient status of 290 tasks which improved stats to100% in two months.
- Provided invaluable, critical administrative support of active duty cases of fallen heroes.
- Prepared monthly reports capturing training status of 97 personnel for senior management and headquarters.
Education
Bachelors of Science : Healthcare Administration , 2014 Southwestern College ï¼ City , State GPA: Summa Cum Laude Graduate
Healthcare Administration Summa Cum Laude Graduate
Associate of Applied Science : Leadership and Management , 2010 U.S. Air Force, Air University Distance Learning
Training Specialized in Leadership Management, Managerial Communication, Total Quality Management, Management Theory, Management by Objective, Motivation, Counseling and Diversity
Associates of Applied Science : Fitness, Recreation, and Services Management , 2007 Community College of the Air Force Fitness, Recreation, and Services Management
Skills
Account Management, Accounts Receivables, Administrative Support, Subordinate Counseling, Computer Proficiency, Customer Service, Financial Report Reconcile, Funds Management, Inventory, Logistics Director, Risk Management, Secretarial Duties, Communication Skills, Total Quality Management, Executive Management Support, Filing, Grammar, Internet Research, Letters and Memos, Lotus Notes, Microsoft Excel, Microsoft Office Suite, Microsoft Outlook, Microsoft PowerPoint, Microsoft Word, Minute Taking, Multi-Task Management, Organizational Skills, Prioritization, Scheduling, Service Orientation, Speaking, Spreadsheets, Telephone Skills, Time Management, Vendor Management, Coordination, Documentation,
[ref1]: 002949d5-e09b-4273-83a0-b300fc906e75.001.png
[ref2]: 002949d5-e09b-4273-83a0-b300fc906e75.002.png
Madison Ann Green
[[email protected]]([email protected])
[[email protected]]([email protected])
+2669574888755
7285 Birch St, Charlotte PR
---"""
pprint(generate_and_parse(model, tokenizer, system, user))
'''{'education': [{'course': 'Healthcare Administration',
'institution': 'Southwestern College',
'year': '2014'},
{'course': 'Leadership and Management',
'institution': 'Air University Distance Learning',
'year': '2010'},
{'course': 'Fitness, Recreation, and Services Management',
'institution': 'Community College of the Air Force',
'year': '2007'}],
'email': ['[email protected]', '[email protected]'],
'experience': [{'company': 'Quantico Government Services',
'year': 'September 2007 to February 2012'},
{'company': 'Aero Systems Inc.',
'year': 'July 2006 to September 2007'},
{'company': 'UAE Gold Trading DMCC',
'year': 'February 2004 to July 2006'},
{'company': 'Horizon Aerospace Technologies',
'year': 'August 2001 to February 2004'}],
'job_post': 'Administrative Assistant',
'name': {'full_name': 'Madison Ann Green'},
'phone_number': '+2669574888755',
'skills': ['Account Management',
'Accounts Receivables',
'Administrative Support',
'Subordinate Counseling',
'Computer Proficiency',
'Customer Service',
'Financial Report Reconcile',
'Funds Management',
'Inventory',
'Logistics Director',
'Risk Management',
'Secretarial Duties',
'Communication Skills',
'Total Quality Management',
'Executive Management Support',
'Filing',
'Grammar',
'Internet Research',
'Letters and Memos',
'Lotus Notes',
'Microsoft Excel',
'Microsoft Office Suite',
'Microsoft Outlook',
'Microsoft PowerPoint',
'Microsoft Word',
'Minute Taking',
'Multi-Task Management',
'Organizational Skills',
'Prioritization',
'Scheduling',
'Service Orientation',
'Speaking',
'Spreadsheets',
'Telephone Skills',
'Time Management',
'Vendor Management',
'Coordination',
'Documentation'],
'summary': 'A motivated professional seeking a position as an administrative '
'assistant utilizing my 20 years of experience managing business '
'operations and personnel to achieve organizational profitability. '
'Dynamic personality with the interpersonal communication skills '
'to engage and develop strong rapport with individuals at all '
'levels. Excellent organizational and planning competencies '
'utilized in providing full support to a department of 59 '
'employees. Adapt worker, capable of efficiently maintaining '
'departmental records in a multi-faceted organization to meet '
'tight deadlines. Proficient in multiple computer applications '
'with fast and accurate, typing skills and proven ability to plan, '
'organize, direct, and execute while yielding optimum results. '
'Well versed in Microsoft Office Suite.'}
'''
Training Data
The training data included tasks such as:
- Future query prediction based on a given conversation history.
- Rephrasing of follow-up query in context with the conversation history.
- Email template generation based based on guidance.
- Title generation based on a conversation.
- Resume structured data extraction and analysis provided an unstructured resume text data.
- Multi-query generation based on a user query suitable for semantic vector search to search from multiple aspects.
- Questions prediction based on a provided document text chunk.
Training Procedure
Training took 3hr on a single Tesla T4 15GB - Colab notebook
Training Hyperparameters
-
Framework versions
- PEFT 0.7.2.dev0