|
--- |
|
language: |
|
- code |
|
license: llama2 |
|
pipeline_tag: text-generation |
|
tags: |
|
- facebook |
|
- meta |
|
- pytorch |
|
- llama |
|
- llama-2 |
|
- mlx |
|
extra_gated_heading: You need to share contact information with Meta to access this |
|
model |
|
extra_gated_prompt: "### LLAMA 2 COMMUNITY LICENSE AGREEMENT\n\"Agreement\" means\ |
|
\ the terms and conditions for use, reproduction, distribution and modification\ |
|
\ of the Llama Materials set forth herein. \"Documentation\" means the specifications,\ |
|
\ manuals and documentation accompanying Llama 2 distributed by Meta at https://ai.meta.com/resources/models-and-libraries/llama-downloads/.\ |
|
\ \"Licensee\" or \"you\" means you, or your employer or any other person or entity\ |
|
\ (if you are entering into this Agreement on such person or entity's behalf), of\ |
|
\ the age required under applicable laws, rules or regulations to provide legal\ |
|
\ consent and that has legal authority to bind your employer or such other person\ |
|
\ or entity if you are entering in this Agreement on their behalf. \"Llama 2\"\ |
|
\ means the foundational large language models and software and algorithms, including\ |
|
\ machine-learning model code, trained model weights, inference-enabling code, training-enabling\ |
|
\ code, fine-tuning enabling code and other elements of the foregoing distributed\ |
|
\ by Meta at ai.meta.com/resources/models-and-libraries/llama-downloads/. \"Llama\ |
|
\ Materials\" means, collectively, Meta's proprietary Llama 2 and documentation\ |
|
\ (and any portion thereof) made available under this Agreement. \"Meta\" or \"\ |
|
we\" means Meta Platforms Ireland Limited (if you are located in or, if you are\ |
|
\ an entity, your principal place of business is in the EEA or Switzerland) and\ |
|
\ Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland). \ |
|
\ By clicking \"I Accept\" below or by using or distributing any portion or element\ |
|
\ of the Llama Materials, you agree to be bound by this Agreement. 1. License Rights\ |
|
\ and Redistribution. a. Grant of Rights. You are granted a non-exclusive, worldwide,\ |
|
\ non- transferable and royalty-free limited license under Meta's intellectual property\ |
|
\ or other rights owned by Meta embodied in the Llama Materials to use, reproduce,\ |
|
\ distribute, copy, create derivative works of, and make modifications to the Llama\ |
|
\ Materials. \n \nb. Redistribution and Use. i. If you distribute or make\ |
|
\ the Llama Materials, or any derivative works thereof, available to a third party,\ |
|
\ you shall provide a copy of this Agreement to such third party. ii. If you\ |
|
\ receive Llama Materials, or any derivative works thereof, from a Licensee as\ |
|
\ part of an integrated end user product, then Section 2 of this Agreement will\ |
|
\ not apply to you. iii. You must retain in all copies of the Llama Materials that\ |
|
\ you distribute the following attribution notice within a \"Notice\" text file\ |
|
\ distributed as a part of such copies: \"Llama 2 is licensed under the LLAMA 2\ |
|
\ Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.\"\ |
|
\ iv. Your use of the Llama Materials must comply with applicable laws and regulations\ |
|
\ (including trade compliance laws and regulations) and adhere to the Acceptable\ |
|
\ Use Policy for the Llama Materials (available at https://ai.meta.com/llama/use-policy),\ |
|
\ which is hereby incorporated by reference into this Agreement. v. You will not\ |
|
\ use the Llama Materials or any output or results of the Llama Materials to improve\ |
|
\ any other large language model (excluding Llama 2 or derivative works thereof).\ |
|
\ 2. Additional Commercial Terms. If, on the Llama 2 version release date, the\ |
|
\ monthly active users of the products or services made available by or for Licensee,\ |
|
\ or Licensee's affiliates, is greater than 700 million monthly active users in\ |
|
\ the preceding calendar month, you must request a license from Meta, which Meta\ |
|
\ may grant to you in its sole discretion, and you are not authorized to exercise\ |
|
\ any of the rights under this Agreement unless or until Meta otherwise expressly\ |
|
\ grants you such rights. 3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE\ |
|
\ LAW, THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON\ |
|
\ AN \"AS IS\" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,\ |
|
\ INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY,\ |
|
\ OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING\ |
|
\ THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND ASSUME\ |
|
\ ANY RISKS ASSOCIATED WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND\ |
|
\ RESULTS. 4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE\ |
|
\ LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE,\ |
|
\ PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST\ |
|
\ PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE\ |
|
\ DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY\ |
|
\ OF ANY OF THE FOREGOING. 5. Intellectual Property. a. No trademark licenses are\ |
|
\ granted under this Agreement, and in connection with the Llama Materials, neither\ |
|
\ Meta nor Licensee may use any name or mark owned by or associated with the other\ |
|
\ or any of its affiliates, except as required for reasonable and customary use\ |
|
\ in describing and redistributing the Llama Materials. b. Subject to Meta's ownership\ |
|
\ of Llama Materials and derivatives made by or for Meta, with respect to any derivative\ |
|
\ works and modifications of the Llama Materials that are made by you, as between\ |
|
\ you and Meta, you are and will be the owner of such derivative works and modifications.\ |
|
\ c. If you institute litigation or other proceedings against Meta or any entity\ |
|
\ (including a cross-claim or counterclaim in a lawsuit) alleging that the Llama\ |
|
\ Materials or Llama 2 outputs or results, or any portion of any of the foregoing,\ |
|
\ constitutes infringement of intellectual property or other rights owned or licensable\ |
|
\ by you, then any licenses granted to you under this Agreement shall terminate\ |
|
\ as of the date such litigation or claim is filed or instituted. You will indemnify\ |
|
\ and hold harmless Meta from and against any claim by any third party arising\ |
|
\ out of or related to your use or distribution of the Llama Materials. 6. Term\ |
|
\ and Termination. The term of this Agreement will commence upon your acceptance\ |
|
\ of this Agreement or access to the Llama Materials and will continue in full\ |
|
\ force and effect until terminated in accordance with the terms and conditions\ |
|
\ herein. Meta may terminate this Agreement if you are in breach of any term or\ |
|
\ condition of this Agreement. Upon termination of this Agreement, you shall delete\ |
|
\ and cease use of the Llama Materials. Sections 3, 4 and 7 shall survive the \ |
|
\ termination of this Agreement. 7. Governing Law and Jurisdiction. This Agreement\ |
|
\ will be governed and construed under the laws of the State of California without\ |
|
\ regard to choice of law principles, and the UN Convention on Contracts for the\ |
|
\ International Sale of Goods does not apply to this Agreement. The courts of California\ |
|
\ shall have exclusive jurisdiction of any dispute arising out of this Agreement.\ |
|
\ USE POLICY ### Llama 2 Acceptable Use Policy Meta is committed to promoting safe\ |
|
\ and fair use of its tools and features, including Llama 2. If you access or use\ |
|
\ Llama 2, you agree to this Acceptable Use Policy (“Policy”). The most recent copy\ |
|
\ of this policy can be found at [ai.meta.com/llama/use-policy](http://ai.meta.com/llama/use-policy).\ |
|
\ #### Prohibited Uses We want everyone to use Llama 2 safely and responsibly. You\ |
|
\ agree you will not use, or allow others to use, Llama 2 to: 1. Violate the law\ |
|
\ or others’ rights, including to:\n 1. Engage in, promote, generate, contribute\ |
|
\ to, encourage, plan, incite, or further illegal or unlawful activity or content,\ |
|
\ such as: \n 1. Violence or terrorism \n 2. Exploitation or harm to children,\ |
|
\ including the solicitation, creation, acquisition, or dissemination of child exploitative\ |
|
\ content or failure to report Child Sexual Abuse Material\n 3. Human trafficking,\ |
|
\ exploitation, and sexual violence\n 4. The illegal distribution of information\ |
|
\ or materials to minors, including obscene materials, or failure to employ legally\ |
|
\ required age-gating in connection with such information or materials.\n 5.\ |
|
\ Sexual solicitation\n 6. Any other criminal activity\n 2. Engage in, promote,\ |
|
\ incite, or facilitate the harassment, abuse, threatening, or bullying of individuals\ |
|
\ or groups of individuals\n 3. Engage in, promote, incite, or facilitate discrimination\ |
|
\ or other unlawful or harmful conduct in the provision of employment, employment\ |
|
\ benefits, credit, housing, other economic benefits, or other essential goods and\ |
|
\ services\n 4. Engage in the unauthorized or unlicensed practice of any profession\ |
|
\ including, but not limited to, financial, legal, medical/health, or related professional\ |
|
\ practices \n 5. Collect, process, disclose, generate, or infer health, demographic,\ |
|
\ or other sensitive personal or private information about individuals without rights\ |
|
\ and consents required by applicable laws\n 6. Engage in or facilitate any action\ |
|
\ or generate any content that infringes, misappropriates, or otherwise violates\ |
|
\ any third-party rights, including the outputs or results of any products or services\ |
|
\ using the Llama 2 Materials\n 7. Create, generate, or facilitate the creation\ |
|
\ of malicious code, malware, computer viruses or do anything else that could disable,\ |
|
\ overburden, interfere with or impair the proper working, integrity, operation\ |
|
\ or appearance of a website or computer system \n2. Engage in, promote, incite,\ |
|
\ facilitate, or assist in the planning or development of activities that present\ |
|
\ a risk of death or bodily harm to individuals, including use of Llama 2 related\ |
|
\ to the following:\n 1. Military, warfare, nuclear industries or applications,\ |
|
\ espionage, use for materials or activities that are subject to the International\ |
|
\ Traffic Arms Regulations (ITAR) maintained by the United States Department of\ |
|
\ State\n 2. Guns and illegal weapons (including weapon development)\n 3. Illegal\ |
|
\ drugs and regulated/controlled substances\n 4. Operation of critical infrastructure,\ |
|
\ transportation technologies, or heavy machinery\n 5. Self-harm or harm to others,\ |
|
\ including suicide, cutting, and eating disorders\n 6. Any content intended to\ |
|
\ incite or promote violence, abuse, or any infliction of bodily harm to an individual\n\ |
|
3. Intentionally deceive or mislead others, including use of Llama 2 related to\ |
|
\ the following:\n 1. Generating, promoting, or furthering fraud or the creation\ |
|
\ or promotion of disinformation\n 2. Generating, promoting, or furthering defamatory\ |
|
\ content, including the creation of defamatory statements, images, or other content\n\ |
|
\ 3. Generating, promoting, or further distributing spam\n 4. Impersonating another\ |
|
\ individual without consent, authorization, or legal right\n 5. Representing that\ |
|
\ the use of Llama 2 or outputs are human-generated\n 6. Generating or facilitating\ |
|
\ false online engagement, including fake reviews and other means of fake online\ |
|
\ engagement \n 4. Fail to appropriately disclose to end users any known dangers\ |
|
\ of your AI system \nPlease report any violation of this Policy, software “bug,”\ |
|
\ or other problems that could lead to a violation of this Policy through one of\ |
|
\ the following means: * Reporting issues with the model: [github.com/facebookresearch/llama](http://github.com/facebookresearch/llama)\ |
|
\ * Reporting risky content generated by the model: [developers.facebook.com/llama_output_feedback](http://developers.facebook.com/llama_output_feedback)\ |
|
\ * Reporting bugs and security concerns: [facebook.com/whitehat/info](http://facebook.com/whitehat/info)\ |
|
\ * Reporting violations of the Acceptable Use Policy or unlicensed uses of Llama:\ |
|
\ [[email protected]](mailto:[email protected])" |
|
extra_gated_fields: |
|
First Name: text |
|
Last Name: text |
|
Date of birth: date_picker |
|
Country: country |
|
Affiliation: text |
|
geo: ip_location |
|
? By clicking Submit below I accept the terms of the license and acknowledge that |
|
the information I provide will be collected stored processed and shared in accordance |
|
with the Meta Privacy Policy |
|
: checkbox |
|
extra_gated_description: The information you provide will be collected, stored, processed |
|
and shared in accordance with the [Meta Privacy Policy](https://www.facebook.com/privacy/policy/). |
|
extra_gated_button_content: Submit |
|
--- |
|
|
|
# ipetrukha/CodeLlama-34b-Instruct-hf-4bit |
|
|
|
The Model [ipetrukha/CodeLlama-34b-Instruct-hf-4bit](https://huggingface.co/ipetrukha/CodeLlama-34b-Instruct-hf-4bit) was converted to MLX format from [meta-llama/CodeLlama-34b-Instruct-hf](https://huggingface.co/meta-llama/CodeLlama-34b-Instruct-hf) using mlx-lm version **0.16.1**. |
|
|
|
## Use with mlx |
|
|
|
```bash |
|
pip install mlx-lm |
|
``` |
|
|
|
```python |
|
from mlx_lm import load, generate |
|
|
|
model, tokenizer = load("ipetrukha/CodeLlama-34b-Instruct-hf-4bit") |
|
response = generate(model, tokenizer, prompt="hello", verbose=True) |
|
``` |
|
|
