Safetensors
qwen2
linqq9 commited on
Commit
9d00b78
1 Parent(s): df6751c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -6
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- license: apache-2.0
3
  datasets:
4
  - Salesforce/xlam-function-calling-60k
5
  base_model: Qwen/Qwen2-7B-Instruct
@@ -7,12 +7,13 @@ base_model: Qwen/Qwen2-7B-Instruct
7
  # Hammer-7b Function Calling Model
8
 
9
  ## Introduction
10
- In the realm of evaluating other function calling models, several critical issues have come to light.
11
- - The fine-tuned model shows an over-reliance on function names and parameter names when doing function calling. Function names and parameter names frequently encapsulate semantic information as they are commonly constructed through combinations or abbreviations of related terms. Nevertheless, this approach diverges from the modus operandi of normal individuals when engaging in function calls. Ordinary people tend to place greater emphasis on the function description and parameter description when making such calls.
12
- - For tools that bear resemblance to those encountered during training, the model might rely on its previously trained knowledge and consequently make incorrect function calls, particularly when it comes to default parameter values.
13
 
14
- To tackle these issues head-on, we fine-tuned a better function calling model based on the *Qwen2-7B-instruct*.
15
- The methods and processes implemented during training are detailed below:
 
 
 
16
 
17
  1. **Data Extraction and Preparation**:
18
  We extracted 7.5k sample data from *Salesforce/xlam-function-calling-60k* and removed the target tools from the candidate toolset to generate irrelevant data samples. This data was mixed with 60k XLAM data samples for training.
@@ -40,6 +41,14 @@ The model is capable of handling various function calling scenarios, including:
40
  - Multiple Parallel Function Calling
41
  - Irrelevance Detection
42
 
 
 
 
 
 
 
 
 
43
  ## Upcoming Developments
44
 
45
  We are actively working on preparing smaller models derived from this architecture, which will be open-sourced soon.
 
1
  ---
2
+ license: cc-by-4.0
3
  datasets:
4
  - Salesforce/xlam-function-calling-60k
5
  base_model: Qwen/Qwen2-7B-Instruct
 
7
  # Hammer-7b Function Calling Model
8
 
9
  ## Introduction
10
+ Function calling is a pivotal capability for large language models, enabling them to interact with software systems and databases in a dynamic and efficient manner. However, during the evaluation of the function calling capabilities of open-source large language models, several critical issues have been identified that warrant attention and refinement.
 
 
11
 
12
+ - **1**: One paramount issue identified is the marked propensity of fine-tuned language models to rely extensively on the lexical cues provided by function and parameter names during the invocation process. These names often act as semantic beacons, frequently derived from the concatenation or abbreviation of related concepts. Although such nomenclature serves useful in encapsulating semantic information, this reliance presents a stark divergence from the cognitive processes employed by human users during function calls. Humans, in contrast, typically prioritize the semantic richness of function and parameter descriptions, which encapsulate a more holistic understanding of the intended operations.
13
+ - **2**: Furthermore, for tools resembling those encountered during training, models tend to rely on previously ingrained knowledge, potentially resulting in erroneous function calls—particularly when dealing with default parameter values.
14
+
15
+
16
+ Addressing these multifaceted issues necessitates a refined and sophisticated approach to model training and optimization. To this end, we have meticulously developed an advanced function calling model through the fine-tuning of the *Qwen2-7B-instruct*. The ensuing sections provide a comprehensive overview of the methods and processes implemented during the training phase to mitigate these issues effectively:
17
 
18
  1. **Data Extraction and Preparation**:
19
  We extracted 7.5k sample data from *Salesforce/xlam-function-calling-60k* and removed the target tools from the candidate toolset to generate irrelevant data samples. This data was mixed with 60k XLAM data samples for training.
 
41
  - Multiple Parallel Function Calling
42
  - Irrelevance Detection
43
 
44
+
45
+ ## Performance
46
+
47
+ 1. First, we evaluate our model on the Berkeley Function-Calling Leaderboard (BFCL), and the performance is as follows:
48
+
49
+
50
+
51
+
52
  ## Upcoming Developments
53
 
54
  We are actively working on preparing smaller models derived from this architecture, which will be open-sourced soon.