<commit_before>Block Implementation Specification

Copyright 2008-2009 Apple, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

0. History

2008/7/14  - created
2008/8/21  - revised, C++
2008/9/24  - add NULL isa field to __block storage
2008/10/1  - revise block layout to use a static descriptor structure
2008/10/6  - revise block layout to use an unsigned long int flags
2008/10/28 - specify use of _Block_object_assign/dispose for all "Object" types in helper functions
2008/10/30 - revise new layout to have invoke function in same place
2008/10/30 - add __weak support

This document describes the Apple ABI implementation specification of Blocks.

1. High Level

A Block consists of a structure of the following form:

struct Block_literal_1 {
    void *isa; // initialized to &_NSConcreteStackBlock or &_NSConcreteGlobalBlock
    int flags;
    int reserved; 
    void (*invoke)(void *, ...);
    struct Block_descriptor_1 {
	unsigned long int reserved;	// NULL
    	unsigned long int size;  // sizeof(struct Block_literal_1)
	// optional helper functions
    	void (*copy_helper)(void *dst, void *src);
    	void (*dispose_helper)(void *src); 
    } *descriptor;
    // imported variables
};

The following flags bits are used by the compiler:

enum {
    BLOCK_HAS_COPY_DISPOSE =  (1 << 25),
    BLOCK_HAS_CTOR =          (1 << 26), // helpers have C++ code
    BLOCK_IS_GLOBAL =         (1 << 28),
    BLOCK_HAS_DESCRIPTOR =    (1 << 29), // interim until complete world build is accomplished
};

Block literals may occur within functions where the structure is created in stack local memory.  They may also appear as initialization expressions for Block variables of global or static local variables.

When a Block literal expression is evaluated the stack based structure is initialized as follows:

1) static descriptor structure is declared and initialized as follows:
1a) the invoke function pointer is set to a function that takes the Block structure as its first argument and the rest of the arguments (if any) to the Block and executes the Block compound statement.
1b) the size field is set to the size of the following Block literal structure.
1c) the copy_helper and dispose_helper function pointers are set to respective helper functions if they are required by the Block literal
2) a stack (or global) Block literal data structure is created and initialized as follows:
2a) the isa field is set to the address of the external _NSConcreteStackBlock, which is a block of uninitialized memory supplied in libSystem, or _NSConcreteGlobalBlock if this is a static or file level block literal.
2) The flags field is set to zero unless there are variables imported into the block that need helper functions for program level Block_copy() and Block_release() operations, in which case the (1<<25) flags bit is set.

As an example, the Block literal expression
   ^ { printf("hello world\n"); }
would cause to be created on a 32-bit system:

struct __block_literal_1 {
    void *isa;
    int flags;
    int reserved; 
    void (*invoke)(struct __block_literal_1 *);
    struct __block_descriptor_1 *descriptor;
};

void __block_invoke_1(struct __block_literal_1 *_block) {
    printf("hello world\n");<commit_msg>rev existing doc to account for recent ABI changes<commit_after>Block Implementation Specification

Copyright 2008-2009 Apple, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

0. History

2008/7/14  - created
2008/8/21  - revised, C++
2008/9/24  - add NULL isa field to __block storage
2008/10/1  - revise block layout to use a static descriptor structure
2008/10/6  - revise block layout to use an unsigned long int flags
2008/10/28 - specify use of _Block_object_assign/dispose for all "Object" types in helper functions
2008/10/30 - revise new layout to have invoke function in same place
2008/10/30 - add __weak support

2010/3/16  - rev for stret return, signature field

This document describes the Apple ABI implementation specification of Blocks.

The first shipping version of this ABI is found in Mac OS X 10.6, and shall be referred to as 10.6.ABI. Proposals have been made to enhance the blocks compiler and runtime, and these changes would form a new ABI called post-10.6.ABI if these changes were ever shipped in a product, which is purely speculative.

Since the Apple ABI references symbols from other elements of the system, any attempt to use this ABI on systems prior to SnowLeopard is undefined.

1. High Level

The ABI of blocks consist of their layout and the runtime functions required by the compiler.
A Block consists of a structure of the following form:

struct Block_literal_1 {
    void *isa; // initialized to &_NSConcreteStackBlock or &_NSConcreteGlobalBlock
    int flags;
    int reserved; 
    void (*invoke)(void *, ...);
    struct Block_descriptor_1 {
	unsigned long int reserved;	// NULL
    	unsigned long int size;         // sizeof(struct Block_literal_1)
	// optional helper functions
    	void (*copy_helper)(void *dst, void *src);     // IFF (1<<25)
    	void (*dispose_helper)(void *src);             // IFF (1<<25)
        // required post 10.6.ABI
        const char *signature;                         // IFF (1<<30)
    } *descriptor;
    // imported variables
};

The following flags bits are in use thusly for a possible post.10.6.ABI:

enum {
    BLOCK_HAS_COPY_DISPOSE =  (1 << 25),
    BLOCK_HAS_CTOR =          (1 << 26), // helpers have C++ code
    BLOCK_IS_GLOBAL =         (1 << 28),
    BLOCK_HAS_STRET =         (1 << 29),
    BLOCK_HAS_SIGNATURE =     (1 << 30), 
};

In 10.6.ABI the (1<<29) was unconditionally set and ignored by the runtime - it was a transitional marker that did not get deleted after the transition. This bit is now paired with (1<<30), and represented as the pair (3<<30), for the following combinations of valid bit settings, and their meanings.

switch (flags & (3<<29)) {
  case (0<<29):  <unused>    , error
  case (1<<29):      10.6.ABI, no signature field available
  case (2<<29): post-10.6.ABI, regular calling convention, presence of signature field
  case (3<<29): post-10.6.ABI, street calling convention, presence of signature field,
}

The following discussions are presented as 10.6.ABI otherwise.

Block literals may occur within functions where the structure is created in stack local memory.  They may also appear as initialization expressions for Block variables of global or static local variables.

When a Block literal expression is evaluated the stack based structure is initialized as follows:

1) static descriptor structure is declared and initialized as follows:
1a) the invoke function pointer is set to a function that takes the Block structure as its first argument and the rest of the arguments (if any) to the Block and executes the Block compound statement.
1b) the size field is set to the size of the following Block literal structure.
1c) the copy_helper and dispose_helper function pointers are set to respective helper functions if they are required by the Block literal
2) a stack (or global) Block literal data structure is created and initialized as follows:
2a) the isa field is set to the address of the external _NSConcreteStackBlock, which is a block of uninitialized memory supplied in libSystem, or _NSConcreteGlobalBlock if this is a static or file level block literal.
2) The flags field is set to zero unless there are variables imported into the block that need helper functions for program level Block_copy() and Block_release() operations, in which case the (1<<25) flags bit is set.


As an example, the Block literal expression
   ^ { printf("hello world\n"); }
would cause to be created on a 32-bit system:

struct __block_literal_1 {
    void *isa;
    int flags;
    int reserved; 
    void (*invoke)(struct __block_literal_1 *);
    struct __block_descriptor_1 *descriptor;
};

void __block_invoke_1(struct __block_literal_1 *_block) {
    printf("hello world\n");
{'fraction_non_alphanumeric': 0.058419243986254296, 'fraction_numerical': 0.02779462300384071, 'mean_word_length': 3.9574148296593186, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3368622', 'n_tokens_mistral': 2729, 'n_tokens_neox': 2512, 'n_words': 1376}