user129540: I'm reading Programming Bitcoin. And in Chapter13, I am confused with how the witness part is serialized and parsed. The auther gives the following method to parse it:
def parse_segwit(cls, s, testnet=False):
    version = little_endian_to_int(s.read(4))
    marker = s.read(2)
    if marker != b'\x00\x01':  
        raise RuntimeError('Not a segwit transaction {}'.format(marker))
    num_inputs = read_varint(s)
    inputs = []
    for _ in range(num_inputs):
        inputs.append(TxIn.parse(s))
    num_outputs = read_varint(s)
    outputs = []
    for _ in range(num_outputs):
        outputs.append(TxOut.parse(s))
    for tx_in in inputs:  
        num_items = read_varint(s)
        items = []
        for _ in range(num_items):
            item_len = read_varint(s)
            if item_len == 0:        # where I can't understand 
                items.append(0)      #
            else:
                items.append(s.read(item_len))
        tx_in.witness = items
    locktime = little_endian_to_int(s.read(4))
    return cls(version, inputs, outputs, locktime, 
               testnet=testnet, segwit=True)

And the following code to serialize it:
def serialize_segwit(self):
    result = int_to_little_endian(self.version, 4)
    result += b'\x00\x01'  
    result += encode_varint(len(self.tx_ins))
    for tx_in in self.tx_ins:
        result += tx_in.serialize()
    result += encode_varint(len(self.tx_outs))
    for tx_out in self.tx_outs:
        result += tx_out.serialize()
    for tx_in in self.tx_ins:  
        result += int_to_little_endian(len(tx_in.witness), 1)
        for item in tx_in.witness:
            if type(item) == int:
                result += int_to_little_endian(item, 1)
            else:
                result += encode_varint(len(item)) + item
    result += int_to_little_endian(self.locktime, 4)
    return result

What I can't understand are the last for loops in both methods. In my understanding, these are where segwit part get serialized and deserialized, and these codes should be in one-to-one correspondence. As they do in the first half part. But they don't match in the for loops of the last half part. I mean:

For "num_items = read_varint(s)", there should be a "result += encode_varint(len(tx_in.witness))".
For "item_len = read_varint(s)", there should be a "result += encode_varint(len(item))".
And for "if item_len == 0:", I don't know what this if do and where the corresponding serialization logic is.

So is anything wrong with my understanding of the segwit part logic?

user208: 
For "num_items = read_varint(s)", there should be a "result += encode_varint(len(tx_in.witness))".

The corresponding expression seems to be the result += int_to_little_endian(len(tx_in.witness), 1), though that's WRONG, as it only works up to length of 252 witness stack elements (for more than 252 elements, varint encoding is more than a single byte encoding). Of course, such long witness stacks are pretty rare, so this probably works almost always in practice.

For "item_len = read_varint(s)", there should be a "result += encode_varint(len(item))".

The corresponding expression seems to be the first part of result += encode_varint(len(item)) + item, which also serializes the item iteself.

And for "if item_len == 0:", I don't know what this if do and where the corresponding serialization logic is.

It appears that this code supports two ways of encoding the witness stack items: they can either be integers or byte arrays. The Bitcoin script language only has one data type (byte arrays), but there are a number of opcodes that interpret these byte arrays as numbers. The encoding for the number 0 is the empty byte array.
What this code does:

On encoding:

The number 0 gets serialized as the empty array (result += int_to_little_endian(item, 1)Th). That's correct.
Other integers get serialized as 1-byte encodings of the respective integer. This is WRONG. For small integers, the correct encoding is two bytes long.
Byte arrays are serialized as varlen encoding of their length, followed by the byte array itself. This is correct.


On decoding:

An encoding of the empty byte array is converted to the number 0.
Everything else is left as a byte array.



Overall, this looks like clumsily written that nobody should use in production. It probably works within a few simple cases (never more than 252 stack elements, and no integers other than 0 being encoded), but wouldn't pass rigorous testing.

129540: <p>I'm reading <a href="https://github.com/jimmysong/programmingbitcoin" rel="nofollow noreferrer">Programming Bitcoin</a>. And in Chapter13, I am confused with how the witness part is serialized and parsed. The auther gives the following method to parse it:</p>
<pre><code>def parse_segwit(cls, s, testnet=False):
    version = little_endian_to_int(s.read(4))
    marker = s.read(2)
    if marker != b'\x00\x01':  
        raise RuntimeError('Not a segwit transaction {}'.format(marker))
    num_inputs = read_varint(s)
    inputs = []
    for _ in range(num_inputs):
        inputs.append(TxIn.parse(s))
    num_outputs = read_varint(s)
    outputs = []
    for _ in range(num_outputs):
        outputs.append(TxOut.parse(s))
    for tx_in in inputs:  
        num_items = read_varint(s)
        items = []
        for _ in range(num_items):
            item_len = read_varint(s)
            if item_len == 0:        # where I can't understand 
                items.append(0)      #
            else:
                items.append(s.read(item_len))
        tx_in.witness = items
    locktime = little_endian_to_int(s.read(4))
    return cls(version, inputs, outputs, locktime, 
               testnet=testnet, segwit=True)
</code></pre>
<p>And the following code to serialize it:</p>
<pre><code>def serialize_segwit(self):
    result = int_to_little_endian(self.version, 4)
    result += b'\x00\x01'  
    result += encode_varint(len(self.tx_ins))
    for tx_in in self.tx_ins:
        result += tx_in.serialize()
    result += encode_varint(len(self.tx_outs))
    for tx_out in self.tx_outs:
        result += tx_out.serialize()
    for tx_in in self.tx_ins:  
        result += int_to_little_endian(len(tx_in.witness), 1)
        for item in tx_in.witness:
            if type(item) == int:
                result += int_to_little_endian(item, 1)
            else:
                result += encode_varint(len(item)) + item
    result += int_to_little_endian(self.locktime, 4)
    return result
</code></pre>
<p>What I can't understand are the last for loops in both methods. In my understanding, these are where segwit part get serialized and deserialized, and these codes should be in one-to-one correspondence. As they do in the first half part. But they don't match in the for loops of the last half part. I mean:</p>
<ul>
<li>For &quot;num_items = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(tx_in.witness))&quot;.</li>
<li>For &quot;item_len = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(item))&quot;.</li>
<li>And for &quot;if item_len == 0:&quot;, I don't know what this if do and where the corresponding serialization logic is.</li>
</ul>
<p>So is anything wrong with my understanding of the segwit part logic?</p>
 208: <blockquote>
<p>For &quot;num_items = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(tx_in.witness))&quot;.</p>
</blockquote>
<p>The corresponding expression seems to be the <code>result += int_to_little_endian(len(tx_in.witness), 1)</code>, though that's <strong>WRONG</strong>, as it only works up to length of 252 witness stack elements (for more than 252 elements, varint encoding is more than a single byte encoding). Of course, such long witness stacks are pretty rare, so this probably works almost always in practice.</p>
<blockquote>
<p>For &quot;item_len = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(item))&quot;.</p>
</blockquote>
<p>The corresponding expression seems to be the first part of <code>result += encode_varint(len(item)) + item</code>, which also serializes the item iteself.</p>
<blockquote>
<p>And for &quot;if item_len == 0:&quot;, I don't know what this if do and where the corresponding serialization logic is.</p>
</blockquote>
<p>It appears that this code supports two ways of encoding the witness stack items: they can either be integers or byte arrays. The Bitcoin script language only has one data type (byte arrays), but there are a number of opcodes that interpret these byte arrays as numbers. The encoding for the number 0 is the empty byte array.</p>
<p>What this code does:</p>
<ul>
<li>On encoding:
<ul>
<li>The number 0 gets serialized as the empty array (<code>result += int_to_little_endian(item, 1)</code>Th). That's correct.</li>
<li>Other integers get serialized as 1-byte encodings of the respective integer. This is <strong>WRONG</strong>. For small integers, the correct encoding is two bytes long.</li>
<li>Byte arrays are serialized as varlen encoding of their length, followed by the byte array itself. This is correct.</li>
</ul>
</li>
<li>On decoding:
<ul>
<li>An encoding of the empty byte array is converted to the number 0.</li>
<li>Everything else is left as a byte array.</li>
</ul>
</li>
</ul>
<p>Overall, this looks like clumsily written that nobody should use in production. It probably works within a few simple cases (never more than 252 stack elements, and no integers other than 0 being encoded), but wouldn't pass rigorous testing.</p>
