user87716: I do realize that in bitcoin when we return the signature result for signing a message the first byte includes information for recovering the public key required for verification using a formula like this: 
27 + (4 if comp. 0 if not) + (0<=num<=3)

What I am struggling with is how do we determine that num number?
According to section 4.1.6 of SEC:1 for recovering public keys:

x = r + jn
Calculate Q based on R if invalid move to next
Calculate Q based on -R

Since j (cofactor of secp256k1) is 0 and 1 and there are two values for R. So there should be 4 possible public keys. Is num the number of public keys that we rejected before reaching the correct one? Because that is the only explanation that I can come up with based on comparisons I have made between signatures generated using Electrum and public key recovery I have done myself using SEC1.
Bonus question: Why was 27 chosen?

user6441: This will be less of a theoretical answer and more of practical guidance; while developing Secp256k1 or Bitcoin libraries, I always found the following table handy:




y-parity
x-order
compression
recovery id
v




even
less than n
false
0
27


odd
less than n
false
1
28


even
more than n
false
2
29


odd
more than n
false
3
30


even
less than n
true
0
31


odd
less than n
true
1
32


even
more than n
true
2
33


odd
more than n
true
3
34




let me explain the columns:

y-parity or often just parity describes whether the y-coordinate of the public key point R(x, y) is even. this is important because in most cases, the signature only contains r = R.x and not R.y. This speeds up signature recovery. (Secp256k1)
x-order is a super rare edge-case where a point's x-coordinate is actually above the order of the curve n. In 99.99999999% of the cases, this is false. (Secp256k1)
compression is just an indicator whether you want a compressed or an uncompressed public key. (Bitcoin)
recovery_id just indicates which of the four potential point's y-coordinates is to be used. (Secp256k1)
v is how we encode this value. For uncompressed keys we use v = recovery_id + 27 and for compressed v = recovery_id + 31. (Bitcoin)

If you want to understand how this works for Ethereum, you would have to read EIP-155 which also introduces an additional variable of chain_id for replay protection: v = 2 * chain_id + 35 + recovery_id. (Ethereum)

What I am struggling with is how do we determine that num number?

That's the recovery ID from the table above.

user87716: FWIW this is what I have found regarding this question. Basically there are two ways of finding what libraries call "recid" or recovery ID. Most of them use the first method but there is another way:
Method 1:

Only requires r and s so it can be performed by anyone as long as you have the signature and the "message" that was signed.
It is so much slower because it has 3 scalar multiplication and based on cofactor of the elliptic curve used the whole operation may be repeated 2*(h+1) times, which is up to 4 times for bitcoin with secp256k1 curve (99% of the time it is 1 or 2 times).

The steps are explained in section 4.1.6 of SEC:1, I won't repeat them here.
To find recid you check the calculated public key with the given public key or hash of it and report number of rejected public keys as recid. For example if x=r+(0*order) and Q was used then recid=0 and if -Q was used then recid=1 and so on.
method 2:

It requires full r or R=(xR, yR) so it can only be done while signing (requires having the private key).
It is much faster because there is no additional calculations. It just needs value checks which are fast.
This is usually referred to as v in libraries

byte v = if(R.X > curve.N) then 2 else 0) | (if R.Y.IsEven then 0 else 1);
Basically it is check to see if xR is bigger than curve order (N) and whether yR is even or odd. Something that should not be forgotten is a "flip" that is performed based on s and whether s was used in the signature or -s
if s > curve.N/2 then v^=1 else do nothing
** Note that in the end you need to calculate the following:
recid = 27 + v + (if compressed 4 else 0)

87716: <p>I do realize that in bitcoin when we return the signature result for signing a message the first byte includes information for recovering the public key required for verification using a formula like this: </p>

<pre><code>27 + (4 if comp. 0 if not) + (0&lt;=num&lt;=3)
</code></pre>

<p>What I am struggling with is how do we determine that <code>num</code> number?
According to section 4.1.6 of <a href="http://www.secg.org/sec1-v2.pdf" rel="nofollow noreferrer">SEC:1</a> for recovering public keys:</p>

<ol>
<li><code>x = r + jn</code></li>
<li>Calculate <code>Q</code> based on <code>R</code> if invalid move to next</li>
<li>Calculate <code>Q</code> based on <code>-R</code></li>
</ol>

<p>Since j (cofactor of secp256k1) is 0 and 1 and there are two values for R. So there should be 4 possible public keys. Is <code>num</code> the number of public keys that we rejected before reaching the correct one? Because that is the only explanation that I can come up with based on comparisons I have made between signatures generated using Electrum and public key recovery I have done myself using SEC1.</p>

<p><strong>Bonus question</strong>: Why was 27 chosen?</p>
 6441: <p>This will be less of a theoretical answer and more of practical guidance; while developing Secp256k1 or Bitcoin libraries, I always found the following table handy:</p>
<div class="s-table-container">
<table class="s-table">
<thead>
<tr>
<th>y-parity</th>
<th>x-order</th>
<th>compression</th>
<th>recovery id</th>
<th><code>v</code></th>
</tr>
</thead>
<tbody>
<tr>
<td>even</td>
<td>less than <code>n</code></td>
<td>false</td>
<td>0</td>
<td>27</td>
</tr>
<tr>
<td><em>odd</em></td>
<td>less than <code>n</code></td>
<td>false</td>
<td>1</td>
<td>28</td>
</tr>
<tr>
<td>even</td>
<td><em>more</em> than <code>n</code></td>
<td>false</td>
<td>2</td>
<td>29</td>
</tr>
<tr>
<td><em>odd</em></td>
<td><em>more</em> than <code>n</code></td>
<td>false</td>
<td>3</td>
<td>30</td>
</tr>
<tr>
<td>even</td>
<td>less than <code>n</code></td>
<td><em>true</em></td>
<td>0</td>
<td>31</td>
</tr>
<tr>
<td><em>odd</em></td>
<td>less than <code>n</code></td>
<td><em>true</em></td>
<td>1</td>
<td>32</td>
</tr>
<tr>
<td>even</td>
<td><em>more</em> than <code>n</code></td>
<td><em>true</em></td>
<td>2</td>
<td>33</td>
</tr>
<tr>
<td><em>odd</em></td>
<td><em>more</em> than <code>n</code></td>
<td><em>true</em></td>
<td>3</td>
<td>34</td>
</tr>
</tbody>
</table>
</div>
<p>let me explain the columns:</p>
<ul>
<li><code>y-parity</code> or often just <code>parity</code> describes whether the <code>y</code>-coordinate of the public key point <code>R(x, y)</code> is even. this is important because in most cases, the signature only contains <code>r = R.x</code> and not <code>R.y</code>. This speeds up signature recovery. (Secp256k1)</li>
<li><code>x-order</code> is a super rare edge-case where a point's <code>x</code>-coordinate is actually above the order of the curve <code>n</code>. In 99.99999999% of the cases, this is false. (Secp256k1)</li>
<li><code>compression</code> is just an indicator whether you want a compressed or an uncompressed public key. (Bitcoin)</li>
<li><code>recovery_id</code> just indicates which of the four potential point's <code>y</code>-coordinates is to be used. (Secp256k1)</li>
<li><code>v</code> is how we encode this value. For uncompressed keys we use <code>v = recovery_id + 27</code> and for compressed <code>v = recovery_id + 31</code>. (Bitcoin)</li>
</ul>
<p>If you want to understand how this works for Ethereum, you would have to read <a href="https://github.com/ethereum/EIPs/blob/master/EIPS/eip-155.md" rel="nofollow noreferrer">EIP-155</a> which also introduces an additional variable of <code>chain_id</code> for replay protection: <code>v = 2 * chain_id + 35 + recovery_id</code>. (Ethereum)</p>
<blockquote>
<p>What I am struggling with is how do we determine that <code>num</code> number?</p>
</blockquote>
<p>That's the recovery ID from the table above.</p>
 87716: <p>FWIW this is what I have found regarding this question. Basically there are two ways of finding what libraries call &quot;recid&quot; or recovery ID. Most of them use the first method but there is another way:</p>
<h1>Method 1:</h1>
<ul>
<li>Only requires <code>r</code> and <code>s</code> so it can be performed by anyone as long as you have the signature and the &quot;message&quot; that was signed.</li>
<li>It is so much slower because it has 3 scalar multiplication and based on cofactor of the elliptic curve used the whole operation may be repeated 2*(h+1) times, which is up to 4 times for bitcoin with secp256k1 curve (99% of the time it is 1 or 2 times).</li>
</ul>
<p>The steps are explained in section 4.1.6 of <a href="http://www.secg.org/sec1-v2.pdf" rel="nofollow noreferrer">SEC:1</a>, I won't repeat them here.</p>
<p>To find <code>recid</code> you check the calculated public key with the given public key or hash of it and report number of rejected public keys as <code>recid</code>. For example if <code>x=r+(0*order)</code> and <code>Q</code> was used then <code>recid=0</code> and if <code>-Q</code> was used then <code>recid=1</code> and so on.</p>
<h1>method 2:</h1>
<ul>
<li>It requires full <code>r</code> or <code>R=(x<sub>R</sub>, y<sub>R</sub>)</code> so it can only be done while signing (requires having the private key).</li>
<li>It is much faster because there is no additional calculations. It just needs value checks which are fast.</li>
<li>This is usually referred to as <code>v</code> in libraries</li>
</ul>
<p><code>byte v = if(R.X &gt; curve.N) then 2 else 0) | (if R.Y.IsEven then 0 else 1);</code></p>
<p>Basically it is check to see if <code>x<sub>R</sub></code> is bigger than curve order (N) and whether <code>y<sub>R</sub></code> is even or odd. Something that should not be forgotten is a &quot;flip&quot; that is performed based on <code>s</code> and whether <code>s</code> was used in the signature or <code>-s</code></p>
<p><code>if s &gt; curve.N/2 then v^=1 else do nothing</code></p>
<p>** Note that in the end you need to calculate the following:<br />
<code>recid = 27 + v + (if compressed 4 else 0)</code></p>
