user96871: I've installed two major Lightning Network implementations, CLN (formerly c-lightning) and LND, with bitcoind testnet as backchain on the same server. On the first startup I noticed CLN synchronizes all blocks from the beginning of the blockchain:
2022-07-05T23:31:29.487Z DEBUG   lightningd: Still waiting for initial block download
2022-07-05T23:31:29.593Z DEBUG   lightningd: Adding block 1: 00000000b873e79784647a6c82962c70d228557d24a747ea4d1b8bbe878e1206
2022-07-05T23:31:29.647Z DEBUG   lightningd: Adding block 2: 000000006c02c8ea6e4ff69651f7fcde348fb9d557a06e6957b65552002a7820
2022-07-05T23:31:29.685Z DEBUG   lightningd: Adding block 3: 000000008b896e272758da5297bcd98fdc6d97c9b765ecec401e286dc1fdbe10
2022-07-05T23:31:29.732Z DEBUG   lightningd: Adding block 4: 000000008b5d0af9ffb1741e38b17b193bd12d7683401cecd2fd94f548b6e5dd
2022-07-05T23:31:29.752Z DEBUG   lightningd: Adding block 5: 00000000bc45ac875fbd34f43f7732789b6ec4e8b5974b4406664a75d43b21a1
2022-07-05T23:31:29.774Z DEBUG   lightningd: Adding block 6: 000000006633685edce4fa4d8f12d001781c6849837d1632c4e2dd6ff2090a7b
2022-07-05T23:31:29.792Z DEBUG   lightningd: Adding block 7: 00000000e29e3aa65f3d12440eac9081844c464aeba7c6e6121dfc8ac0c02ba6

while LND only scans blocks from the wallet's birthday onwards:
2022-07-05 23:21:55.757 [INF] LTND: Waiting for chain backend to finish sync, start_height=2284292
2022-07-05 23:21:56.613 [DBG] LNWL: Chain backend synced to tip!
2022-07-05 23:21:56.614 [DBG] LNWL: Locating suitable block for birthday 2022-07-03 15:15:05 -0300 -03 between blocks 0-2284292
2022-07-05 23:21:56.617 [DBG] LNWL: Checking candidate block: height=1142146, hash=00000000000bee59e6acdb88deee299fcce4c8603a04a60b022d70f256b4d880, timestamp=2017-06-18 16:00:52 -0300 -03
2022-07-05 23:21:56.619 [DBG] LNWL: Checking candidate block: height=1713219, hash=000000000000905bf5e522da9cc5d30a93728feb51f3409396193f120505e2ca, timestamp=2020-04-14 06:51:58 -0300 -03
2022-07-05 23:21:56.621 [DBG] LNWL: Checking candidate block: height=1998755, hash=00000000000009aa098093884b3f352bfc1efa24ec2af3d541e2fa2ba7e54d6b, timestamp=2021-06-02 19:57:47 -0300 -03
2022-07-05 23:21:56.623 [DBG] LNWL: Checking candidate block: height=2141523, hash=00000000849bdb7b93b3deb7182cb1ec3dc2e470d4359ac6fe67d19e5bd5f84e, timestamp=2022-02-14 17:16:49 -0300 -03
2022-07-05 23:21:56.625 [DBG] LNWL: Checking candidate block: height=2212907, hash=00000000000004e43dbee8c2f5ae622bb6a984183ef07a8341b1eec6ce331ce5, timestamp=2022-04-28 00:06:02 -0300 -03
2022-07-05 23:21:56.627 [DBG] LNWL: Checking candidate block: height=2248599, hash=00000000000009c8763aaa3757774b8c8611495dd2f190672d1c4084d40368d6, timestamp=2022-05-26 08:36:29 -0300 -03
2022-07-05 23:21:56.629 [DBG] LNWL: Checking candidate block: height=2266445, hash=00000000000022f256ef974c1467221afc19f6c087ffb147554191d128099438, timestamp=2022-06-16 10:46:47 -0300 -03
2022-07-05 23:21:56.631 [DBG] LNWL: Checking candidate block: height=2275368, hash=0000000000000069a27c710313fc5a764e8bab5fed550af837d0c8b1d6fdd558, timestamp=2022-06-16 14:23:46 -0300 -03
2022-07-05 23:21:56.633 [DBG] LNWL: Checking candidate block: height=2279830, hash=00000000000001d54cbb97ed368723e4ae7681f6c3d625d517b9dbe3a7d0678a, timestamp=2022-06-18 00:41:09 -0300 -03
2022-07-05 23:21:56.635 [DBG] LNWL: Checking candidate block: height=2282061, hash=00000000000000b1517a474a0c8e32a9bd452bd1117cc2a799195991d10c3306, timestamp=2022-06-23 13:04:59 -0300 -03
2022-07-05 23:21:56.637 [DBG] LNWL: Checking candidate block: height=2283176, hash=00000000000000498bf2dbab757cba69b77a8d31ea437b2a377aabdb08c0d3cc, timestamp=2022-06-28 19:37:39 -0300 -03
2022-07-05 23:21:56.639 [DBG] LNWL: Checking candidate block: height=2283734, hash=000000000000003a55f287f3243841f215572c092995ccf286d9d92398d21785, timestamp=2022-07-02 14:50:17 -0300 -03
2022-07-05 23:21:56.641 [DBG] LNWL: Checking candidate block: height=2284013, hash=000000000000004f39aada91825e5d6121e6d7bcda6be504a6f2c13dac133df9, timestamp=2022-07-04 05:13:13 -0300 -03
2022-07-05 23:21:56.643 [DBG] LNWL: Checking candidate block: height=2283873, hash=000000003c07419de729aa383f639e506e24dc5f7e947f80cfaf5d2e67837c43, timestamp=2022-07-03 08:31:57 -0300 -03
2022-07-05 23:21:56.645 [DBG] LNWL: Checking candidate block: height=2283943, hash=0000000000000017b3b716a4e380900fbf2e114d2536a6f5e71427e5b10a34f8, timestamp=2022-07-03 17:57:30 -0300 -03
2022-07-05 23:21:56.647 [DBG] LNWL: Checking candidate block: height=2283908, hash=000000000000004aef1f2fc4dc6973f7c9ac67e5f6c69e1df089f92c83e91324, timestamp=2022-07-03 13:34:25 -0300 -03
2022-07-05 23:21:56.647 [DBG] LNWL: Found birthday block: height=2283908, hash=000000000000004aef1f2fc4dc6973f7c9ac67e5f6c69e1df089f92c83e91324, timestamp=2022-07-03 13:34:25 -0300 -03
2022-07-05 23:21:56.656 [INF] LNWL: Started rescan from block 000000000000004aef1f2fc4dc6973f7c9ac67e5f6c69e1df089f92c83e91324 (height 2283908) for 0 addresses
2022-07-05 23:21:56.687 [INF] LNWL: Catching up block hashes to height 2283919, this might take a while
2022-07-05 23:21:56.693 [INF] LNWL: Done catching up block hashes
2022-07-05 23:21:56.693 [INF] LNWL: Rescanned through block 0000000000000005cdcf62d03147fc812f5b01b21a919bdbe38a3588cafdd5f8 (height 2283919)
2022-07-05 23:21:59.062 [INF] LNWL: Catching up block hashes to height 2284292, this might take a while
2022-07-05 23:21:59.063 [INF] LNWL: Done catching up block hashes
2022-07-05 23:21:59.063 [INF] LNWL: Finished rescan for 0 addresses (synced to block 00000000eb61cbd41eaaa4c3646418dd1acf75d30b04239385b19c51fef43cec, height 2284292)
2022-07-05 23:21:59.785 [INF] LTND: Chain backend is fully synced (end_height=2284292)!

This seems to give a huge advantage over CLN on first startup times. As of now, my CLN node has spent over 14 hours synchronizing while LND worked in a few seconds. Also, as far I understand, a Lightning node should sync the full UTXO set to verify open channels. But this should be provided by the backchain.
Is there a good reason for CLN to reprocess everything? Also, if LND only syncs from the wallet's birthday, how does it verify channels' funding transactions?
Thanks in advance.

user47827: For a wallet to assess how many Bitcoin it controls it needs to scan the blockchain for UTXOs that are controlled by the wallet.
LND uses a wallet seed format called aezeed which defines a wallet birthday (days since Bitcoin's genesis). It scans the blockchain from this birthday rather than from genesis for relevant UTXOs and assumes that it doesn't control any UTXOs created prior to this birthday.

I don't understand why I should synchronize my Lightning node at all if I run a Bitcoin full node on the same server (why not just tell CLN/LND where the blocks are?).

Your Bitcoin full node verifies the blockchain from genesis. It is interested in all transactions and all blocks being valid and meeting consensus rules, not just the UTXOs that are relevant to the wallet. Once your full node has done this your wallet can assume it is scanning the valid blockchain.

user120: CLN should start from close to the current blockchain height, however that current blockchain height is learned from bitcoind which in your case was still syncing.
By starting bitcoind and lightningd at roughly the same time, we get a bitcoind that reports the current height very close to the genesis block at height 0. When lightningd then asks bitcoind for the current blockchain height it gets that low-balled height back and assumes it should start syncing from that height, leading to the entire blockchain to be processed.
There are a couple of solutions:

Wait for bitcoind to sync completely (or almost completely if you're fine with CLN processing a few more blocks) and only then start CLN
If you started CLN before bitcoind was synced you can reset the height from which lightningd syncs by specifying the absolute height to sync from with a dash (-, or minus) in front of it: lightningd --rescan=-600000 [other options] will reset the blockchain height to 600k and lightningd should continue scanning from that height.
Use an alternative backend such as sauron or btcli4j which talk to an explorer instead of a local bitcoind for super lightweight (but more trusted) bitcoin backend. You can later switch over to a local bitcoind once that's synced too.

If there is an idea on how to make this more user-friendly we're open to suggestions.
Disclaimer: I'm one of the CLN maintainers.

96871: <p>I've installed two major Lightning Network implementations, <a href="https://github.com/ElementsProject/lightning" rel="nofollow noreferrer">CLN</a> (formerly c-lightning) and <a href="https://github.com/lightningnetwork/lnd" rel="nofollow noreferrer">LND</a>, with bitcoind testnet as backchain on the same server. On the first startup I noticed CLN synchronizes all blocks from the beginning of the blockchain:</p>
<pre><code>2022-07-05T23:31:29.487Z DEBUG   lightningd: Still waiting for initial block download
2022-07-05T23:31:29.593Z DEBUG   lightningd: Adding block 1: 00000000b873e79784647a6c82962c70d228557d24a747ea4d1b8bbe878e1206
2022-07-05T23:31:29.647Z DEBUG   lightningd: Adding block 2: 000000006c02c8ea6e4ff69651f7fcde348fb9d557a06e6957b65552002a7820
2022-07-05T23:31:29.685Z DEBUG   lightningd: Adding block 3: 000000008b896e272758da5297bcd98fdc6d97c9b765ecec401e286dc1fdbe10
2022-07-05T23:31:29.732Z DEBUG   lightningd: Adding block 4: 000000008b5d0af9ffb1741e38b17b193bd12d7683401cecd2fd94f548b6e5dd
2022-07-05T23:31:29.752Z DEBUG   lightningd: Adding block 5: 00000000bc45ac875fbd34f43f7732789b6ec4e8b5974b4406664a75d43b21a1
2022-07-05T23:31:29.774Z DEBUG   lightningd: Adding block 6: 000000006633685edce4fa4d8f12d001781c6849837d1632c4e2dd6ff2090a7b
2022-07-05T23:31:29.792Z DEBUG   lightningd: Adding block 7: 00000000e29e3aa65f3d12440eac9081844c464aeba7c6e6121dfc8ac0c02ba6
</code></pre>
<p>while LND only scans blocks from the wallet's birthday onwards:</p>
<pre><code>2022-07-05 23:21:55.757 [INF] LTND: Waiting for chain backend to finish sync, start_height=2284292
2022-07-05 23:21:56.613 [DBG] LNWL: Chain backend synced to tip!
2022-07-05 23:21:56.614 [DBG] LNWL: Locating suitable block for birthday 2022-07-03 15:15:05 -0300 -03 between blocks 0-2284292
2022-07-05 23:21:56.617 [DBG] LNWL: Checking candidate block: height=1142146, hash=00000000000bee59e6acdb88deee299fcce4c8603a04a60b022d70f256b4d880, timestamp=2017-06-18 16:00:52 -0300 -03
2022-07-05 23:21:56.619 [DBG] LNWL: Checking candidate block: height=1713219, hash=000000000000905bf5e522da9cc5d30a93728feb51f3409396193f120505e2ca, timestamp=2020-04-14 06:51:58 -0300 -03
2022-07-05 23:21:56.621 [DBG] LNWL: Checking candidate block: height=1998755, hash=00000000000009aa098093884b3f352bfc1efa24ec2af3d541e2fa2ba7e54d6b, timestamp=2021-06-02 19:57:47 -0300 -03
2022-07-05 23:21:56.623 [DBG] LNWL: Checking candidate block: height=2141523, hash=00000000849bdb7b93b3deb7182cb1ec3dc2e470d4359ac6fe67d19e5bd5f84e, timestamp=2022-02-14 17:16:49 -0300 -03
2022-07-05 23:21:56.625 [DBG] LNWL: Checking candidate block: height=2212907, hash=00000000000004e43dbee8c2f5ae622bb6a984183ef07a8341b1eec6ce331ce5, timestamp=2022-04-28 00:06:02 -0300 -03
2022-07-05 23:21:56.627 [DBG] LNWL: Checking candidate block: height=2248599, hash=00000000000009c8763aaa3757774b8c8611495dd2f190672d1c4084d40368d6, timestamp=2022-05-26 08:36:29 -0300 -03
2022-07-05 23:21:56.629 [DBG] LNWL: Checking candidate block: height=2266445, hash=00000000000022f256ef974c1467221afc19f6c087ffb147554191d128099438, timestamp=2022-06-16 10:46:47 -0300 -03
2022-07-05 23:21:56.631 [DBG] LNWL: Checking candidate block: height=2275368, hash=0000000000000069a27c710313fc5a764e8bab5fed550af837d0c8b1d6fdd558, timestamp=2022-06-16 14:23:46 -0300 -03
2022-07-05 23:21:56.633 [DBG] LNWL: Checking candidate block: height=2279830, hash=00000000000001d54cbb97ed368723e4ae7681f6c3d625d517b9dbe3a7d0678a, timestamp=2022-06-18 00:41:09 -0300 -03
2022-07-05 23:21:56.635 [DBG] LNWL: Checking candidate block: height=2282061, hash=00000000000000b1517a474a0c8e32a9bd452bd1117cc2a799195991d10c3306, timestamp=2022-06-23 13:04:59 -0300 -03
2022-07-05 23:21:56.637 [DBG] LNWL: Checking candidate block: height=2283176, hash=00000000000000498bf2dbab757cba69b77a8d31ea437b2a377aabdb08c0d3cc, timestamp=2022-06-28 19:37:39 -0300 -03
2022-07-05 23:21:56.639 [DBG] LNWL: Checking candidate block: height=2283734, hash=000000000000003a55f287f3243841f215572c092995ccf286d9d92398d21785, timestamp=2022-07-02 14:50:17 -0300 -03
2022-07-05 23:21:56.641 [DBG] LNWL: Checking candidate block: height=2284013, hash=000000000000004f39aada91825e5d6121e6d7bcda6be504a6f2c13dac133df9, timestamp=2022-07-04 05:13:13 -0300 -03
2022-07-05 23:21:56.643 [DBG] LNWL: Checking candidate block: height=2283873, hash=000000003c07419de729aa383f639e506e24dc5f7e947f80cfaf5d2e67837c43, timestamp=2022-07-03 08:31:57 -0300 -03
2022-07-05 23:21:56.645 [DBG] LNWL: Checking candidate block: height=2283943, hash=0000000000000017b3b716a4e380900fbf2e114d2536a6f5e71427e5b10a34f8, timestamp=2022-07-03 17:57:30 -0300 -03
2022-07-05 23:21:56.647 [DBG] LNWL: Checking candidate block: height=2283908, hash=000000000000004aef1f2fc4dc6973f7c9ac67e5f6c69e1df089f92c83e91324, timestamp=2022-07-03 13:34:25 -0300 -03
2022-07-05 23:21:56.647 [DBG] LNWL: Found birthday block: height=2283908, hash=000000000000004aef1f2fc4dc6973f7c9ac67e5f6c69e1df089f92c83e91324, timestamp=2022-07-03 13:34:25 -0300 -03
2022-07-05 23:21:56.656 [INF] LNWL: Started rescan from block 000000000000004aef1f2fc4dc6973f7c9ac67e5f6c69e1df089f92c83e91324 (height 2283908) for 0 addresses
2022-07-05 23:21:56.687 [INF] LNWL: Catching up block hashes to height 2283919, this might take a while
2022-07-05 23:21:56.693 [INF] LNWL: Done catching up block hashes
2022-07-05 23:21:56.693 [INF] LNWL: Rescanned through block 0000000000000005cdcf62d03147fc812f5b01b21a919bdbe38a3588cafdd5f8 (height 2283919)
2022-07-05 23:21:59.062 [INF] LNWL: Catching up block hashes to height 2284292, this might take a while
2022-07-05 23:21:59.063 [INF] LNWL: Done catching up block hashes
2022-07-05 23:21:59.063 [INF] LNWL: Finished rescan for 0 addresses (synced to block 00000000eb61cbd41eaaa4c3646418dd1acf75d30b04239385b19c51fef43cec, height 2284292)
2022-07-05 23:21:59.785 [INF] LTND: Chain backend is fully synced (end_height=2284292)!
</code></pre>
<p>This seems to give a huge advantage over CLN on first startup times. As of now, my CLN node has spent over 14 hours synchronizing while LND worked in a few seconds. Also, as far I understand, a Lightning node should sync the full UTXO set to verify open channels. But this should be provided by the backchain.</p>
<p>Is there a good reason for CLN to reprocess everything? Also, if LND only syncs from the wallet's birthday, how does it verify channels' funding transactions?</p>
<p>Thanks in advance.</p>
 47827: <p>For a wallet to assess how many Bitcoin it controls it needs to scan the blockchain for UTXOs that are controlled by the wallet.</p>
<p>LND uses a wallet seed format called <a href="https://github.com/lightningnetwork/lnd/blob/master/aezeed/README.md" rel="nofollow noreferrer">aezeed</a> which defines a wallet <a href="https://github.com/lightningnetwork/lnd/blob/738241d5fa1dcdd29c5ceb53bca20ea28365a216/aezeed/cipherseed.go#L123" rel="nofollow noreferrer">birthday</a> (days since Bitcoin's genesis). It scans the blockchain from this birthday rather than from genesis for relevant UTXOs and assumes that it doesn't control any UTXOs created prior to this birthday.</p>
<blockquote>
<p>I don't understand why I should synchronize my Lightning node at all if I run a Bitcoin full node on the same server (why not just tell CLN/LND where the blocks are?).</p>
</blockquote>
<p>Your Bitcoin full node verifies the blockchain from genesis. It is interested in <strong>all</strong> transactions and <strong>all</strong> blocks being valid and meeting consensus rules, not just the UTXOs that are relevant to the wallet. Once your full node has done this your wallet can assume it is scanning the valid blockchain.</p>
 120: <p>CLN should start from close to the current blockchain height, however that current blockchain height is learned from <code>bitcoind</code> which in your case was still syncing.</p>
<p>By starting <code>bitcoind</code> and <code>lightningd</code> at roughly the same time, we get a <code>bitcoind</code> that reports the current height very close to the genesis block at height 0. When <code>lightningd</code> then asks <code>bitcoind</code> for the current blockchain height it gets that low-balled height back and assumes it should start syncing from that height, leading to the entire blockchain to be processed.</p>
<p>There are a couple of solutions:</p>
<ol>
<li>Wait for <code>bitcoind</code> to sync completely (or almost completely if you're fine with CLN processing a few more blocks) and only then start CLN</li>
<li>If you started CLN before <code>bitcoind</code> was synced you can reset the height from which <code>lightningd</code> syncs by specifying the absolute height to sync from with a dash (<code>-</code>, or minus) in front of it: <code>lightningd --rescan=-600000 [other options]</code> will reset the blockchain height to 600k and <code>lightningd</code> should continue scanning from that height.</li>
<li>Use an alternative backend such as <a href="https://github.com/lightningd/plugins/tree/master/sauron" rel="nofollow noreferrer">sauron</a> or <a href="https://github.com/clightning4j/btcli4j" rel="nofollow noreferrer">btcli4j</a> which talk to an explorer instead of a local <code>bitcoind</code> for super lightweight (but more trusted) bitcoin backend. You can later switch over to a local <code>bitcoind</code> once that's synced too.</li>
</ol>
<p>If there is an idea on how to make this more user-friendly we're open to suggestions.</p>
<p>Disclaimer: I'm one of the CLN maintainers.</p>
