Q:

How to reconcile prāyaścitta (expiation) involving death with suicide (itself a big sin)?

A twice-born person, having, through folly, drunk wine, shall drink wine red-hot; he becomes freed from his guilt, when his body has been completely burnt by it. — (90)

(Manusmriti verse 11.90 - Expiation of drinking Wine (surā)

‘Red hot’— Though the text uses the word ‘varṇa’ colour, yet it is mere heat that is meant; as is clear from what follows, about ‘the body being burnt.’—(90)

(Medhātithi’s commentary)

...according to Nārāyaṇa the death-penance is meant for all twice-born men partaking of liquor distilled from grains, and by Brāhmaṇas who have drunk any of the three kinds of liquor described under verse

(Explanatory notes by Ganganath Jha)

Gautama (23.1).—‘They shall pour hot wine into the mouth of a Brāhmaṇa who has drunk wine; he will be purified by death.’
Āpastamba (1.25.3).—‘A wine-drinker has to drink exceedingly hot liquor, so that he dies.’

(Comparative notes by various authors)

What do dharma śāstras say about such atonements that can result in death or getting admitted to a hospital due to third-degree burns? Isn't it same as suicide which itself is a big sin? Is suicide, if part of atonement, not a sin? Were there any scholarly debates on this, if so, what's the conclusion?

A:

As shown in this answer one definition of ShAstra (scripture) is the following:

ShAsanAdanisham devi varnAshramanivAsinAm | TAranAt sarva
  pApebhyoh ShAstra mityabhidhiyate ||
O Devi! Because it constantly rules (ShAsana) all dwellers of all the
  orders of life, and because it protects (TArana or TrAna) all from all
  kinds of demerits, it is called ShAstra.
KulArnava Tantram 17.40.

So, it is evident that following scriptural injunctions can not lead one to accumulate sins. It's in fact the other way round. Because scriptures are there for protecting one from accumulating sins.
Similarly, Manu Smriti says, one who follows the rules mentioned in the Sruti and Smriti, attains bliss and higher stations. So, by not following such injunctions one incurs sins in fact and not by following those.

2.9. For that man who obeys the law prescribed in the revealed texts and in the sacred tradition, gains fame in this (world) and after
  death unsurpassable bliss.
2.12. The Veda, the sacred tradition, the customs of virtuous men, and one’s own pleasure, they declare to be visibly the fourfold means of
  defining the sacred law.

So, in particular, whatever is stated in Manu Smriti is the sacred law (Dharma).
But, at the same time, suicide is clearly held as a bad Karma (Adharma) as per the same Smriti and other scriptures.

5.89. Libations of water shall not be offered to those who (neglect the prescribed rites and may be said to) have been born in vain, to
  those born in consequence of an illegal mixture of the castes, to
  those who are ascetics (of heretical sects), and to those who have
  committed suicide

So, the simple argument here is that such atonement measures can not be regarded as the same as suicide, because the same scripture which prescribes them as sin-cleansing, regards suicide as a sin.
It should be understood like this.
Also, note that, although suicide is considered as a sin, one can end his life by fasting under certain circumstances, and in which case it is regarded as a good Karma as opposed to a sin.

6.29. These and other observances must a Brahmana who dwells in the forest diligently practise, and in order to attain complete (union
  with) the (supreme) Soul, (he must study) the various sacred texts
  contained in the Upanishads,
6.30. (As well as those rites and texts) which have been practised and studied by the sages (Rishis), and by Brahmana householders, in order
  to increase their knowledge (of Brahman), and their austerity, and in
  order to sanctify their bodies;
6.31. Or let him walk, fully determined and going straight on, in a north-easterly direction, subsisting on water and air, until his body
  sinks to rest.
6.32. A Brahmana, having got rid of his body by one of those modes practised by the great sages, is exalted in the world of Brahman, free
  from sorrow and fear.

UPDATE:
All the Smritis agree with such punishments for some grave sins, so according to those scriptures the punishment obviously leads to removal of sins as opposed to accumulation of the same.
The only contrary view on this matter is held by Harita, the author of Harita Smriti. According to him, suicide itself is a big sin and hence it can not be a remedy. This is mentioned in Apasthambha Dharma Sutras 1.28.16:

A man who has had sex with the wife of an elder, however, should enter
  a hollow metal column , have fires lit on both sides, and burn himself
  up.
  ‘That is wrong,’ says Harita; for anyone who kills himself or another man becomes a heinous sinner without a doubt . What such a
  man should do is to live in the above manner* until his last breath.
  No rehabilitation is possible for such a man in this life; his sin,
  however, is removed.

So, among all the Dharma Shastra writers, only Harita agrees with your view. Also Apasthambha seems to be agreeing with Harita's views as well. In his Smriti Apasthambha cites Harita on many occasions.
(Also note that in the above passage "wife of an elder" is actually "wife of the Guru".)

