<story><title>Show HN: Big List of Naughty Strings for testing user-input data</title><url>https://github.com/minimaxir/big-list-of-naughty-strings</url></story><parent_chain><item><author>rspeer</author><text>Most of what I do involves the messy world of text, and I think this is a great resource. I wish the software I depended on tested against it.&lt;p&gt;I can think of a few more cases that I&amp;#x27;ve seen cause havoc:&lt;p&gt;- U+FEFF in the middle of a string (people are used to seeing it at the beginning of a string, because Microsoft, but elsewhere it may be more surprising)&lt;p&gt;- U+0 (it&amp;#x27;s encoded as the null byte!)&lt;p&gt;- U+1B (the codepoint for &amp;quot;escape&amp;quot;)&lt;p&gt;- U+85 (Python&amp;#x27;s &amp;quot;codecs&amp;quot; module thinks this is a newline, while the &amp;quot;io&amp;quot; module and the Python 3 standard library don&amp;#x27;t)&lt;p&gt;- U+2028 and U+2029 (even weirder linebreaks that cause disagreement when used in JSON literals)&lt;p&gt;- A glyph with a million combining marks on it, but not in NFC order (do your Unicode algorithms use insertion sort?)&lt;p&gt;- The sequence U+100000 U+010000 (triggers a weird bug in Python 3.2 only)&lt;p&gt;- &amp;quot;Forbidden&amp;quot; strings that are still encodable, such as U+FFFF, U+1FFFF, and for some reason U+FDD0&lt;p&gt;People should also test what happens with isolated surrogate codepoints, such as U+D800. But these can&amp;#x27;t properly be encoded in UTF-8, so I guess don&amp;#x27;t put them in the BLNS. (If you put the fake UTF-8 for them in a file, the best thing for a program to do would be to give up on reading the file.)</text></item></parent_chain><comment><author>grapeshot</author><text>Isolated UTF-16 surrogate code points definitely crash Unity when it tries to display them. (Seen when I pasted some emoji in a text box in TIS-100 and tried to backspace.)</text></comment>
<story><title>Show HN: Big List of Naughty Strings for testing user-input data</title><url>https://github.com/minimaxir/big-list-of-naughty-strings</url></story><parent_chain><item><author>rspeer</author><text>Most of what I do involves the messy world of text, and I think this is a great resource. I wish the software I depended on tested against it.&lt;p&gt;I can think of a few more cases that I&amp;#x27;ve seen cause havoc:&lt;p&gt;- U+FEFF in the middle of a string (people are used to seeing it at the beginning of a string, because Microsoft, but elsewhere it may be more surprising)&lt;p&gt;- U+0 (it&amp;#x27;s encoded as the null byte!)&lt;p&gt;- U+1B (the codepoint for &amp;quot;escape&amp;quot;)&lt;p&gt;- U+85 (Python&amp;#x27;s &amp;quot;codecs&amp;quot; module thinks this is a newline, while the &amp;quot;io&amp;quot; module and the Python 3 standard library don&amp;#x27;t)&lt;p&gt;- U+2028 and U+2029 (even weirder linebreaks that cause disagreement when used in JSON literals)&lt;p&gt;- A glyph with a million combining marks on it, but not in NFC order (do your Unicode algorithms use insertion sort?)&lt;p&gt;- The sequence U+100000 U+010000 (triggers a weird bug in Python 3.2 only)&lt;p&gt;- &amp;quot;Forbidden&amp;quot; strings that are still encodable, such as U+FFFF, U+1FFFF, and for some reason U+FDD0&lt;p&gt;People should also test what happens with isolated surrogate codepoints, such as U+D800. But these can&amp;#x27;t properly be encoded in UTF-8, so I guess don&amp;#x27;t put them in the BLNS. (If you put the fake UTF-8 for them in a file, the best thing for a program to do would be to give up on reading the file.)</text></item></parent_chain><comment><author>gsnedders</author><text>Bi-directional text is probably another one. All the bidi control characters, especially. Probably really all Unicode control characters in general.</text></comment>