<story><title>How to Get into Natural Language Processing</title><url>https://blog.ycombinator.com/how-to-get-into-natural-language-processing/</url></story><parent_chain><item><author>d_burfoot</author><text>&amp;gt; Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning.&lt;p&gt;This is true, but it is only part of the answer.&lt;p&gt;Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure, rare, archaic, or useable only in specific contexts or with specific words. However, they are part of the language, a native speaker will be able to use them and comprehend them without difficulty, and an NLP system must be able to &amp;quot;understand&amp;quot; them in order to extract the correct meaning from a sentence.&lt;p&gt;As just a minor example off the top of my head, compare the phrase &amp;quot;peeled peach&amp;quot; with &amp;quot;hairy-peeled peach&amp;quot;. The former phrase means a peach without a peel, while the latter means a peach with a hairy peel. So a good NLP system must not only recognize the existence of the two grammatical rules, but also be able to disambiguate them correctly.</text></item></parent_chain><comment><author>chch</author><text>&amp;gt; or useable only in specific contexts or with specific words.&lt;p&gt;A good example of this is the Winograd Schema. You might think you can figure out a good algorithm for anaphoric resolution (i.e. If you see &amp;quot;Sally called and she said hello.&amp;quot;, who is &amp;quot;she&amp;quot;?) that just relies on the structure of a sentence, without considering semantics.&lt;p&gt;But here&amp;#x27;s a counterexample:&lt;p&gt;&amp;quot;The city councilmen refused the demonstrators a permit because they feared violence.&amp;quot;&lt;p&gt;Who are &amp;#x27;they&amp;#x27;?&lt;p&gt;&amp;quot;The city councilmen refused the demonstrators a permit because they advocated violence.&amp;quot;&lt;p&gt;Now who are &amp;#x27;they&amp;#x27;?&lt;p&gt;If you&amp;#x27;re like most people, even though only the verb changed, the binding of &amp;#x27;they&amp;#x27; based on the deeper semantic meaning also changed.&lt;p&gt;These sentences are called Winograd Schema[1], and there are plenty more like it.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Winograd_Schema_Challenge&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Winograd_Schema_Challenge&lt;/a&gt;</text></comment>
<story><title>How to Get into Natural Language Processing</title><url>https://blog.ycombinator.com/how-to-get-into-natural-language-processing/</url></story><parent_chain><item><author>d_burfoot</author><text>&amp;gt; Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning.&lt;p&gt;This is true, but it is only part of the answer.&lt;p&gt;Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure, rare, archaic, or useable only in specific contexts or with specific words. However, they are part of the language, a native speaker will be able to use them and comprehend them without difficulty, and an NLP system must be able to &amp;quot;understand&amp;quot; them in order to extract the correct meaning from a sentence.&lt;p&gt;As just a minor example off the top of my head, compare the phrase &amp;quot;peeled peach&amp;quot; with &amp;quot;hairy-peeled peach&amp;quot;. The former phrase means a peach without a peel, while the latter means a peach with a hairy peel. So a good NLP system must not only recognize the existence of the two grammatical rules, but also be able to disambiguate them correctly.</text></item></parent_chain><comment><author>will_pseudonym</author><text>Not just that, grammar rules can come into existence&amp;#x2F;change spontaneously. Here&amp;#x27;s an example and name for this phenomenon:&lt;p&gt;Because syntactic change.</text></comment>