<story><title>The most talented person in the world</title><url>https://matt.sh/the-most-talented-person</url></story><parent_chain><item><author>frabjoused</author><text>Has anyone built a search engine that uses LLMs to pre-grade every page with metrics such as:&lt;p&gt;- Commercial bias (content compared to the source, which it learns about)&lt;p&gt;- Insincere motives&lt;p&gt;- Bloat (how many words it takes to say how little to penalize SEO bloat)&lt;p&gt;I would assume that using LLMs, we can get a pretty good idea of what is SEO bloat and who the bad actors are by this point, and just penalize those results.</text></item></parent_chain><comment><author>pona-a</author><text>Did exactly that, hacked together a small pipeline in Nushell with simonw&amp;#x2F;llm. Even with GPT-4 turbo and given direct guidelines on common spam heuristics, it seems to perform worse than a Bayesian BoW. Endless trails of questions with no relation to the title it describes as informative, presence of affiliate links often gets forgotten in a mass of tokens in their tracking parameters, relevance is consistently near 0.8 even if there&amp;#x27;s relationship between title and content, and as for insincerity, our favorite BS generator cannot for the life of it correctly recognize its own creations.&lt;p&gt;Your ideas for metrics are good, but LLMs seem to be quite terrible at any of these. A simple set of heuristics and maybe a tiny language model for named entity detection and &amp;quot;vibe checking&amp;quot; would serve you much better.&lt;p&gt;Also, a lot of the worst offenders seem to use the same Q&amp;amp;A +- conclusion structure, which Viktor from marginalia.nu wrote a simple heuristic for, which I recall he said did wonders for pruning it. Solving SEO spam is easy when you aren&amp;#x27;t the one being optimized against. What&amp;#x27;s left is scaling and information retrieval.</text></comment>
<story><title>The most talented person in the world</title><url>https://matt.sh/the-most-talented-person</url></story><parent_chain><item><author>frabjoused</author><text>Has anyone built a search engine that uses LLMs to pre-grade every page with metrics such as:&lt;p&gt;- Commercial bias (content compared to the source, which it learns about)&lt;p&gt;- Insincere motives&lt;p&gt;- Bloat (how many words it takes to say how little to penalize SEO bloat)&lt;p&gt;I would assume that using LLMs, we can get a pretty good idea of what is SEO bloat and who the bad actors are by this point, and just penalize those results.</text></item></parent_chain><comment><author>Uehreka</author><text>What makes you think that LLMs will be better at combating spam than they are at creating it? There’s no universal rule that innovations in AI will go hand in hand with innovations in detecting AI, yet I feel like I see people talking all the time like that’s the case.&lt;p&gt;As of right now, LLMs are prolific but unreliable, which makes them extremely well suited for generating spam, but unsuited to detecting it without a large number of false positives and negatives.</text></comment>