<story><title>Financial Statement Analysis with Large Language Models</title><url>https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4835311</url></story><parent_chain><item><author>posting_mess</author><text>No hedge fund registered before the last 2 weeks will use Llama3 for their &amp;quot;prod work&amp;quot; beyond &amp;quot;experiments&amp;quot;.&lt;p&gt;Quant trading is about &amp;quot;going fast&amp;quot; or &amp;quot;being super right&amp;quot;, so either you&amp;#x27;d need to be sitting on some huge llama.cpp&amp;#x2F;transformer improvement (possible but unlikely) or its more likely just some boring math applied faster than others.&lt;p&gt;Even if they are using a &amp;quot;LLM&amp;quot;, they wont tell you or even hint at it - &amp;quot;efficient market&amp;quot; n all that.&lt;p&gt;Remember all quants need to be &amp;quot;the smartest in the world&amp;quot; or their whole industry falls apart, wait till you find out its all &amp;quot;high school math&amp;quot; based on algo&amp;#x27;s largely derived 30&amp;#x2F;40 years ago (okay not as true for &amp;quot;quants&amp;quot; but most &amp;quot;trading&amp;quot; isn&amp;#x27;t as complex as they&amp;#x27;d like you&amp;#x2F;us to believe).</text></item><item><author>bethekind</author><text>Do you use llama 3 for your work?</text></item><item><author>flourpower471</author><text>Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art.&lt;p&gt;People didn&amp;#x27;t stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.</text></item><item><author>antimatter15</author><text>Figure 3 on p.40 of the paper seems to show that their LLM based model does not statistically significantly outperform a 3 layer neural network using 59 variables from 1989.&lt;p&gt;&lt;pre&gt;&lt;code&gt;  This figure compares the prediction performance of GPT and quantitative models based on machine learning. Stepwise Logistic follows Ou and Penman (1989)’s structure with their 59 financial predictors. ANN is a three-layer artificial neural network model using the same set of variables as in Ou and Penman (1989). GPT (with CoT) provides the model with financial statement information and detailed chain-of-thought prompts. We report average accuracy (the percentage of correct predictions out of total predictions) for each method (left) and F1 score (right). We obtain bootstrapped standard errors by randomly sampling 1,000 observations 1,000 times and include 95% confidence intervals.&lt;/code&gt;&lt;/pre&gt;</text></item></parent_chain><comment><author>flourpower471</author><text>Well I work in prop trading and have only ever worked for prop firms- our firm trades it&amp;#x27;s own capital and distributes it to the owners and us under profit share agreements - so we have no incentive to sell ourselves as any smarter than the reality.&lt;p&gt;Saying it&amp;#x27;s all high school math is a bit of a loaded phrase. &amp;quot;High school math&amp;quot; incorporates basically all practical computer science and machine learning and statistics.&lt;p&gt;If I suspect you could probably build a particle accelerator without using more math than a bit of calculus - that doesn&amp;#x27;t make it easy or simple to build one.&lt;p&gt;Very few people I&amp;#x27;ve worked with have ever said they are doing cutting edge math - it&amp;#x27;s more like scientific research . The space of ideas is huge, and the ways to ruin yourself innumerable. It&amp;#x27;s more about people who have a scientific mindset who can make progress in a very high noise and adaptive environment.&lt;p&gt;It&amp;#x27;s probably more about avoiding blunders than it is having some genius paradigm shifting idea.</text></comment>
<story><title>Financial Statement Analysis with Large Language Models</title><url>https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4835311</url></story><parent_chain><item><author>posting_mess</author><text>No hedge fund registered before the last 2 weeks will use Llama3 for their &amp;quot;prod work&amp;quot; beyond &amp;quot;experiments&amp;quot;.&lt;p&gt;Quant trading is about &amp;quot;going fast&amp;quot; or &amp;quot;being super right&amp;quot;, so either you&amp;#x27;d need to be sitting on some huge llama.cpp&amp;#x2F;transformer improvement (possible but unlikely) or its more likely just some boring math applied faster than others.&lt;p&gt;Even if they are using a &amp;quot;LLM&amp;quot;, they wont tell you or even hint at it - &amp;quot;efficient market&amp;quot; n all that.&lt;p&gt;Remember all quants need to be &amp;quot;the smartest in the world&amp;quot; or their whole industry falls apart, wait till you find out its all &amp;quot;high school math&amp;quot; based on algo&amp;#x27;s largely derived 30&amp;#x2F;40 years ago (okay not as true for &amp;quot;quants&amp;quot; but most &amp;quot;trading&amp;quot; isn&amp;#x27;t as complex as they&amp;#x27;d like you&amp;#x2F;us to believe).</text></item><item><author>bethekind</author><text>Do you use llama 3 for your work?</text></item><item><author>flourpower471</author><text>Not to mention, as somebody who works in quant trading doing ml all day on this kind of data. That ann benchmark is nowhere near state of the art.&lt;p&gt;People didn&amp;#x27;t stop working on this in 1989 - they realised they can make lots of money doing it and do it privately.</text></item><item><author>antimatter15</author><text>Figure 3 on p.40 of the paper seems to show that their LLM based model does not statistically significantly outperform a 3 layer neural network using 59 variables from 1989.&lt;p&gt;&lt;pre&gt;&lt;code&gt;  This figure compares the prediction performance of GPT and quantitative models based on machine learning. Stepwise Logistic follows Ou and Penman (1989)’s structure with their 59 financial predictors. ANN is a three-layer artificial neural network model using the same set of variables as in Ou and Penman (1989). GPT (with CoT) provides the model with financial statement information and detailed chain-of-thought prompts. We report average accuracy (the percentage of correct predictions out of total predictions) for each method (left) and F1 score (right). We obtain bootstrapped standard errors by randomly sampling 1,000 observations 1,000 times and include 95% confidence intervals.&lt;/code&gt;&lt;/pre&gt;</text></item></parent_chain><comment><author>qeternity</author><text>It’s impressive how incorrect so much of this information is. High frequency trading is about going fast. There is a huge mid and low freq quant industry. Also most quant strategies are absolutely not about being “super right”…that would be the province of concentrated discretionary strategies. Quant is almost always about being slightly more right than wrong but at large scale.&lt;p&gt;What algos are you referring to derived 30 or 40 years ago? Do you understand the decay for a typical strategy? None of this makes any sense.</text></comment>