<story><title>QuiLLMan: Voice chat with Vicuna-13B</title><url>https://github.com/modal-labs/quillman</url></story><parent_chain><item><author>lxe</author><text>For &amp;quot;actually serverless&amp;quot; voice chat, check out &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;whisper.ggerganov.com&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;whisper.ggerganov.com&amp;#x2F;&lt;/a&gt;</text></item></parent_chain><comment><author>jancsika</author><text>This link is the most worthwhile hijack of an HN thread that I&amp;#x27;ve ever seen.&lt;p&gt;How is tiny.en so damned accurate?&lt;p&gt;How much faster does this run natively?&lt;p&gt;Do the NEON instructions work for arm devices like an RPI or is it just tuned for Apple?&lt;p&gt;With this and Alpaca 13B you could probably replace an entire window manager.&lt;p&gt;Edit: it seems like it was less than a year ago that local speech recognition was a slow slog through a swamp of crufty, complicated research projects with most the high quality training data hidden behind walled gardens. Now I can stutter-step over a word and a demo in my browser correctly transcribes what I meant sans stutter. What happened?</text></comment>
<story><title>QuiLLMan: Voice chat with Vicuna-13B</title><url>https://github.com/modal-labs/quillman</url></story><parent_chain><item><author>lxe</author><text>For &amp;quot;actually serverless&amp;quot; voice chat, check out &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;whisper.ggerganov.com&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;whisper.ggerganov.com&amp;#x2F;&lt;/a&gt;</text></item></parent_chain><comment><author>arthurcolle</author><text>This is so good. I tried playing around with whisper.cpp last week and got absolutely terrible performance. I played around with the &amp;quot;step size&amp;quot; as well as &amp;quot;n&amp;quot; (not sure what these do, I should probably read the docs seriously).&lt;p&gt;I tried a lot of tweaks but this one is definitely the best I&amp;#x27;ve seen.&lt;p&gt;How did it know how to spell my name correctly when I just spoke into my microphone??? The two L&amp;#x27;s usually trips up the transcription models. What????</text></comment>