<story><title>Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit</title><url>https://github.com/mezbaul-h/june</url></story><parent_chain></parent_chain><comment><author>modeless</author><text>Coqui&amp;#x27;s XTTSv2 is good for this because it has a streaming mode. I have my own version of this where I got ~500ms end-to-end response latency, which is much faster than any other open source project I&amp;#x27;ve seen. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;jdarpinian&amp;#x2F;chirpy&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;jdarpinian&amp;#x2F;chirpy&lt;/a&gt;&lt;p&gt;These are easy to make and fun to play with and it&amp;#x27;s awesome to have everything local. But it will take more to build something truly useable. A truly natural conversational AI needs to understand the nuances of conversation, most importantly when to speak and when to wait. It also needs to know subtleties of the user&amp;#x27;s voice that no speech recognizer can output, and it needs control over the output voice more precise than any TTS provides. Audio-to-audio models in the style of GPT-4o are clearly the way forward. (And someday soon, video-to-video models for video calling with a virtual avatar. And the step after that is robotics for physical avatars).&lt;p&gt;There aren&amp;#x27;t any open source audio-to-audio models yet but there are some promising approaches. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;ultravox.ai&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;ultravox.ai&lt;/a&gt; has the input half at least. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;tincans.ai&amp;#x2F;slm&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;tincans.ai&amp;#x2F;slm&lt;/a&gt; has a cool approach too.</text></comment>
<story><title>Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit</title><url>https://github.com/mezbaul-h/june</url></story><parent_chain></parent_chain><comment><author>replete</author><text>I tried a similar project out last week, which uses Ollama, FastWhisperAPI, and MeloTTS: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;PromtEngineer&amp;#x2F;Verbi&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;PromtEngineer&amp;#x2F;Verbi&lt;/a&gt;&lt;p&gt;Docker is a great option if you want lots of people to try out your project, but not many apps in this space come with a dockerfile</text></comment>