<story><title>OpenVoice: Versatile instant voice cloning</title><url>https://research.myshell.ai/open-voice</url></story><parent_chain><item><author>randkyp</author><text>This is HN, so I&amp;#x27;m surprised that no one in the comments section has run this locally. :)&lt;p&gt;Following the instructions in their repo (and moving the checkpoints&amp;#x2F; and resources&amp;#x2F; folder into the &amp;quot;nested&amp;quot; openvoice subfolder), I managed to get the Gradio demo running. Simple enough.&lt;p&gt;It appears to be quicker than XTTS2 on my machine (RTX 3090), and utilizes approximately 1.5GB of VRAM. The Gradio demo is limited to 200 characters, perhaps for resource usage concerns, but it seems to run at around 8x realtime (8 seconds of speech for about 1 second of processing time.)&lt;p&gt;EDIT: patched the Gradio demo for longer text; it&amp;#x27;s way faster than that. One minute of speech only took ~4 seconds to render. Default voice sample, reading this very comment: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;voca.ro&amp;#x2F;18JIHDs4vI1v&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;voca.ro&amp;#x2F;18JIHDs4vI1v&lt;/a&gt; 
I had to write out acronyms -- XTTS2 to &amp;quot;ex tee tee ess two&amp;quot;, for example.&lt;p&gt;The voice clarity is better than XTTS2, too, but the speech can sound a bit stilted and, well, robotic&amp;#x2F;TTS-esque compared to it. The cloning consistency is definitely a step above XTTS2 in my experience -- XTTS2 would sometimes have random pitch shifts or plosives&amp;#x2F;babble in the middle of speech.</text></item></parent_chain><comment><author>bambax</author><text>I am trying to run it locally but it doesn&amp;#x27;t quite work for me.&lt;p&gt;I was able to run the demos allright, but when trying to use another reference speaker (in demo_part1), the result doesn&amp;#x27;t sound at all like the source (it&amp;#x27;s just a random male voice).&lt;p&gt;I&amp;#x27;m also trying to produce French output, using a reference audio file in French for the base speaker, and a text in French. This triggers an error in api.py line 75 that the source language is not accepted.&lt;p&gt;Indeed, in api.py line 45 the only two source languages allowed are English and Chineese; simply adding French to language_marks in api.py line 43 avoids errors but produces a weird&amp;#x2F;unintelligible result with a super heavy English accent and pronunciation.&lt;p&gt;I guess one would need to generate source_se again, and probably mess with config.json and checkpoint.pth as well, but I could not find instructions on how to do this...?&lt;p&gt;Edit -- tried again on &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;app.myshell.ai&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;app.myshell.ai&amp;#x2F;&lt;/a&gt; The result sounds French alright, but still nothing like the original reference. It would be absolutely impossible to confuse one with the other, even for someone who didn&amp;#x27;t know the person very well.</text></comment>
<story><title>OpenVoice: Versatile instant voice cloning</title><url>https://research.myshell.ai/open-voice</url></story><parent_chain><item><author>randkyp</author><text>This is HN, so I&amp;#x27;m surprised that no one in the comments section has run this locally. :)&lt;p&gt;Following the instructions in their repo (and moving the checkpoints&amp;#x2F; and resources&amp;#x2F; folder into the &amp;quot;nested&amp;quot; openvoice subfolder), I managed to get the Gradio demo running. Simple enough.&lt;p&gt;It appears to be quicker than XTTS2 on my machine (RTX 3090), and utilizes approximately 1.5GB of VRAM. The Gradio demo is limited to 200 characters, perhaps for resource usage concerns, but it seems to run at around 8x realtime (8 seconds of speech for about 1 second of processing time.)&lt;p&gt;EDIT: patched the Gradio demo for longer text; it&amp;#x27;s way faster than that. One minute of speech only took ~4 seconds to render. Default voice sample, reading this very comment: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;voca.ro&amp;#x2F;18JIHDs4vI1v&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;voca.ro&amp;#x2F;18JIHDs4vI1v&lt;/a&gt; 
I had to write out acronyms -- XTTS2 to &amp;quot;ex tee tee ess two&amp;quot;, for example.&lt;p&gt;The voice clarity is better than XTTS2, too, but the speech can sound a bit stilted and, well, robotic&amp;#x2F;TTS-esque compared to it. The cloning consistency is definitely a step above XTTS2 in my experience -- XTTS2 would sometimes have random pitch shifts or plosives&amp;#x2F;babble in the middle of speech.</text></item></parent_chain><comment><author>causi</author><text>We&amp;#x27;re so close to me being able to open a program, feed in an epub, and get a near-human level audiobook out of it. I&amp;#x27;m so excited.</text></comment>