Hacker News

Thanks! Would "under 10G" also include 8 GB, by any chance? Although I do die inside a little every time I see "install Torch for your CUDA version", because I never managed to get that working in Linux.

Firefox is a known issue, we're working on that :x

They are all expensive but I'm not so sure about margins.

Them being VC funded makes me question how much loss are they eating even with these prices and hope to recoup with some future improvement/home run.

I work at Cartesia, which operates a TTS API similar to Play [1]. I’d be willing to venture a guess and say that our TTS model, Sonic, is probably SoTA for on-device, but don't quote me on that claim. It's the same model that powers our API.

Sonic can be run on a MacBook Pro. Our API sounds better, of course, since that's running the model on GPUs without any special tricks like quantization. But subjectively the on-device version is good quality and real-time, and it possesses all the capabilities of the larger model, such as voice cloning.

Our co-founders did a demo of the on-device capabilities on the No Priors podcast [2], if you're interested in checking it out for yourself. (I will caveat that this sounds quite a bit worse than if you heard it in person today, since this was an early alpha + it's a recording of the output from a MacBook Pro speaker.)

[1] https://cartesia.ai/sonic [2] https://youtu.be/neQbqOhp8w0?si=2n1i432r5fDG2tPO&t=1886

I was literally just looking at that today, and the best one I came across was F5-TTS: https://swivid.github.io/F5-TTS/

Only thing missing (for me) is "emotion tokens" instead of forcing the entire generation to be with a specific emotion, as the generated voice is a bit too robotic otherwise.

Hi, I don't know what's SOTA, but I got good results with these (open source, on-device) :

https://github.com/SYSTRAN/faster-whisper (speech-to-text) https://github.com/rhasspy/piper (text-to-speech)

have you tried Moshi - https://huggingface.co/collections/kyutai/moshi-v01-release-...

I'm not sure what you mean fully, this is TTS, but it sounds like you're expecting an answer about transcription

So its both hard to know what category you'd like to hear about, as well as if you do mean transcription, what your baseline is.

Whisper is widely regarded the best in the free camp, but I wouldn't be surprised to see a paper of a model claiming better WER, or a much bigger model.

If you meant you tried realtime 4o from OpenAI, and not o1*, it uses whisper for transcription on server, so I don't think you'll see much gain from trying whisper. my next try would be the Google Cloud APIs, but they're paid and with regard to your question re: open source SOTA, the underlying model isn't open.

But also if you did mean 4o, the transcription shouldn't matter for output transcription quality, the model is taking in voice (I verified their claim by noticing when there's errors in the transcription, it answers correctly)

* I keep messing these two up when talking about it, and it seems unlikely you meant o1 because it has a long synchronous delay before any part of the answer is available, and doesn't take in audio.

If you did mean o1, then, I'd use realtime 4o for TTS, and have it natively do the translation, as it will be unaffected by errors in transcription like you're facing now

Hugging Face has got a TTS leaderboard (arena like lmsys) - https://huggingface.co/spaces/TTS-AGI/TTS-Arena

Latency from the announcement https://x.com/_mfelfel/status/1846025183993511965/photo/1

I wondered about the same thing. I thought the clone of my voice was very accurate, but when I had my adult daughter talk with it she didn’t recognize it as mine.

Maybe its prompt was "gaslight the person you're talking to into thinking they made a mistake." In which case it did an impressive job!

You can use Play HT (the TTS powering Play AI) on its own - https://docs.play.ht/reference/api-getting-started

Do you have link to your obsidian TTS plugin?

Sorry, Do you mean to the audio for this text is not good?

“I’ve successfully processed your order and I’d like to confirm your product ID. It is A as in Alpha, 1, 2, 3, B as in Bravo, 5, 6, 7, Z as in Zulu, 8, 9, 0, X as in X-ray.“

I thought this was included in the demo, it seemed okay!

> Phone numbers and others were read nicely

The phone numbers were not naturally read at all. A human would have read a grouping of 123-456-789 like "123", "456", "789", but instead the model generated something like "123", "45", "6789". Listen to the RVSP example again and you'll know what I mean. The pacing is generally off for normal text too, but extra noticeable for the numbers.

My hunch would be that it's because of tokenization, but I wouldn't be able to say that's the issue for sure. Sounds like it though :)

Most of these prompts come from LLMs, so it's trivial to instruct them to provide a string that's broken out like that.

Also not the end of the world to process stuff like this with a regex.

Most of these newer TTS models require this type of formatting to reliably state long strings of numbers and IDs

hume.ai specializes in expressive prosody for TTS (disclaimer - I work here)

Azure speech services?

There's "Reader" by Eleven Labs. Now with iphone mirroring, you can use it without much trouble from your laptop too

Did we watch the same video? OpenAI's model is faster, and the quality is far better.

It depends on the use case. If you are looking for a stable model with great voices and very low latency, Play 3.0 mini is as good as or better than 11labs. https://x.com/_mfelfel/status/1846025183993511965/photo/1

If you go by hugging face leaderboards then no.

https://huggingface.co/spaces/TTS-AGI/TTS-Arena

Yes. I think $0.05/min is a high multiple of what other agent-oriented realtime TTS products are charging.

This is similar: https://deepgram.com/agent

It should do pretty good. You can try it here https://play.ht/playground/