Kyutai’s Moshi just pushed the boundaries of conversational AI.
Moshi is a new AI-powered chatbot developed by the French AI company Kyutai, which they claim to have developed from scratch within just 6 months! The team publicly unveiled its experimental prototype on July 3rd in Paris. Judging by the stats provided by Kyutai, it seems to be a direct competitor to the upcoming GPT-4o from OpenAI.
Here is where where you can queue up to talk to the Moshi chatbot.
Through my testing, I’ve noticed that the current model does not have the same low latency as the model that was demonstrated live during the unveiling. This is expected to improve in the weeks leading on from the release, so make sure to stay put.
Some of Moshi’s key features include:
- Real-Time Voice Interaction: Moshi can understand tone of voice, interrupt conversations, and respond in real-time, with a response time of just 200 milliseconds. This is faster than the upcoming GPT-4o’s “Advanced Voice Mode” from OpenAI.
- Emotional and Stylistic Speech: Moshi can speak in 70 different emotional and speaking styles, allowing it to modulate its voice to convey various tones and emotions. This is powered by its text-to-speech engine that was refined using human-recorded audio data.
- Multimodal Capabilities: Moshi can process both audio and text simultaneously, allowing for natural back-and-forth conversations where it can listen and respond verbally. It has a unique two-channel system for processing both modalities.
A multimodal asynchronous input and output model can receive and process different types of data (like text and images) at different times. It doesn’t need all the data at once and can handle each piece of information as it arrives, providing results when each piece of processing is complete.
For example, such a model could receive a photo and some text separately, process them independently, and then generate a description of the photo based on the text, even if the text and photo were provided at different times.
- Open-Source Approach: Kyutai plans to make Moshi an open-source project, releasing the model’s code and framework so that users can use it safely without privacy concerns. This aims to democratise access to advanced AI technology.
- Offline Capabilities: Unlike cloud-based AI assistants, Moshi can run locally on devices without needing to connect to a server, improving privacy and latency.
- Responsible Development: Kyutai is incorporating audio watermarking and identification systems into Moshi to help address issues of authenticity and misinformation around AI-generated content.
How will Moshi change our workplaces?
Moshi’s ability to engage in natural, back-and-forth conversations using voice could revolutionise how we interact with AI assistants in the workplace. Instead of typing commands, we could simply speak to Moshi and receive instant, contextual responses.
Moshi can recognise and respond to the emotional tone of voice could make interactions with AI more empathetic and personalised. This could be particularly useful for customer service, counseling, or other roles that require emotional awareness.
Moshi’s range of accents and speaking styles could help break down language barriers in global organizations. Workers could converse with Moshi in their preferred manner, whether that’s a formal business tone or a more casual, conversational style.
Lastly, the cherry on the cake, the open-source nature of Moshi could accelerate the integration of voice AI into a wide range of workplace applications, leading to faster innovation cycles and quicker time-to-market for new products and services.
Here’s to an exciting new world of real-time conversational AI.
