Pika Labs, the substitute intelligence video platform, makes its sound results function out there to all customers. It permits creators to simply add sound to any video, whether or not earlier than or after technology.
First launched to Professional plan subscribers, the sound results come from an inside Pika Labs text-to-audio conversion mannequin that makes an attempt to match the textual content of the video message.
This takes AI video to a brand new dimension, increasing on the character voice and lip sync instruments beforehand added to the Pika Labs platform.
Different gamers are additionally experimenting in AI video with sound house. ElevenLabs has its personal sound results library and Runway has a restricted launch lip sync function.
Placing Pika Labs sound results to the check
There are two methods so as to add sound results to a video inside Pika Laboratoriesas a part of the preliminary message when toggling the sound results swap, or after the very fact with a separate message.
I attempted each for this hands-on assessment and located that except you must choose the sound, akin to specifying a time of 12 months or a particular noise, the primary method offers higher outcomes.
1. The bar at Christmas
That is based mostly on a picture I created for a earlier story concerning the Suno AI music app, depicting a bar on the moon at Christmas. I left the textual content message clean and turned on the Sound Results swap to see how properly it might match the picture.
He did a surprisingly good job. Just a little noisy, however so are a number of the finest bars. He did not actually seize the Christmas aspect, however the one indication was a tree within the nook.
2. The supercar
For the primary check, I gave Pika Labs the next message and enabled sound results, permitting the mannequin to observe each the message and the video: “A glossy crimson sports activities automotive racing down a metropolis road at night time , leaving a path of movement blur behind him. .”
It captured movement properly and generated a sound impact of an engine revving, but it surely was surprisingly quiet. I needed to enhance the sound in iMovie to listen to it correctly, however general it sounded good.
3. Splash in a pool
This is named a splash check. How properly it not solely handles water sounds, but in addition the way it does when synced to video. I used the message: “An individual within the air, silhouetted in opposition to a transparent blue sky, about to pe right into a glowing pool.”
It did not observe the message notably properly by way of photographs, though it kind of resembled the traces of the message. She made the character look extra like a mermaid than an individual, however the splashing sound was good as have been different common water noises.
4. The busy kitchen
I used to be actually impressed with this picture I generated on MidJourney. It virtually appeared fully out of focus, which was spectacular for an AI mannequin, but it surely has simply the fitting deal with the meals, placing the cooks within the background.
This made it the proper candidate to check out the Pika Labs sound results. I merely gave it the picture and chosen the sound results after which hit the generate button. He captured the motion properly and created appears like a kitchen at work.
5. The fireworks present
The ultimate check was one I’ve utilized in each sound results check up to now: fireworks. It must be simple, however the problem is timing the sound with the explosion.
I used the immediate: “A vibrant explosion of fireworks illuminating an evening sky, with multi-colored sparks radiating outward.” It did not do a foul job contemplating there was no handbook synchronization, every thing was executed by the AI mannequin.
Conclusion
Sound results add an necessary dimension to AI video content material; the truth is, they add an necessary dimension to any video content material.
You possibly can virtually hear the sound of a automotive revving its engines simply by trying on the picture, so it is important that the sounds will not be solely appropriate however timed appropriately.
That is one thing Pika Labs appears to have achieved with its software. It isn’t excellent, it does not all the time get you precisely the noise you need, but it surely’s designed with sufficient flexibility to make use of it as a throwaway thought or for a full, orchestrated manufacturing.
- I attempted the brand new AI music software Suno – that is what it appears like
- Runway vs Pika Labs: which is the very best AI video software?
- Right here Comes the Holodeck: New AI 3D Video Mannequin Creates Animations of Any Object
