Baidu’s Wizshot ME turns portraits and voice references into short videos

Baidu introduced ERNIE-Wizshot-ME, a model for generating short videos of people speaking, at Baidu World on November 12. It opened an API for the model on its Qianfan cloud platform. Qianfan’s product page lists 1080p output, clips lasting four to 15 seconds, and a price of 0.7 yuan per second of video.

Production teams can supply a portrait, a voice reference, dialogue and directions for the performance to make a close-up of a person speaking. Baidu is targeting moments such as a glance to the side or a change in expression around the eyes or mouth. Its product page lists podcast segments, close-ups in short dramas and storytelling as examples. The video model is a separate offering from the ERNIE 5.5 Flash Preview language model shown at the same event.

A portrait and voice reference establish the character

Qianfan’s request documentation separates the cast from the scenes. Each character has an image showing their face and a WAV file containing a human voice. The image can be a JPEG, PNG or WebP. The voice reference must run for 10 to 60 seconds. The portrait must be between 300 and 5,000 pixels on each side. The short edge of the separate shot reference must be at least 720 pixels. The words heard in the generated scene are specified separately as dialogue.

For each shot, the production tool can supply a reference image and describe the camera position, the people on screen and who speaks. Written performance directions can call for a small nod or a shift in gaze alongside a particular line. The documentation also describes how to indicate a change in pace, tone or emotion. A Baidu example specifies a brief interjection from a second person during a scene. A character image establishes who appears, while a shot image gives the model a reference for the composition. A request can contain up to 10 shots in sequence. Baidu recommends four or fewer.

The server measures the spoken duration

Baidu lists four to 15 seconds as the product’s video output range. The API documentation gives a separate rule for submitted dialogue. The server measures the synthesized speech and any specified silence together. A request fails when that total falls outside four to 16 seconds.

The guide estimates that 20 to 60 Chinese characters or 13 to 37 English words will fill roughly five to 15 seconds. Those estimates assume typical speaking speeds. Punctuation adds pauses, and a number or abbreviation may take longer to say than its written length suggests. Silent shots count toward the measured duration and go in a separate shot from spoken dialogue. The API also limits each dialogue segment to 450 bytes.

A separate request returns the finished video

Qianfan creates a job when it accepts the submission and gives the caller an identifier. The production tool uses that identifier to check whether generation is pending, running, successful or failed. The model registry lists one concurrent job and one queue slot. Those limits matter when a team submits several scenes in succession. The API documentation gives no fixed turnaround time.

Baidu’s billing documentation lists a charge of 0.7 yuan per second of generated video. Qianfan’s five-second example costs 3.5 yuan. At the same rate, a 15-second clip costs 10.5 yuan. The response for a finished job can include the video’s duration and resolution. The billing unit is the length of the delivered video.

The completion response supplies a URL for the video. Baidu’s job documentation says the URL is valid for 24 hours and the service keeps the video for up to 30 days. The file can be saved with the rest of a production’s assets during the download window. Forwarding the URL leaves the recipient with the same expiry.