I thought the community might enjoy this little experiment: a 20-second, eight-part scat test using Synthesizer V Studio Pro and Choir Voices – Collection 1, with each voice driving a separate animated face.
It’s part of a larger project I’m building: eight robot heads, each with an Android tablet for a face, which will eventually sing alongside me playing live accordion.
What you can hear—and see
The arrangement is SSAATTBB, with a separate vocal track for each singer. It begins with a quick relay of individual voices, then moves through overlapping phrases, exchanges between the higher and lower voices, and passages where everyone sings together.
The syllables are deliberately varied: “ooh”, “ah”, “ee”, “oh”, “ma”, “bah”, “pah”, “fee”, “vee” and “la”. Each singer gets to try every sound. This gives us different vocal colours while also testing rounded lips, open vowels, lip closures, lower-lip contact and tongue movement in the animation.
Watch the opening entries and the shared pauses: each face follows its own part, rather than all eight mouths responding to the volume of the combined recording.
The workflow
I’ve been collaborating with ChatGPT on the score, visual assets and animation prototype:
-
ChatGPT generated a roughly 20-second, eight-part MusicXML score, with the scat syllables written into the lyrics.
-
I imported it into Synthesizer V Studio, assigned voices from my purchased Choir collection and rendered the vocal mix.
-
I returned the recording to ChatGPT.
-
The score’s notes, lyrics and timing were used to build eight independent mouth-animation tracks, which were combined with the rendered audio.
The useful connection is the MusicXML: it tells us who sings what, and when. Synthesizer V supplies the vocal performance, while the same score provides the starting point for the facial cues.
We still estimate the exact consonant boundaries from the score, rather than using Synthesiser V’s phoneme timing. That is an area I’d like to improve.
Why the Choir collection is useful here
Although I’m building a choir, I also want its members to have individual characters. The collection’s solo voices, together with the Vocal Modes and voice parameters, give me scope to develop eight recognisable singers.
I’m now saving named presets for each character so that their voice settings can be recalled consistently across different pieces. The portraits are visual characters; I’m shaping the voices from the Choir collection rather than attempting to reproduce the famous singers depicted.
It certainly isn’t going to win a Grammy, but I was pleasantly surprised by how musical this first test sounded!
What comes next
The clip is currently a browser-rendered animation prototype. My robot-face system uses Rive and Flutter, so the next stage is to bring this mouth behaviour into the Rive faces and run each singer on its own Android tablet.
Musically, I want to refine the eight voice presets, work on phrasing and pronunciation, and move from test syllables to proper repertoire. I’d also like a dependable workflow that keeps the vocal audio and facial cues together as an arrangement develops.
I’d particularly welcome advice on extracting or exporting the final phoneme timings from Synthesizer V Studio. Has anyone used those timings to drive facial animation, lip-sync or another external system? That could help us move from score-based mouth cues to something that follows the generated performance more closely.