Hello guys,

I have a song audio clip that I need to play in sync with its lyrics. The approach I'm using right now is for the lyrics, I have a bink animation created in after effects and exported to my custom engine format. And I try to synchronize the animation by setting its current frame based on the current position of the audio.  But I'm facing issues with synchronization. Could you give any suggestions?

One way I'm thinking is to try and seek the audio to the position determined by the current frame of the animation. This approach again might not work and it would be more noticeable if there is an issue while syncing. Another approach I'm thinking is maybe use a text file that acts as a sort of the subtitle file which I parse and display. But I'm not sure how to get this voice recognition. Are there existing libraries that process the audio file and let you know what is being said? It'd essentially be implementing an audio subtitle system. Any suggestions on that?

If the main aim is to "play some music" seeking, or otherwise messing with it is likely to be a bad idea, as the seeking will cause noticeable audio "glitches". You could possibly vary the playback rate very slightly, or the speed of the animation.

It is not clear what issue you are having exactly? Is the animation always behind the song? Is the animation always ahead? Is the animation stuttering?

To get very precise synchronisation, you also have to accout for the latency in audio playback and video display.

So far, I have only noticed the animation seems to be ahead. This is noticeable only when there is a considerable frame rate drop. If the game is in idle mode, it works fine.