Skip to main content

Subtitle Sync

The Subtitle Sync task fixes subtitles that are out of sync with the audio.

It uses ffsubsync to detect when speech occurs in the reference audio, then finds the timing shift that best lines the subtitle cues up with that speech. It corrects both constant offsets and framerate mismatches (for example subtitles timed for 25 fps on a 23.976 fps video).

SRT, ASS and SSA subtitles are supported. Subtitles are returned in the same format they were sent in, so ASS/SSA styling such as fonts, colours and positioning is kept.

This task is used by the Subtitle Sync flow element. The Agent only uploads a small mono speech track extracted from the video, not the video itself.

Configuration Settings​

Concurrent Tasks​

  • Description: The maximum number of subtitle sync tasks that can run simultaneously.
  • Usage: Syncing is light on resources and usually takes seconds to a minute per subtitle, so the default of 2 suits most systems.

Voice Activity Detector​

  • Description: The detector used to find speech in the reference audio.
  • Options:
    • WebRTC (default): Fast and works well for most content.
    • Auditok: An alternative energy-based detector that can help with some audio where WebRTC struggles.
    • Silero: More accurate on noisy audio, such as dialogue under loud music or effects, but slower. Downloads its model on first use.

Default Max Offset (seconds)​

  • Description: The maximum number of seconds a subtitle may be shifted when the request does not specify a value.
  • Range: 5 to 600.
  • Default: 60.

Verbose Debug Output​

  • Description: Logs the synchronized subtitle text in the task log for troubleshooting.

Limitations​

  • Only constant offsets and framerate differences are corrected. Subtitles that drift unevenly, such as subtitles made for a different cut of the video, cannot be fixed.
  • The reference audio should contain the dialogue the subtitles were written for. Commentary tracks will give poor results.