new whisperX feature [research]

[m-bain]

Please vote which is most useful to you. Also let me know if any feature is really important for a big usecase, e.g. your business, or downstream research project

ctx: i have a few weeks until i start new job so have some free time. To a prosperous 2025 🫶

[sulutian]

Currently, when identifying movies, some clips are not transcribed. Can this be fixed?

[Barabazs]

Are you using subtitle edit?

[sulutian]

I put the subtitle file into SubtitleEdit and found that it has less content compared to the outputs from faster-whisper and whisper.

[Barabazs]

It doesn't seem like your issue is caused by whisperX though?

[sulutian]

I'm not entirely sure, but I transcribed multiple videos using WhisperX, faster-whisper, and whisper, and found that WhisperX sometimes misses certain segments. I tried adjusting the VAD threshold, which seemed to help a bit, but some segments still didn't have subtitles.

[tukipona]

I'd really appreciate it if the numerals in transcriptions could be more precisely aligned. (As mentioned in Limitations)

[m-bain]

did you try --supress_numerals flag?

[7k50]

I agree strongly that numerals alignment, must be a highly desirable feature.

Here is someone's pull request proposing a solution: #986
I have tried it, at first glance it seems indeed to align numerals.

The lack of numerals alignment is a problem, since I parse .srt outputs through highlight_words=True with post-processing logic to reconstitute timestamped words into blocks of text, with timestamp URLs, etc. Trying to create custom logic around this and various edge cases, has been challenging. I would rather prefer rough timestamps on numerals in WhisperX, than none at all.

--suppress_numerals with a post-processing text normalizer, could have worked, if it didn't seem to give so peculiar results, e.g. with text2num:

Unemployment now stands at seven point two million, up three hundred and fifty two thousand.
Unemployment now stands at 7.2000000, up 300 and 52000.