Corrected read production very slow

[chklopp]

I'm running DeChat on 50x of a 2Gb genome and the first steps ran quite quickly but the read extraction is very slow : 10G in 12 hours with 32 cpus. How could this be accelerated?

[LuoGroup2023]

I apologize, but there is currently no better way to accelerate this step.

The earlier stages run quickly because constructing the de Bruijn Graph (dBG) doesn't consume much time. However, the "read extraction" stage you mentioned is actually the process of performing parallel error correction on every single read. The speed here is strictly bottlenecked by the read lengths and the overall size of your FASTA file, making further acceleration highly difficult.

As far as I know, with the exception of hifiasm's built-in correction module, DeChat's speed still outperforms other standalone error correction tools currently available.

I am very sorry for the inconvenience!

[chklopp]

Thank you for you answer.

I tried twice to correcte R9.4.1 reads and twice I got a core dump not after the same number of corrected reads.

malloc(): invalid size (unsorted)
/var/spool/slurmd/job36022491/slurm_script: line 10: 1781792 Aborted (core dumped) dechat -i reads_sup.fastq -o reads_dechat -t 32

Is there a way to run only the correction step once we have the reads h5 file?
Would it be possible run in parallel several corrections on the same h5 file starting and stopping a different read ranks?