[ideas] Add last executed target when job fails because of out-of-memory

[stemangiola]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

When I see this (for me) out of memory message

'errorValue' int 5 | Timed out
 A {mirai} dispatcher process is running at pid 3067866 with status "sleeping". There may be a network connection issue, or the dispatcher is simply busy managing lots of tasks or uploading/downloading large datasets. Maybe try increasing seconds_timeout in your {crew} controller.
 Please also try upgrading R packages {nanonext}, {mirai}, and {crew} to their latest versions on CRAN. (Likewise with {targets} if you are using it.) Upgrading these packages solves many kinds of errors. 
Another possibility is an out-of-memory error. The dispatcher process can run out of memory if it is overwhelmed with data objects too large or too many to comfortably fit inside a single R process. Please read https://wlandau.github.io/crew/articles/logging.html to learn about proactive resource usage logging which can provide useful data in the event of a crash. 
If memory consumption is a problem, then as a workaround, you could make each task save/load files instead of using/returning large R objects in memory. If you are using {targets}, you might consider storage = "worker", retrieval = "worker", and/or cloud storage as documented at https://books.ropensci.org/targets/performance.html and https://books.ropensci.org/targets/cloud-storage.html.

I would like to know the last executed target so I can see what resources I have to adjust.

[wlandau]

That error happens at the level of mirai (which crew uses) and so the target cannot be reported in this error message. I recommend either looking at tar_progress() or doing prospective logging with autometric if you want to find out what was running when you saw the error (c.f. https://wlandau.github.io/crew/articles/logging.html).