Add log rotation support

[bhamiltoncx]

I think we ought to rotate this log once in a while. Mine is 202 MiB..

-rw------- 1 bgertzfield THEFACEBOOK\Domain Users 202409754 Oct 7 08:57 .watchman.bgertzfield.log

[ghost]

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

[wez]

This is doable in most common cases; if we were passed a log file name argument, then we can periodically check the size and rotate.

[qmaruf]

I am interested to work on this feature. @wez Could you please suggest me from where should I start? I've built _watchman_ on my machine.

[wez]

@qmaruf yep, the high level idea is that we'd want to rotate the log file. This is typically implemented in one of two ways:

1. External software that manages log files can rename the current log file to a new name, then send some kind of signal to the service to tell it to re-open its log file
2. Modify the service itself so that it has some kind of rotation policy.

We can support both of these things. The high level plan is:

1. Add a function that can reopen the log file. This is logically as simple as closing the old log file handle and then opening the configured log file again. The existing code for opening the log file is towards the top of main.c; it should be moved into log.c. Care should be taken to ensure that we don't lose log events while making this switch.
2. Add a watchman command that will trigger the log file to be re-opened. This will facilitate controlling log file rotation via software like logrotate; it would just run watchman log-reopen
3. Add a configuration option that specifies:
   1. how often to check the log file size in seconds
   2. the maximum size
4. If size checking is enabled (the checking interval is non-zero), every interval seconds, compare the size. If the size is larger than desired, rename the log file by appending the current unix timestamp to the log file name and then reopen the log file.

I consider the following things to be non-goals for watchman; our focus is on answering questions about the filesystem rather than re-implementing features that exist in other widely deployed software:

1. Configuring an alternative or more flexible way of naming the moved log files
2. Limiting the number of re-opened log files (I'm on the fence about this, but am leaning towards saying that the user can script cleaning this up for themselves, outside of watchman. Other software exists with the sole purpose of performing this kind of cleanup)

[qmaruf]

Is there any way to print something in console using _printf()_ while watchman is running, for debugging purpose? I am using _watchman --server-encoding=json --persistent log-level debug_, but it only prints log info. I need to understand the sequence of execution.

[wez]

you can use w_log(W_LOG_ERR, "format %s\n", "foo") to log to the log file at ERR level.
You can run: watchman --foreground --logfile=/dev/stdout to run in the foreground and have the log file go to your terminal. You can use the -v option to change the default log level.

[qmaruf]

@wez What does watchman do after w_event_set()? AFAIU, w_event_set() is called after enqueuing log message in client queue. I am looking for the code where client is dequeued and log message is written to file.

[wez]

I'm not sure I understand your question. The client responses are de-queued dispatched here: https://github.com/facebook/watchman/blob/master/listener.c#L495 but that is completely unrelated to the log file related functions that are in log.c

[qmaruf]

I thought message is being written to log file after dequeuing from client. I planned to check log size before writing to it so that we could avoid checking its size continuously whenever we don't need to .

[oxalica]

Is this feature still planned to be added? I'm on Linux and noticed one day that /tmp/<user>-state/log is consuming more than 1.6GB of my disk. It contains logs up to more than one years ago. I guess it's because I'm using nvim/coc.nvim daily which depends on watchman, thus it keeps updating the last modified time of the log file, prevent it to be deleted by the temp files cleaner (systemd-tmpfiles-clean). Is there any workaround to prevent it from growing indefinitely?