The pull request #4227 fixed a hardware overload situation but brought to larger questions regarding the logging lock mechanisms involved within the project:
-
Should we parameterize the timeout there? If we update log_line to take a timeout argument (defaulting to 10) and pass it to _acquire_lock, it would make the system more flexible. That way, users on very slow hardware can increase it themselves, and we won't have to keep raising the default hard limit (e.g., to 30 or higher). (comment from @YongxueHong)
-
The issue seems to be caused by all the logging procedures in this module sharing one single lock (the global _log_lock instance) and hence it can not result in good performance upon big/large scale use cases or slow speed log storage. All of this needs more refactoring. (comment from @luckyh)
I am therefore creating this issue to track progress on simplifying the entire log lock mechanism - from making it more configurable to users to eventually making it more compatible with parallel VT tests that some of us are already running.
The pull request #4227 fixed a hardware overload situation but brought to larger questions regarding the logging lock mechanisms involved within the project:
Should we parameterize the timeout there? If we update
log_lineto take a timeout argument (defaulting to 10) and pass it to_acquire_lock, it would make the system more flexible. That way, users on very slow hardware can increase it themselves, and we won't have to keep raising the default hard limit (e.g., to 30 or higher). (comment from @YongxueHong)The issue seems to be caused by all the logging procedures in this module sharing one single lock (the global
_log_lockinstance) and hence it can not result in good performance upon big/large scale use cases or slow speed log storage. All of this needs more refactoring. (comment from @luckyh)I am therefore creating this issue to track progress on simplifying the entire log lock mechanism - from making it more configurable to users to eventually making it more compatible with parallel VT tests that some of us are already running.