Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit test for each queue jobs class #9899

Open
43 of 46 tasks
touhidurabir opened this issue Apr 19, 2024 · 22 comments
Open
43 of 46 tasks

Add unit test for each queue jobs class #9899

touhidurabir opened this issue Apr 19, 2024 · 22 comments
Assignees
Labels
Housekeeping:2:Urgent Any dependency management or refactor that must be done soon to fix or avoid a significant problem.
Milestone

Comments

@touhidurabir
Copy link
Member

touhidurabir commented Apr 19, 2024

Describe the problem
Starting from 3.4.0 we have queue jobs to handle long running tasks or for tasks that need to done later . Our queue jobs implementation is an extended implementation of Laravel queue jobs.

The problem arise or will arise when any of the jobs structure will update but there is one or more pending jobs with older structure already dispatched in queue waiting to get executed . This problem get more complex with the upgrade of newer version when older structure based job still in queue as they may continue to fail and no way to resolve it after upgrade which may lead to undesirable outcome.

Describe the proposal
The initial plan was to force to run all pending queue jobs before upgrade which proposed at Add pre-flight check for pending jobs before running upgrade . This was a very simple and easy approach but may not suited for all possible cases . Also a job class structure or handle implementation may need to update based on requirement.

After some detailed discussion, we decided to add unit tests for each job class to make sure the consistence update the job class structure and handle method implementation . The idea is to try to mock each of jobs class to make sure it's on even after any update to it.

Impacted version
OJS, OMP or OPS version 3.4.0 and main

Additional information
Details discussion at Right approach to handle pending queue jobs at upgrade

Job classes to have unit tests

Stable-3.4.0

OJS
  • APP\jobs\doi\DepositIssue
  • APP\jobs\notifications\IssuePublishedNotifyUsers
  • APP\jobs\notifications\OpenAccessMailUsers
  • APP\jobs\statistics\CompileCounterSubmissionDailyMetrics
  • APP\jobs\statistics\CompileCounterSubmissionInstitutionDailyMetrics
  • APP\jobs\statistics\CompileIssueMetrics
  • APP\jobs\statistics\CompileSubmissionGeoDailyMetrics
  • APP\jobs\statistics\CompileUniqueInvestigations
  • APP\jobs\statistics\CompileUniqueRequests
  • APP\jobs\statistics\CompileUsageStatsFromTemporaryRecords [deprecated 3.4.0.5]
  • APP\jobs\statistics\DeleteUsageStatsTemporaryRecords
  • APP\jobs\statistics\ProcessUsageStatsLogFile
OMP
  • APP\jobs\statistics\CompileCounterSubmissionDailyMetrics
  • APP\jobs\statistics\CompileCounterSubmissionInstitutionDailyMetrics
  • APP\jobs\statistics\CompileSeriesMetrics
  • APP\jobs\statistics\CompileSubmissionGeoDailyMetrics
  • APP\jobs\statistics\CompileUniqueInvestigations
  • APP\jobs\statistics\CompileUniqueRequests
  • APP\jobs\statistics\CompileUsageStatsFromTemporaryRecords [deprecated 3.4.0.5]
  • APP\jobs\statistics\DeleteUsageStatsTemporaryRecords
  • APP\jobs\statistics\ProcessUsageStatsLogFile
OPS
  • APP\jobs\statistics\CompileCounterSubmissionDailyMetrics
  • APP\jobs\statistics\CompileCounterSubmissionInstitutionDailyMetrics
  • APP\jobs\statistics\CompileSubmissionGeoDailyMetrics
  • APP\jobs\statistics\CompileUniqueInvestigations
  • APP\jobs\statistics\CompileUniqueRequests
  • APP\jobs\statistics\CompileUsageStatsFromTemporaryRecords [deprecated 3.4.0.5]
  • APP\jobs\statistics\DeleteUsageStatsTemporaryRecords
  • APP\jobs\statistics\ProcessUsageStatsLogFile
PKP-LIB
  • PKP\jobs\bulk\BulkEmailSender
  • PKP\jobs\doi\DepositContext
  • PKP\jobs\doi\DepositSubmission
  • PKP\jobs\email\EditorialReminder
  • PKP\jobs\metadata\BatchMetadataChangedJob
  • PKP\jobs\metadata\MetadataChangedJob
  • PKP\jobs\notifications\NewAnnouncementNotifyUsers
  • PKP\jobs\notifications\StatisticsReportMail
  • PKP\jobs\notifications\StatisticsReportNotify
  • PKP\jobs\statistics\ArchiveUsageStatsLogFile
  • PKP\jobs\statistics\CompileContextMetrics
  • PKP\jobs\statistics\CompileMonthlyMetrics
  • PKP\jobs\statistics\CompileSubmissionMetrics
  • PKP\jobs\statistics\RemoveDoubleClicks
  • PKP\jobs\submissions\RemoveSubmissionFileFromSearchIndexJob
  • PKP\jobs\submissions\RemoveSubmissionFromSearchIndexJob
  • PKP\jobs\submissions\UpdateSubmissionSearchJob

PRs (Merged)
pkp-lib --> #9981
ojs --> pkp/ojs#4285
omp --> pkp/omp#1666
ops --> pkp/ops#745

PR Patch
pkp-lib --> #10368
ojs --> pkp/ojs#4426
ops --> pkp/ops#763
omp --> pkp/omp#1693

Need to merge the PR for issue pkp/pkp-github-actions#68

@touhidurabir touhidurabir added the Housekeeping:2:Urgent Any dependency management or refactor that must be done soon to fix or avoid a significant problem. label Apr 19, 2024
@touhidurabir touhidurabir added this to the 3.4.0-6 milestone Apr 19, 2024
@touhidurabir touhidurabir modified the milestones: 3.4.0-6, 3.4.0-x Apr 19, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 23, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 30, 2024
…d PKP\jobs\bulk\BulkEmailSender along with mockery package version update
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue May 30, 2024
…omSearchIndexJob and PKP\jobs\submissions\RemoveSubmissionFileFromSearchIndexJob
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Jun 6, 2024
@touhidurabir touhidurabir self-assigned this Jun 9, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 9, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
…d PKP\jobs\bulk\BulkEmailSender along with mockery package version update
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
…omSearchIndexJob and PKP\jobs\submissions\RemoveSubmissionFileFromSearchIndexJob
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Jun 13, 2024
@touhidurabir
Copy link
Member Author

touhidurabir commented Nov 8, 2024

@touhidurabir, I wasn't able to reproduce my earlier errors with the stable-3_4_0 PRs -- so please go ahead and merge! I think you'll need a rebase on at least the OJS PR, but otherwise it looks good to me.

@asmecher Thanks for the confirmation . I have rebased but I can not merge it as it's for stable 3.4.0 which required 2 approval to merge . can you please merge the PRs at #9899 (comment), see the section PR Patch ? But make sure to merge the PR from issue pkp/pkp-github-actions#68 first as otherwise no unit tests for job will run .

asmecher pushed a commit that referenced this issue Nov 8, 2024
* #9899 job unit tests patch fix

* #9899 Updated StatisticsReportNotifyTest to have app specific serialized data

* #9899 Updated StatisticsReportNotifyTest to have OPS specific serialized data

* #9899 job mocking for submission deposition test

* #9899 updated mock class implementation for doi registration

* #9899 Added test skipping for OMP

* #9899 unused class import cleanup

* #9899 removed direct mocking of job class

* #9899 fixed PHPUnit deprecation warning

* #9899 notification mocks added

* #9899 added backup/restore for dao and container keys

* #9899 fixed usage stats file create/update issue for tests
asmecher pushed a commit to pkp/ojs that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ojs that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ojs that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ojs that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ojs that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ops that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ops that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ops that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ops that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ops that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/ops that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/omp that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/omp that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/omp that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/omp that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/omp that referenced this issue Nov 8, 2024
asmecher pushed a commit to pkp/omp that referenced this issue Nov 8, 2024
@asmecher
Copy link
Member

asmecher commented Nov 8, 2024

@touhidurabir, all merged -- I think this is ready for closing!

@touhidurabir
Copy link
Member Author

@touhidurabir, all merged -- I think this is ready for closing!

@asmecher I think we still have same issue . No unit tests for job is running . I can see that the PR pkp/pkp-github-actions#69 is merged and see the changes in main branch . However when tests are running via githnb action , for the unit test, command as follow

/home/runner/work/omp/omp/pkp-github-actions/.github/actions/run-unit-tests.sh -CcPpd

the jJ flag is missing and as a result no unit tests for jobs are running . see for stable-3.4.0 at https://github.com/pkp/ojs/actions/runs/11736982412/job/32697055043#step:3:3102 and also same for main branch (see at https://github.com/pkp/omp/actions/runs/11756112412/job/32751920366?pr=1685#step:3:2667 which is rebased few hours ago after the github action PR got merged) . Wondering if the github action being cached or something we are missing ?

@withanage any idea why the updated command not being used when running unit tests via github actions ?

@withanage
Copy link
Member

Hi @touhidurabir ,

Sorry, I missed the PR there. After merge, we have to release a newpkp-github-actions version, so that PKP applications , pick the new changes. I released a new version. Now your changes should be visible in newer runs.

@touhidurabir
Copy link
Member Author

@withanage Thanks a lot for releasing a new version . Now all unit tests are running .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Housekeeping:2:Urgent Any dependency management or refactor that must be done soon to fix or avoid a significant problem.
Projects
None yet
Development

No branches or pull requests

4 participants