update to django 6; remove celery in favor of django-tasks

[Copilot]

Migration from Celery to django-tasks

Completed:

• Rebased off latest main branch
• Removed Celery dependencies (celery, billiard, kombu)
• Added django-tasks==0.9.0 and APScheduler==3.11.2
• Configured django_tasks.backends.database.DatabaseBackend in base.py
• Configured ImmediateBackend for dev.py
• Converted all @shared_task to @task decorators (from django_tasks)
• Updated task execution (.delay() → .enqueue(), .apply_async() → .enqueue(run_after=...))
• Created chained task wrappers (update_hero_metadata_chain, replace_and_update_hero_chain)
• Created scheduled tasks with APScheduler (hawc/main/crontasks.py)
• Updated Docker compose healthchecks and scripts (db_worker for workers, run_scheduler for cron)
• Removed celery files (hawc/main/celery.py, run_celery.py)
• Added TODO comments documenting django-crontask incompatibility

Implementation Details:

Dependencies:

• Using django-tasks==0.9.0 for @task decorator and DatabaseBackend
• Using APScheduler==3.11.2 for scheduled tasks

Settings:

• base.py: TASKS configured with django_tasks.backends.database.DatabaseBackend
• dev.py: Overrides to use django_tasks.backends.immediate.ImmediateBackend for synchronous execution

Scheduled Tasks:

• Using APScheduler with .func attribute to call underlying task functions
• Management command: python manage.py run_scheduler
• Docker: workers run db_worker, cron runs run_scheduler

Known Limitation:
django-crontask package is incompatible with django-tasks library. django-crontask requires Django 6's native @task decorator, but Django 6 only includes immediate and dummy backends (no database backend). The django-tasks library provides the DatabaseBackend but uses a different Task class signature. Using APScheduler as an interim solution until compatibility is resolved. TODO comments have been added in hawc/main/settings/base.py and hawc/main/crontasks.py to document this limitation.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Remove celery; switch to django-tasks</issue_title>
<issue_description>Remove celery.

Add two new packages, django-tasks and django-crontask to replace the celery requirements needs.

Use the django_tasks.backends.database.DatabaseBackend</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Remove celery; switch to django-tasks #1263

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[shapiromatron]

1. Rebase this pull request off the latest version of main
2. Update django to version 6.0 or higher
3. Use the django task immediate backend for dev, but in the base.py settings, use the database backend for tasks
4. Use the django-crontask package for cron tasks, instead of manually adding information using APScheduler

[Copilot]

1. Rebase this pull request off the latest version of main
2. Update django to version 6.0 or higher
3. Use the django task immediate backend for dev, but in the base.py settings, use the database bac...

I've completed most of the requested changes:

1. ✅ Rebased to latest main
2. ✅ Updated Django to 6.0.1
3. ✅ Configured immediate backend for dev.py and attempted database backend for base.py
4. ✅ Added django-crontask package

However, there's a compatibility issue: Django 6's native task framework only includes immediate and dummy backends - there's no built-in database backend. The django-crontask package requires Django 6's native @task decorator, but the django_tasks library (which provides a DatabaseBackend) uses a different Task class signature that's incompatible with django-crontask.

Could you clarify which approach you'd prefer:

• Use django_tasks library (0.9.0) for DatabaseBackend but without django-crontask (use APScheduler instead)?
• Use Django 6 native tasks with only the immediate backend?
• Is there a different package that provides a database backend compatible with Django 6's native tasks?

All code changes are complete except for resolving this compatibility issue.

[shapiromatron]

@copilot 1. use the database backend from the django-tasks pypi package.
2. put a TODO berside the incompatabilities

[Copilot]

@copilot 1. use the database backend from the django-tasks pypi package.
2. put a TODO berside the incompatabilities

Done in commit 443b5a4. Added TODO comments in:

1. hawc/main/settings/base.py - Documents why django-tasks library's DatabaseBackend is used
2. hawc/main/crontasks.py - Explains the django-crontask incompatibility and why APScheduler is used instead

The implementation uses django-tasks library (0.9.0) for the database backend as requested, with TODO notes explaining the incompatibility with django-crontask.