Skip to content

feat: Failure reporting and dead letter queue #55

@Snider

Description

@Snider

Summary

Track all failures and provide options for handling them.

Use Case

After a large collection, need to know what failed and why, and optionally retry just the failures.

Dead Letter Queue

Failed items go to a queue for later processing:

.borg-failures/
├── 2024-01-15T10-30-00/
│   ├── failures.json
│   └── retry.sh

Failure Report

{
  "collection": "github:repos:org",
  "started": "2024-01-15T10:30:00Z",
  "completed": "2024-01-15T11:45:00Z",
  "stats": {
    "total": 150,
    "success": 145,
    "failed": 5
  },
  "failures": [
    {
      "url": "https://github.com/org/deleted-repo",
      "error": "404 Not Found",
      "attempts": 3,
      "retryable": false
    },
    {
      "url": "https://github.com/org/large-repo",
      "error": "timeout after 300s",
      "attempts": 3,
      "retryable": true
    }
  ]
}

Commands

# Retry failures from a run
borg retry .borg-failures/2024-01-15T10-30-00/

# Show failure summary
borg failures show .borg-failures/2024-01-15T10-30-00/

# Retry only specific errors
borg retry .borg-failures/... --only-retryable

# Clear old failures
borg failures clear --older-than 7d

Options

Flag Description
--on-failure continue Log and continue (default)
--on-failure stop Stop on first failure
--on-failure prompt Ask user what to do
--failures-dir Custom failures directory

Acceptance Criteria

  • Track all failures with context
  • Generate retry scripts
  • Retry command for failures
  • Failure categorization (retryable vs permanent)
  • Failure summary command

Metadata

Metadata

Assignees

No one assigned

    Labels

    julesFor Jules AI to work on

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions