Skip to content

About

Distributed background job orchestration for .NET built for seamless horizontal scaling and multi-provider support

Topics

Resources

Stars

123 stars

Watchers

1 watching

Forks

Repository files navigation

JobMaster

Distributed job orchestration engine for .NET. Built for horizontal scale, designed for resilience.

NuGet (pre) Unit Tests Sponsor

📖 docs.jobmaster.hugoj0s3.dev


Benchmarks

50k-job bursts, 20 workers, real HTTP scheduling (not in-process microbenchmarks):

Config Scheduling throughput
RavenDB + NATS 23.8k jobs/sec
RavenDB (single DB)* 6.5k jobs/sec
PostgreSQL 6.1k jobs/sec
MySQL 4.4k jobs/sec

*RavenDB ran at half the CPU/memory of the other engines (a Community Edition license cap) and still led the field.

Full methodology, resource limits, and detailed results: docs.jobmaster.hugoj0s3.dev/docs/benchmarks/results.


Quick Start

Standalone is the simplest way to run JobMaster. A single database connection handles coordination, job storage, and the transport layer — no additional brokers required.

Register in Program.cs

builder.Services.AddJobMasterCluster(config =>
{
    config.UseStandaloneCluster()
          .ClusterId("Local-Cluster-01")
          .UsePostgres("Host=localhost;Database=jobmaster_db;Username=postgres;Password=pwd")
          .AddWorker();
});

var app = builder.Build();

await app.Services.StartJobMasterRuntimeAsync();

Implement a Job Handler

public sealed class HelloJobHandler : IJobMasterHandler
{
    public async Task HandleAsync(JobContext job)
    {
        var name = job.MsgData.TryGetStringValue("Name") ?? "World";
        Console.WriteLine($"Hello {name}");
        await Task.CompletedTask;
    }
}

Handlers are resolved from the .NET DI container — inject your services (repositories, HTTP clients, etc.) directly into the constructor.

Schedule a Job

IJobMasterScheduler is registered automatically. Inject it anywhere in your application.

app.MapPost("/schedule-job", async (IJobMasterScheduler jobScheduler) =>
{
    var msg = WriteableMessageData.New().SetStringValue("Name", "John Doe");

    await jobScheduler.OnceNowAsync<HelloJobHandler>(msg);

    return Results.Accepted();
}).WithOpenApi();

Core Concepts

JobMaster separates responsibilities into three layers:

  • Cluster Database (Master) — source of truth. Stores jobs, coordinates agents, and persists configuration.
  • Agents (Transport Layer) — ephemeral, high-speed buffers for in-flight jobs. Supports PostgreSQL, MySQL, SQL Server, and NATS JetStream.
  • Workers (Execution Layer) — claim and execute jobs using atomic locks. Scale horizontally with zero downtime.

Recurring Schedules

JobMaster supports recurrence expressions using the NaturalCron library.

// Fluent builder
var expression = NaturalCronBuilder.Every(1).Minutes().Build();
await jobScheduler.RecurringAsync<HelloJobHandler>(expression);

// Expression string
await jobScheduler.RecurringAsync<HelloJobHandler>(NaturalCronExprCompiler.TypeId, "every 1 minutes");

Dashboard & API

JobMaster ships a browser-based dashboard and a REST API for monitoring clusters, jobs, workers, buckets, and agent connections in real time.

🚀 Live sandbox — sign in with GitHub and explore a realistic multi-cluster deployment.

dotnet add package JobMaster.Api
dotnet add package JobMaster.Dashboard

Both can run in a completely separate process from your workers — all they need is access to the master database.


Documentation

Full documentation is available at docs.jobmaster.hugoj0s3.dev:


Roadmap

See docs.jobmaster.hugoj0s3.dev/docs/roadmap.

About

Distributed background job orchestration for .NET built for seamless horizontal scaling and multi-provider support

Topics

Resources

Stars

123 stars

Watchers

1 watching

Forks

Releases

Contributors

Languages