Dependent job scheduling with Quartz.NET

• Download QuartzDependentJobScheduling.zip - 7.7 MB

Introduction

One way schedule jobs using Quartz.NET that are dependent upon each other and require specific sequential execution.

https://www.quartz-scheduler.net/

Background

There are some tutorials that provide ways of executing jobs sequentially however they don't make clear the notion of Durable vrs Non-Durable in Quartz. Quartz defines jobs that are non-durable having an life span tied to the existence of it's triggers. In the case of how the below sequential execution works, the only trigger exists on first exeuction so the trouble I ran into was knowing how to define the other dependent jobs as Durable. This allows the implementation of the job via Windows Service to execute all jobs sequentially instead of just the first scheduled job.

A few things I hope to address that were issues with my learning Quartz are

• Setting up jobs to execute beyond the first execution
• After the last sequentially executed job runs, run a unspecified number of jobs concurrently.

The situation I'll present here is comprised of 3 steps. 1) You need to charge a bunch of pending payments via a scheduled job. 2) Then once all payments are charged, you generate a report of all payments that were charged and upload it somewhere. 3) Finally, once the payments have been charged, batch report of payments successfully charged has been generated, send email and text messages updating the customers that their cards have been charged.

I'm not actually going to include code to charge payments, generate reports, or email/text others, more the situation of how 2) can only run once 1) has completed and then fire off multiple jobs in 3) that only depend on 2) having completed.

Using the code

The portions of this example can be broken down into

• Job classes
• JobListener classes
• The scheduler itself

Quartz.net can be implemented using their server/enterprise service functionality but for the purpose of this example it will just be a console app. You can easily apply the same code to a windows service also.

Jobs

Declare what jobs you may need to execute in your application. For our purposes. Going to create a ChargePaymentsJob, BatchFileUploadJob, TextMessageJob, and EmailJob. ChargePayments must run first, then BatchFileUploadJob. Once BatchFileUploadJob has completed, we can let Quartz.NET fire off the TextMessageJob and EmailJob at the same time as neither of these jobs depend on the other in order to complete.

All of your jobs need to inherit from IJob interface in order to let the scheduler know that it is something that the scheduler should execute.

public class EmailJob : IJob
{
    public void Execute(IJobExecutionContext context)
    {
        // Do something here to hit the database and get a list of payments that were charged, then loop over those payments to send an email to the customer
        //   indicating their payment was successfully charged for customers who want notifications via email.
        for (int i = 0; i < 3; i++)
        {
            Console.WriteLine("Email sent to customer {0} payment was charged successfully", i);
        }
    }
}

public class TextMessageJob : IJob
{
    public void Execute(IJobExecutionContext context)
    {
        // Do something here to hit the database and get a list of payments that were charged, then loop over those payments to send a text message to the customer
        //   indicating their payment was successfully charged for customers who want notifications via text message.

        for (int i = 0; i < 3; i++)
        {
            Console.WriteLine("Text mesasge sent to customer {0} payment was charged successfully", i);
        }
    }
}

public class BatchFileUploadJob : IJob
{
    public void Execute(IJobExecutionContext context)
    {
        // Idea here would be to hit query your table, get all payments that were successfull charged and generate a report
        Console.WriteLine("Report of payments processed successfully generated");
    }
}

public class ChargePaymentsJob : IJob
{
    public void Execute(IJobExecutionContext context)
    {
        // Query your table and get all payments that are due at this time and loop through those customers, get the required data and charge those payments.

        for (int i = 0; i < 5; i++)
        {
            Console.WriteLine("Charging Customer {0}s payment", i);
        }
    }
}

JobListeners

var jobname = context.JobDetail.Key.Name;

if (jobException != null)
{
    // Do something here to respond to any issues that arise from your jobs that executed.
    Console.WriteLine("Job {0} exploded, unable to complete the tasks it required", jobname);
    return;
}

if (jobname == "BatchFileUploadJob")
{
   // Code to come below
}

var nonDependentJobsFromDataStore = new Dictionary<string, string>();

nonDependentJobsFromDataStore.Add("RemindersJob", "QuartzDependentJobScheduling.TextMessageJob");
nonDependentJobsFromDataStore.Add("EmailJob", "QuartzDependentJobScheduling.EmailJob");

var jobs = new List<Tuple<JobKey, string>>();

foreach (var ndjob in nonDependentJobsFromDataStore)
{
    jobs.Add(Tuple.Create(new JobKey(ndjob.Key, "NonDependentJob"), ndjob.Value));
}

var jobDetails = new List<IJobDetail>();

foreach (var item in jobs)
{
    IJobDetail job = JobBuilder.Create(Type.GetType(item.Item2))
    .WithIdentity(item.Item1)
    .Build();

    jobDetails.Add(job);
}

foreach (var detail in jobDetails)
{
    ITrigger trigger = TriggerBuilder.Create()
            .StartNow()
            .Build();

    context.Scheduler.ScheduleJob(detail, trigger);
}

Full Job Listener classes

public class DependentJobListener : IJobListener
{
    public void JobToBeExecuted(IJobExecutionContext context)
    {

    }

    public void JobExecutionVetoed(IJobExecutionContext context)
    {

    }

    public void JobWasExecuted(IJobExecutionContext context, JobExecutionException jobException)
    {
        var jobname = context.JobDetail.Key.Name;

        if (jobException != null)
        {
            // Do something here to respond to any issues that arise from your jobs that executed.
            Console.WriteLine("Job {0} exploded, unable to complete the tasks it required", jobname);
            return;
        }

        if (jobname == "BatchFileUploadJob")
        {
            var nonDependentJobsFromDataStore = new Dictionary<string, string>();

            nonDependentJobsFromDataStore.Add("RemindersJob", "QuartzDependentJobScheduling.TextMessageJob");
            nonDependentJobsFromDataStore.Add("EmailJob", "QuartzDependentJobScheduling.EmailJob");

            var jobs = new List<Tuple<JobKey, string>>();
            
            foreach (var ndjob in nonDependentJobsFromDataStore)
            {
                jobs.Add(Tuple.Create(new JobKey(ndjob.Key, "NonDependentJob"), ndjob.Value));
            }

            
            var jobDetails = new List<IJobDetail>();

            foreach (var item in jobs)
            {
                IJobDetail job = JobBuilder.Create(Type.GetType(item.Item2))
                .WithIdentity(item.Item1)
                .Build();

                jobDetails.Add(job);
            }

            foreach (var detail in jobDetails)
            {
                ITrigger trigger = TriggerBuilder.Create()
                        .StartNow()
                        .Build();

                context.Scheduler.ScheduleJob(detail, trigger);
            }
        }
    }

    public string Name
    {
        get { return "DependentJobListener"; }
    }
}

public class NonDependentJobListener : IJobListener
{
    public void JobToBeExecuted(IJobExecutionContext context)
    {

    }

    public void JobExecutionVetoed(IJobExecutionContext context)
    {

    }

    public void JobWasExecuted(IJobExecutionContext context, JobExecutionException jobException)
    {
        var jobname = context.JobDetail.Key.Name;

        if (jobException != null)
        {
            // Do something here to respond to any issues that arise from your jobs that executed.
            Console.WriteLine("NonDependent Job {0} exploded, unable to complete the tasks it required", jobname);
            return;
        }

        Console.WriteLine("NonDependent Job - {0} executed successfully", jobname);
    }

    public string Name
    {
        get { return "NonDependentJobListener"; }
    }
}

Scheduler

ISchedulerFactory schedFactory = new StdSchedulerFactory();
IScheduler scheduler = schedFactory.GetScheduler();
scheduler.Start();

var chargePaymentsJobKey = JobKey.Create("ChargePaymentsJob", "DependentJob");
var batchFileUploadJobKey = JobKey.Create("BatchFileUploadJob", "DependentJob");

IJobDetail chargePaymentsJob = JobBuilder.Create<ChargePaymentsJob>()
    .WithIdentity(chargePaymentsJobKey)
    .Build();

IJobDetail batchFileUploadJob = JobBuilder.Create<BatchFileUploadJob>()
    .WithIdentity(batchFileUploadJobKey)
    .StoreDurably(true)
    .Build();

ITrigger dependentJobTrigger = TriggerBuilder.Create()
                .WithCronSchedule("0 0/1 * * * ?")
                .Build();

scheduler.ListenerManager.AddJobListener(new DependentJobListener(), GroupMatcher<JobKey>.GroupEquals("DependentJob"));
scheduler.ListenerManager.AddJobListener(new NonDependentJobListener(), GroupMatcher<JobKey>.GroupEquals("NonDependentJob"));

JobChainingJobListener listener = new JobChainingJobListener("DependentJobChain");
listener.AddJobChainLink(chargePaymentsJobKey, batchFileUploadJobKey);
//listener.AddJobChainLink(batchFileUploadJobKey, nextDependentJobKey);

scheduler.ListenerManager.AddJobListener(listener, GroupMatcher<JobKey>.GroupEquals("DependentJob"));

listener.AddJobChainLink(firstJobKey, secondJobKey);
listener.AddJobChainLink(secondJobKey, thirdJobKey);
listener.AddJobChainLink(thirdJobKey, fourthJobKey);

scheduler.ListenerManager.AddJobListener(listener, GroupMatcher<JobKey>.GroupEquals("DependentJob"));

scheduler.ScheduleJob(chargePaymentsJob, dependentJobTrigger);
scheduler.AddJob(batchFileUploadJob, false, false);

History

8/18/2017 - Initial Article written