Running jobs sequentially using Quartz.Net with the help of Job Listeners & Scheduler Plugins

• Download SequentialQuartz_POC-noexe.zip - 8.8 MB
• Download quartz_settings.zip - 2.2 KB

Introduction

Quartz.Net is a port to .Net of a very famous job scheduling open source library known as Quartz (which can be integrated into a wide veriety of Java applications) which developers have been using for many years. Quartz allows users to schedule multiple jobs by simply adding/updating information in few setting files like quartz_jobs.xml, Quartz.Server.exe.xml & quartz.xml. Quartz uses the fundamentals of multi-threading to execute jobs and hence it runs jobs parallely. We can tell Quartz to prevent concurrent execution of jobs by changing it's setting but to tell it to execute job two after job one & job three after job two is not as straight forward as changing the settings / configurations. Let's understand this better by looking into a problem statement and finding a proper solution to it.

Note:- It is very important to have basic understanding of Quartz to understand this article properly. Please go through Quartz.Net documentation/samples from here.

The Problem

Let's say we have two jobs - Job One & Job Two. Job One collects information and inserts it into a database table after modifying and filtering it based on certain logic and Job Two creates reports and sends them to users using the data inserted by Job One. So, it does not make sense to run job Two with out running job One first because otherwise our users will get reports which might be empty or not latest(old data). So, it is necessary to run job Two only after job One completes.

Arriving to a Solution

To achieve this some people will suggest that we should schedule job Two to run 15 or 20 minutes after running job One based on common sense and hopping that job One has completed by then. First of all, we can not guarantee that job One will complete in 15-20 minutes. There can be multiple reasons for this, like, too much data to process in database or database server is over burdened and responding very slowly etc. Second of all, how can we be sure that the job One executed successfully and there were no exceptions encountered during the execution.

To overcome the above challenges we will use Quartz.Net's ISchedulerPlugin & IJobListner interfaces to write our own plugin and job listener. We will learn more about these in our code below.

Prerequisites :-

Before moving to the code, let"s make sure that we already have or are aware of the following:

• Install Quartz.Net as a windows service and test it. I am using the latest version of Quartz.Net which is 2.3.3. This can be easily done by following the instructions mentioned here.
• Configure logging for Quartz.Net windows service. Follow instructions mentioned here. This will help us track what our jobs are doing by looking at the log files.

Using the code

Now that we have installed Quartz.Net as a windows service and configured logging, let's start writing the code.

I am using Visual Studio to create a new project (class library).  Add Quartz.dll in references using Nuget or use the same Quartz.dll found inside the Quartz install folder (e.g, D:\Program Files\Quartz.net, this is where I installed Quartz. If you remember, we installed Quartz as a windows service during one of the prerequisites steps). Make sure that the version of Quartz.dll is the same inside the install folder and one referenced by the class library project.

Now add a class for JobOne and add the following code.

using Quartz;
using System;

namespace SequentialQuartz_POC
{
    public class JobOne : IJob
    {
        /// <summary> 
        /// empty constructor for job initialization
        /// </summary>
        public JobOne()
        {
            // quartz requires a public empty constructor so that the
            // scheduler can instantiate the class whenever it needs.
        }
        public void Execute(IJobExecutionContext context)
        {
            Console.WriteLine("Job one started");
            // we can basically write code here for the stuff 
            // which we want our JobOne to do like inserting data into DB, etc.
            System.Threading.Thread.Sleep(10000);  
            Console.WriteLine("Job one finished");
        }
    }
}

Similarly add a class for JobTwo and the following code.

namespace SequentialQuartz_POC
{
    public class JobTwo: IJob
    {
        /// <summary> 
        /// empty constructor for job initialization
        /// </summary>
        public JobTwo()
        {
            // quartz requires a public empty constructor so that the
            // scheduler can instantiate the class whenever it needs.
        }

        public void Execute(IJobExecutionContext context)
        {
            Console.WriteLine("Job two started");
            // we can basically write code here for the stuff 
            // which we want our JobOne to do like sending emails to users with reports as attachments, etc.
            System.Threading.Thread.Sleep(10000);
            Console.WriteLine("Job two finished");
        }
    }
}

Notice that in both the classes we are implementing interface IJob. This lets the scheduler identify that these are the jobs which need to be scheduled and executed. Also, in both the classes we have an empty constructor which is required by quartz scheduler to instantiate class whenever required.

After creating jobs, we will create a Job Listener class. But before that....

What are Quartz.Net listeners?

Quartz.Net listeners are objects that can be notified when certain events happen inside a Quartz.Net scheduler. There are 3 types of listeners

• Scheduler listeners
• Job listeners
• Trigger listeners

In our case, we need to add a job listener to the scheduler. Job listeners are used whenever we want to be notified of job level events. To create it, we need to implement IJobListener interface. This interface has 3 methods (JobExecutionVetoed, JobToBeExecuted & JobWasExecuted) and a property (Name). Below is the code for creating our job listener.

using System;
using Quartz;

namespace SequentialQuartz_POC
{
    /// <summary>
    /// this listener class has methods which run before and after job execution
    /// </summary>
    public class JobListenerExample : IJobListener
    {
        /// <summary>
        /// to dismiss/ban/veto a job, we should return true from this method
        /// </summary>
        /// <param name="context"></param>
        public void JobExecutionVetoed(IJobExecutionContext context)
        {
            // this gets called before a job gets executed
            // by returning true from here we can basically prevent a job or all jobs from execution
            // Do nothing
        }

        /// <summary>
        /// this gets called before a job is executed
        /// </summary>
        /// <param name="context"></param>
        public void JobToBeExecuted(IJobExecutionContext context)
        {
            Console.WriteLine("Job {0} in group {1} is about to be executed", context.JobDetail.Key.Name, context.JobDetail.Key.Group);
        }

        /// <summary>
        /// this gets called after a job is executed
        /// </summary>
        /// <param name="context"></param>
        /// <param name="jobException"></param>
        public void JobWasExecuted(IJobExecutionContext context, JobExecutionException jobException)
        {
            Console.WriteLine("Job {0} in group {1} was executed", context.JobDetail.Key.Name, context.JobDetail.Key.Group);

            // only run second job if first job was executed successfully
            if (jobException == null)
            {
                // fetching name of the job to be executed sequentially
                string nextJobName = Convert.ToString(context.MergedJobDataMap.GetString("NextJobName"));

                if (!string.IsNullOrEmpty(nextJobName))
                {
                    Console.WriteLine("Next job to be executed :" + nextJobName);
                    IJobDetail job = null;

                    // define a job and tie it to our JobTwo class
                    if (nextJobName == "JobTwo") // similarly we can write/handle cases for other jobs as well
                    {
                        job = JobBuilder.Create<JobTwo>()
                                .WithIdentity("JobTwo", "JobTwoGroup")
                                .Build();
                    }

                    // create a trigger to run the job now 
                    ITrigger trigger = TriggerBuilder.Create()
                        .WithIdentity("SimpleTrigger", "SimpleTriggerGroup")
                        .StartNow()
                        .Build();

                    // finally, schedule the job
                    if (job != null)
                        context.Scheduler.ScheduleJob(job, trigger);
                }
                else
                {
                    Console.WriteLine("No job to be executed sequentially");
                }
            }
            else
            {
                Console.WriteLine("An exception occured while executing job: {0} in group {1} with following details : {2}",
                    context.JobDetail.Key.Name, context.JobDetail.Key.Group, jobException.Message);
            }
        }

        /// <summary>
        /// returns name of the listener
        /// </summary>
        public string Name
        {
            get { return "JobListenerExample"; }
        }
    }
}

As you can see in the above code, JobWasExecuted is the most important method. This method get called after scheduler executes any job. Here, we are using this method to check for the completion of JobOne (also checking if there were no exceptions) and then scheduling JobTwo to execute after it. We have to add a key entry inside the quartz_jobs.xml file present in the Quartz installation folder which will keep information of the next job(JobTwo) to be executed after the current job(JobOne). So, as soon as JobOne finishes, JobTwo will start running.

Below is the key entry inside job-data-map tag of the job.

<schedule>

    <job>
        <name>JobOne</name>
        <group>JobOneGroup</group>
        <description>Sample job for Quartz Server</description>
        <job-type>SequentialQuartz_POC.JobOne, SequentialQuartz_POC</job-type>
        <durable>true</durable>
        <recover>false</recover>
    <job-data-map>
            <entry>
              <key>NextJobName</key>
              <value>JobTwo</value>
            </entry>
      </job-data-map>
    </job>

    <trigger>
      <simple>
        <name>sampleSimpleTrigger</name>
        <group>sampleSimpleGroup</group>
        <description>Simple trigger to simply fire sample job</description>
        <job-name>JobOne</job-name>
        <job-group>JobOneGroup</job-group>
        <misfire-instruction>SmartPolicy</misfire-instruction>
        <repeat-count>-1</repeat-count>
        <repeat-interval>100000</repeat-interval>
      </simple>
    </trigger>
  </schedule>

The question here is, how will the scheduler know about the job listener we have just created? E.g, scheduler comes to know about the Jobs we have created by reading quartz_jobs.xml file present in the Quartz installation folder. Unfortunately there is no way to let scheduler know of this job listener using and xml or config file. To resolve this problem, we will create a custom plugin and mention/register it in quartz.xml file present inside the quartz installation folder (D:\Program Files\Quartz.net in my case) and use this plugin to add job listener to the scheduler. Let us see how to do this.

Now, we will create a custom plugin. Plugins come handy whenever there is a need to do something before scheduler starts executing jobs or after it stops.<o:p>

Quartz provides an interface (ISchedulerPlugin) for plugging-in additional functionality.<o:p>

Plugins that ship with Quartz to provide various utility capabilities can be found documented in the Quartz.Plugins namespace. They provide functionality such as auto-scheduling of jobs upon scheduler startup, logging a history of job and trigger events, and ensuring that the scheduler shuts down cleanly when the virtual machine exits.<o:p>

In our case, we want to attach a job listener to our scheduler before the scheduler starts executing any of jobs.

<o:p>

Create a class for our plugin as below:<o:p>

using Quartz;
using Quartz.Spi;
using Quartz.Impl.Matchers;

namespace SequentialQuartz_POC
{
    /// <summary>
    /// this class is for our plugin. We will use it to add a job listener to our scheduler
    /// </summary>
    public class PluginExample : ISchedulerPlugin
    {
        public void Initialize(string pluginName, IScheduler scheduler)
        {
            scheduler.ListenerManager.AddJobListener(new JobListenerExample(), EverythingMatcher<JobKey>.AllJobs());
        }

        public void Shutdown()
        {
            //Do Nothing
        }

        public void Start()
        {
            //Do Nothing
        }
    }
}

In the above code, we have used the initialize method of the plugin to add a job listener to our scheduler. To let scheduler know of this newly added custom plugin, go to the Quartz installation folder, open quartz.xml file and add following code.

# job initialization plugin handles adding a listener for our jobs
quartz.plugin.PluginExample.type = SequentialQuartz_POC.PluginExample, SequentialQuartz_POC

Be careful while writing the above code. It should be structured like this:

quartz.plugin.{name}.type = {type name}, {assembly name}

The {name} part is the name that you want to give to your plug-in, and is the name that is passed in to the Initialize method above. The value of the property {type name}, {assembly name} tells the scheduler the Type of the plug-in, so that it can be loaded into memory. {type name} is the full name of your plugin, which in our case would be Examples.PluginExample. {assembly name} is the name of your assembly file, minus the .dll extension.

And this is it. Just build our visual studio solution in release mode, copy project's dll from bin folder and paste it inside the quartz installation folder. We can now start Quartz windows service to execute our jobs. Go to services.msc and look for Quartz service. Start it. After some time, go the quartz installation folder, you will find a trace folder there. Go inside it and open application.log.txt file. You will find the following lines in the log somewhere.

This means our scheduler was started successfully. We can also add custom logging in the job/plugin/listener code instead of/along with console.WriteLine(). With this, we can track all job info and exception details in a separate log file.

To see all the console.WriteLine messages that we added in our code, stop Quartz windows service, go to quartz installation folder, find Quartz.Server (application) and double click on it. It will open a console window and the following messages will be shown there.

Points of Interest

So, we just saw how to schedule a job after the successful execution of another job. We can extend this same logic to run multiple jobs one after the another. I am attaching the project I created for this and also the Quartz settings that I used. Kindly let me know if you have any queries or not able to download the code.

History

Keep a running update of any changes or improvements you've made here.