An Introspection on Custom Rule Engine

Abstract

Defining rules in Rule engine is a type of declarative programming where the business logic is maintained outside of the main software component. There are different ways to build rule engine and different ways to put the use of it. As it may seem simple, it poses the serious challenge as the system grows its importance inside the enterprise. The wide varieties of issues including but not limited to the conception of the rule engine, rule semantics, authoring, testability and change management.

Introduction

An application logic can be built with the declarative approach, when there is need of logic that needs frequent attention and time to market is the priority. Building application this way requires the decision maker to aware of all benefits and consequences. This paper narrates the life cycle of custom rule engine from strategy to maintenance.

Cadence

The situation to build a rule engine arises when the requirements are straightforward and less dependent on one another. It is easy to express in a declarative way using condition and action. A typical rule:

If  A > B  then return X + 1

So writing a simple rule engine requires a system to pick the rules and evaluates its condition and perform the subsequent action defined in that rule.

Business domain with constant requirement change like insurance underwriting or tax calculation for products requires a system that is ready to adopt the change easily and quickly. Rule-based systems are best candidates to fulfill the need. One should not think of building the whole domain problem using rules which makes the system complex to understand and hard to maintain. A complete domain should not be built using rules rather it is better to see which parts of logic do not necessarily change and which part of system changes. Rules approach can be applied to frequent changing part of the system and thus making the rules footprint inside the system to minimal.

Another aspect that influences in building rule-based system is the myth that business users or users who live above the technical people in product paradigm can easily make a change to the system. However, in many cases, the fact remains otherwise where a specialized people require maintaining the rule system. Since the rules introduce new language into the enterprise and retaining the knowledge becomes an additional responsibility.

Rules reduce the semantic gap between the problem domain and product owners of the domain.

Build Complexity

Declarative programming can be done using scripting or rules-based. The scripting is powerful but hard to understand by the non-technical user. Building rules-based system requires two-part, rules and rule engine that executes the rules.

The rule in the context of the system is a single atomic operation that yields a result or not. Rules have the condition which acts like filters and decides whether the rule should participate in the request context or not. The action of the rule may be simple like returning a value or calling other software component or dependent on another rule.

Rule => If  A > B then X +1

Rule condition aka filters may depend on other rules which add complexity to the ordering of the rule execution inside the system. In the above expression, if A and B are direct values, then it becomes easy for the rule engine to evaluate. In case A or B depend on another rule, then system execution has to work in phased steps. First, the engine has to evaluate A and B then execute the rule.

Rule => If  A > B then X + 1

Rule action may depend on another rule. It is called Chaining. In the above expression, X is the other rule (referred to as child rule). There are two strategies to build.

Child Rule Is Totally IndependEnt of the Calling Rule

This is simple and the easiest way to conceptualize the rule. The rule engine needs only one time to execute the rule in this approach. In this case, X is always executed once and it can be referred without further evaluation in all the places it referred.

Child Rule Knows the Context of the Parent Rule

This approach makes the rules versatile and promotes for reuse, however it myriads the underlying programming procedure/function paradigm. The rule that needs to be executed many times in the context of the parent rule are different.

Rule engine which loads interprets and executes the rules needs to be designed for performance. Mostly, memory is traded for performance in rule engine implementation. The rule engine needs to pre-segment the rules into the group. For example, if there are 10,000 rules in a system, then it's unwise to evaluate and execute all rules for each request. The solution to the problem is to group the rules into a segment that closely matches the problem domain.

R =  U( r(1), r(2), r(3), … r(i),….r(n) )

More elaborative techniques can be used to evaluate the rules such as using Rete algorithm. Most of the commercial rule engines support rete algorithm implementation. Machine/Neural learning can be applied for segmentation process.

Authoring

Dealing with the problem of creation of rule semantics and engine is just one side of the coin. The other side is to provide a meaningful platform for the user to define the rule. The medium should supplement the user to understanding and help to define the rules with all its grammar. It should also express the relation of the rules with one another.

Adding rules to the system is easy, however it is better to set up guidelines that logically segment the rules with respect to the problem domain. This makes it easier to understand the rule and maintain them.

The rules can be authored by many people so common design guidelines and communication of the same is necessary. The recurring structure of the rules has to be documented and categorized as a pattern. It promotes consistency and is easy to communicate the complex rule relations.

The rule semantics should support the chronological requirement, e.g., if the tax rate changes, then the rule system may have to keep knowledge of this event to calculate taxes based on date.

Testability

The defined rules are tested against semantics but not logically hence it is always hard to predict the correctness. Even a single rule change can affect the behavior of all the dependent rules. So it is best to conceive the testability of the rule system and rule engine at an early phase of the build activity.

Since there is no compiler support and interpreter support of the rules. It is easy to end up issues like the circular reference. The system should be built capable of identifying such a scenario beforehand.

The engine needs to support instrumentation and tracing facility to monitor. There should be enough test scenarios to support all rules however it is easier said than done since the cyclometric complexity increases in a proportion to the depth of the inter-dependency of rule structure.

Change Management

Most often, the rule engine conceives on the requirement of quick changes to business rules. However, the changes to rules are not straightforward to deploy to the production system. The old rules need to be properly retired and archived. The new rule addition or modification should be propagated to the cache of the rule engine.

Designing a system that can incrementally or selectively update the rule engine is hard. Mostly, engines are cached and any change to underlying rules may require the engine to discard the cache and build again. The implementation of rete algorithm can address these issues of selectively upgrading the rules.

Conclusion

It is easy to put oneself to see a necessity to build rule engine to solve the problem at hand. However, the context has to be understood and post-implementation and maintenance have to be accounted and planned well ahead.

References

• Should I use a Rules Engine?
• Business Rules Framework Architecture
• Best Practices for Rule-Based Application Development
• Rete algorithm
• Building Custom Rule Engines