Mobile Agents - Software on the move

Introduction

What if objects could move from machine to machine? Not just automated downloads, or remotely controlling objects as in remoting, but a system in which the object actually moves about. Mobile software agents can do exactly that. In this article I provide an introduction to this fascinating concept.

Forward

Mobile Agents have been a personal fascination of mine for nearly 20 years now. If I had the resources I would dedicate nearly all my time to them. However life is life and like most I am bound by other responsibilities. Despite the first version of this article having been written in 2005, I still think about mobile agents frequently and wait for the day that I can focus on them. Currently much of my free programming time is being dedicated to C# Open Source Managed Operating System (Cosmos). Cosmos is still in its early stages, but it is planned to make mobile agents a core part of Cosmos. Agent libraries will also exist on Windows and other platforms to allow the agents to move between different operating systems.

Origins

A software agent according to Wikipedia is "a piece of autonomous, or semi-autonomous pro active and reactive, computer software. Many individual communicative software agents may form a multi-agent system."

The idea of software agents has been around for many years - I do not propose that they are my original idea. Software agents originally were discussed in the 70's, and in the mid 90's briefly gained some momentum but then stalled. Proponents at the time claimed they would revolutionize the software industry. For various reasons beyond the scope of this article they have not.

When the idea came about to make them mobile I have not been able to determine. I first came upon the idea myself in 1997, but I suspect others had already considered the idea prior to this. Since then I have been preoccupied with pesky things like employment, family, and the usual whims of life. Recently I have had time to work on some academics, and mobile agents are something I have always wanted to revisit.

Since 1997 a few papers have been written, and even a commercial framework or two have emerged. However mobile agents remain largely unknown. Even if you cannot find use for mobile agents in your systems, I think you will find them fascinating.

From 2005 on, the mobile agent front in all quarters has been so quiet as to make one think they are a failed technology. This however is a false assumption and mobile agents I am convinced some day will become a staple of programming.

Focus on Mobility

Most discussions on agents focus on their autonomy, intelligence, and interaction. These are all worthy topics, however I will focus on mobility and how this feature of mobile agents can be used to build systems. Note that most agents are not mobile, and mobility in agents is a feature of its own.

Remoting versus Mobile Agents

Remoting includes network based RPCs (Remote Procedure Calls), DCOM, .NET remoting, TCP servers, web services, and most service oriented architectures. In remoting an already existing object is controlled remotely. The object only exists on the server but is controlled by a proxy class on the client that exposes an interface. Data is sent to the server, and results are returned.

With mobile agents the actual implementation moves. This makes it easier to deploy newer objects as they do not have to be pre deployed and installed before their use. Mobile agents are simply created, and are self deploying.

To match the remoting scenario, the object would return a value, or return itself. However mobile agents are much more flexible and have a variety of options.

The scenario I presented is a typical client server system. I would not recommend deploying remoting agents from the client to the server in enterprise applications as it creates deployment issues which for years we have strived to eliminate. Do not assume that this is a common scenario with mobile agents as it would be a shame to move backwards in time regarding deployment. While I have presented this scenario purely to demonstrate how it differs from remoting, there are scenarios in which this pattern is useful which I will cover in the future.

A more common scenario in client server or even n-tier systems would be as shown next:

This pattern has some very unique capabilities that may not be immediately obvious. I will cover this in more detail in the future, but as an exercise you may wish to consider the implications of this pattern.

No Single Solution

A unique feature of mobile agents is there is no one "configuration", but there are dozens. In the future I will spend significant time on covering various scenarios. For now, I want to stress that the patterns and examples I will demonstrate here are not what mobile agents are limited to. Some patterns have disadvantages and to think that all patterns of mobile agents have these problems will quickly convince you that mobile agents may be interesting, but are not useful.

No Client, No Server

In traditional designs there are designated clients, and designated servers. Tasks are always initiated by a client, performed by a server, and data returned to the client.

When there are only two tiers, it's very obvious which computer plays which role.

Even in n-tier designs, each relationship defines a role. In an example three tier design, the middle tier is both a client and a server. Whether it is a client or a server, depends on which relationship is examined.

For more information on n-tier systems please see my earlier articles:

• Dude, Where's my business logic?
• Tier Pressure and Isolationism.

With mobile agents, instead of splitting code into two pieces, all code exists in one object and the object moves about as needed. Because the objects themselves move, there is no client server relationship. Each computer is merely another peer. The mobile agent may have a home, or return data to its caller, but this relationship is completely dynamic and applies only to a particular choice of the agent, not the overall design.

As network administrators or developers, we still think of certain computers as servers. Why do we think of them as servers? Because they are in the server room and not on someone's desk. But to the mobile agents, they are just another host.

Mobile agents can certainly function in such an environment, and such "servers" are often used to distribute mobile agents, but in the software sense of the word, they are no longer "servers", but more akin to "birth places", "breeding grounds", "nests", or "hives". Mobile agents do not require servers however to operate, and can use any computer as a base.

Items not Covered

There are many topics related to mobile agents that will not be covered in this article. This article is already rather long, and covering all of them could easily be the topic of a book. Such items are authentication, advanced routing, and other such topics. This article is an overview and introduction, although a basic introduction to security using gateways is included.

Advantages

Single Code Base

In a client server example such as a web service, two pieces of code are necessary. The web service server, and the client proxy. The client proxy usually is auto generated but it must be regenerated and redeployed to match changes or to take advantage of new functionality. In addition, all data must be marshaled across method calls and reprocessed on the other side.

With mobile agents, the creation, modification and deployment is greatly simplified because there is only one set of source code. The complete object moves, including data, instead of only data.

No Interprocess Communication

Because mobile agents exist in one object, not split between a client and a server, no interprocess communication is required. In client server designs, significant effort is put forth to marshal data between the two parts, and to manage state. With mobile agents, none of this is required and thus significantly decreases development time.

No Deployment

Because objects are autonomous and able to move around, there is no need to deploy the objects. Simply tell the mobile agents where to go.

Easily Modified

Mobile agents are easily modified as all source code and thus logic is contained in one place.

Disadvantages

No technology or framework is without disadvantages and mobile agents are no exception. In this section I will discuss the disadvantages.

Bandwidth

Because mobile agents send not only data, but code, more network bandwidth is required. This is often offset or even reduced to lower level than without mobile agents by the fact that data can be processed at various locations in the network. The code portion of mobile agents can also be cached, reducing the bandwidth to just the data portion.

Most mobile agents however do require more bandwidth than sending raw unprocessed data. Compression can help as can efficient routing.

A basic .NET assembly is 16 kilobytes. If mobile agents are each put into their own assembly then code generally is not that large and the impact to the network is negligible, if even measurable.

Concepts and Terms

Tarakan

In my implementation, I have called these mobile agents "tarakans", which is Russian for cockroach. The proper plural in Russian is actually tarakani, but the English pluralization of tarakans will suffice for discussion.

I will not cover their implementation in this article. Instead this article will focus on concepts and discussion. Another article will present the tarakan framework with examples and usage. The tarakan framework will be released as open source and can be used for academic or commercial use.

Within this text, the terms tarakan and mobile agents are often used interchangeably.

Host

Any computer that supports the execution of tarakans.

Spawn

The process of moving a copy of a tarakan to another host. Spawning makes a copy and the copy is moved to another host, and the current tarakan continues to exist. To perform a move, the current tarakan is simply terminated after it is spawned.

Tarakans are not automatically terminated after spawning because often the tarakan continues with its task locally, or is spawned to multiple locations.

Home

Some patterns of tarakans have a home host. The home host is the originating host that tarakans return results to, or physically return to as tarakans. Sometimes the home host is a server class computer that spawns many tarakans, and some or all of the tarakans return to later. Other times the home is a desktop class computer that spawns a single tarakan.

Gateway

With mobile agents, it's already been discussed that there are no permanent clients or servers and that such roles are dynamic. Tarakans however support the concept of a gateway host. A gateway host is used for easy distribution and to implement security in a tarakan network.

Gateways can also provide directory services for tarakans guiding them where to go to perform their tasks.

Security using Gateways

There are many ways to secure mobile agents, this article will not cover them in detail. This section however will provide a basic overview of a core security concept, security using gateways.

If all hosts accept any tarakan without access control, havoc can certainly take hold quickly by abuse from desktop users, hackers, or even viruses. One way to control this is by using gateway hosts. Each host is configured to only accept tarakans from gateway hosts, trusted hosts, or tarakans initiated by a gateway.

Not Permitted

This migration is not permitted because it comes from an untrusted host.

Permitted

This migration is permitted because the two hosts have a trust relationship.

Permitted

This migration is permitted because although the second hop came from a host, the tarakan was originated from the gateway.

Permitted

This migration is permitted because the gateway has allowed it to be relayed and it is now considered a gateway initiated tarakan. Gateways can enforce policies regarding relaying.

In more advanced configurations, hosts may provide two classes of service, one for tarakans originated from gateways and other trusted hosts, and a more restrictive execution host for tarakans from other hosts.

Examples

In this section I will discuss many possible patterns and example solutions for using mobile agents. As I stated before, there is no single solution, and there are certainly more patterns than I will present here. Do not let your designs be constrained by these patterns.

In some examples, I will present a more traditional approach first for comparison. In such cases, it is provided merely for comparison and very likely it is not the only solution available. There are usually many ways to solve a problem.

Some examples have corresponding code examples that I will post later. In the future, all examples will have corresponding code to match the example.

Data Collection

Scenario

A corporate environment exists but the exact specifications of each machine are not known. To facilitate better deployment, data needs to be gathered about each machine regarding the operating system, the latest service pack installed, installed memory, and other similar statistics.

This is not a one time inventory and should be kept up to date. In addition over time, new information will need to be obtained such as checking for specific software, software versions, etc. It is not known what items might need to be checked for in the future.

Traditional

In addition, since the requirements state that the requirements will change and will require new deployment, the MSI must install a process or redeploy on a regular basis to allow updates to be deployed to the clients.

Implementation

The mobile agent example performs the same tasks, yet is easier to create, debug and update.

Data Collection - Larger Data

A mixed environment of laptops and desktops exists. The laptops are mobile and not always connected to the corporate network. Users enter their time worked with project codes, dates, and other information into Excel spreadsheets.

Close observers will notice that this is very similar to the previous example. The key difference is that instead of returning a result, the object is moved back to home. The reason for this is that the data to be returned is more complex, and also because the data will be acted upon by the agent after return. By keeping the code that is executed on both computers in one object, the design is greatly simplified.

Network Mapping

A network exists of multiple segments with many firewalls, routers and proxies. A quick and live manner of mapping the network is required.

Tarakans easily allow mapping across firewalls and proxies. Some hosts are only reachable by other hosts and the tarakans can detect this.

The tarakans are given a maximum hop value so that they do not continue on forever. This maximum hop value determines the maximum number of hops between hosts that they will make before self termination. Tarakans are allowed to visit a machine multiple times as they may reach the machine by multiple routes, and this information is recorded and analyzed.

Searching

The network needs to be searched for a specific criteria. However this criteria often changes and cannot be encapsulated into a predefined set of parameters. One time a computer with Russian language support may be needed, in another a computer with an old copy of Microsoft Office may be needed, yet in another a computer with a 486 processor may be needed. As you can see, the possibilities are endless and cannot be predetermined.

For searching, the Tarakans start out on a host computer. From that host computer, they are spawned to a list of known computers to that host. This list of known computers is carried with each Tarakan. At each computer they reach, they perform the following steps:

In this example it is possible and even likely that multiple tarakans will visit the same machine. This would cause not only duplicate results to be returned to the home, but also unnecessary network bandwidth and processors to be used. This is addressed using local storage. Each time a tarakan visits a host, it leaves an entry in the local storage using its initial instance identifier which was created on the first host. Each tarakan then checks the local storage to see if this host has already been visited by this type of tarakan. This entry is made with an expiration mark so that the local storage does not become cluttered.

Finding Resources to Complete a Task

A computer has a task to accomplish, but it does not have the resources to accomplish it. There are many scenarios that fit this pattern so I shall present a common one here as an example. A user wants to print a document. Normally the user manually selects a printer.

In corporate networks, browsing for a printer often brings up dozens of not hundreds of printers, and choosing one can be quite a daunting task. For laptop users who plug into various networks, manual selection is a must and each network may have numerous printers.

This implementation is a little more complex than the previous ones. Because of this I have not been able to fit all the steps in the diagram and instead will discuss them.

First the client requests a printer tarakan from the gateway. It does this instead of storing the printer object locally because it allows the administrator to update the tarakan. Some preferences could be applied after the tarakan returns, however pulling it first allows client side functionality to be downloaded as well. For example, if the administrator wants to enforce a restriction of a 100 page limit this can be communicated to the user immediately on the client before the tarakan leaves.

Once the tarakan is received, the attributes of the print job are set. In this example, the document type is grey scale, there are 150 pages, and the desired quality is medium. Other attributes such as paper size, priority and proximity could be used.

The tarakan is then pushed back to the gateway. At the gateway the tarakan obtains a list of printers available. Only static properties such as capability (color, black and white, etc.) and location are stored in the directory. Dynamic properties such as current load, ink levels, and paper status are not available in the directory. Using the directory, the tarakan is able to eliminate the plotter based on its static properties. Once it has this list, it can begin to spawn to each printer one at a time to interrogate it and determine its usefulness and eliminate printers in other buildings or on other floors.

In this example, the sampling might go like this:

• Ink jet - Can print grey scale, has ink, but it's currently out of paper.
• Laser printer - Meets our requirements, has a low cost per page.
• Impact Dot Matrix - Used for form and carbon copy prints. Can print but will not meet desired quality and is loaded with form feed carbon copy paper.
• Plotter - Plotter is skipped based on information from directory.
• Color Laser printer - Meets our requirements, but cost of printing is higher.

The tarakan then returns to home. It may select a printer automatically, or present the user with the most suitable ones and let the user make a final decision. It may also report that no printers are available and the user may re-dispatch the tarakan to find printers on an adjacent floor. The tarakan might also take into account the current queue of the printer. If a printer has 500 pages queued, it may decide to exclude it. This tarakan was not the real printer tarakan - it was merely a resource finding tarakan.

Now that a printer has been selected, the print job tarakan is spawned. The print job tarakan is spawned directly to the printer but it does not render locally. It saves network bandwidth by rendering directly on the printer and carrying the source with it. When it has successfully printed, it sends back a notification tarakan to home.

Distributed Processing

A network exists with anywhere from a few desktop computers to thousands. The desktop computers are utilized during the work day, but are idle at lunch and outside of working hours. This provides 16 hours of idle time per work day, and 48 hours of idle time on weekends. Each desktop also has periods of inactivity during the work day, and most desktops utilize on average <5% of the CPU with CPU usage only being used in spurts. Developer machines are more heavily utilized but still only consume CPU usage in spurts.

Using agents, tasks requiring a lot of processing must be custom built to distribute the load between computers. Such schemes exist such as SETI, and others, but each requires a central server, remoting, and a custom installation. With agents, any number of needs can take advantage of distributed processing in idle times by simply creating one agent and giving it a piece of data to work on.

Most applications that can take advantage of distributed processing are video and audio encoding, large numerical calculations, and the like. Very few businesses have such needs, but there are business cases for distributed processing. Data analysis and data mining are common in enterprises and are ideal to be run during off peak hours. Advanced data analysis often is queued to run at off peak hours anyways to avoid excessive load on the database during working hours. Because a single machine often cannot analyze all the data in one night or even a weekend, it is often broken into several nights. By using mobile agents, the load can easily be distributed among many machines and what otherwise would require many days or even weeks to accomplish can be done faster.

Disconnected Processing

Laptops are connected in the morning each day and then taken into the field. Sometimes they are reconnected to the corporate network in the evening, but often not until the next morning. Laptops bring back data from the field and processing of the data requires a significant amount of time. Requests can also be made that cannot be serviced immediately.

A sales force might collect data throughout the day. When they return they want to connect just long enough to offload the data, but not wait for the data to be processed. Requests for advanced research on customers can also be dispatched. When they reconnect the next morning, any errors are reported and the results of any requests returned.

Traditional

Traditional designs of this scenario use queuing, often MSMQ. In fact mobile agents in this scenario could utilize a queuing system for their transport.

Pushing

Processing servers are set up to handle processing for clients. The advantages are that of a typical n-tier design. The servers are closer to the data, have more power, and more memory. The desktops therefore can be on a slower connection, and have lesser processors and lesser memory.

For operations, the desktop computers push mobile agents to processing servers. The mobile agents are then load balanced across the processing servers.

The advantage with this scenario is extremely simplified deployment and easier versioning. While each desktop must be deployed to, only one tier must be deployed and there is never a version mismatch between tiers, because each desktop completely contains the application. Yet logically, it is still an n-tier design.

Servers can also handle a variety of tasks, not simply fixed designations. Users can create their own uses for servers without the need to install any software on the server. This is a very useful way for a set of servers to support a variety of applications.

Because enterprises focus on few versions, and centralized control, this pattern is not common in the enterprise. However outside the enterprise it is very common.

Spam is becoming ever pervasive, as are viruses. Spam and virus scanning is often performed both on the mail server and on the mail client.

Performing scanning on the server requires the owner of the server to maintain the definitions. While the server owners are usually diligent in keeping definitions up to date, it leaves the user with no choice of software. Many ISPs implement spam solutions that interfere with users wishes and the users have little or no configuration options other than to turn them off.

Performing scanning on the client allows the user to choose software that suits their needs precisely and to configure it as needed. However it requires that all mail is downloaded first. With spam becoming so prevalent, it is not unusual for spam to consume more than 25% of a mailbox's messages.

By using mobile agents, each user can choose their own spam control and virus protection. The mobile agents have full access to the user's mail box and can control and filter messages on the server saving the user bandwidth because only the unfiltered messages are downloaded.

Users could also customize the mobile agents to prioritize, reroute, or to perform any task desired.

Hold and Store

Hold and store can be used to run mobile agents on a regular basis. Since mobile agents can be moved across a network, they can also be hibernated to disk. An agent therefore can be moved to a computer and hibernated, but awoken once per day, or at some other interval.

Scenario 1

In a corporate environment, users often install prohibited software. The urge to lock down users' machines is the natural one, however this often restricts the users too prohibitively. In fact I have seen a few corporations lock down developers' machines and remove their local administrator rights. This of course makes development very difficult, if not impossible for most developers' needs.

A mobile agent can be dispatched to watch for prohibited software without disabling the machine to a point that unusable for the person's needs. It might be determined that ICQ is not permitted. But then later the company decides to also prohibit Skype and other new programs that emerge. Since there is no easy way to detect all programs by a set of simple criteria, a mobile agent allows the criteria to be easily expanded and programs to be detected in whatever manner is required.

Scenario 2

A virus scanner is also an ideal candidate for a hold and store mobile agent. Virus scanners are regularly updated. Often the update includes only a definition update, but since viruses are constantly evolving, very frequently the actual implementation of the virus scanner needs to be updated as well. This makes mobile agents an ideal solution.

In an enterprise, the mobile agents can be pushed each time an update is required. Outside the enterprise, existing mobile agents can check for newer versions and pull newer versions if they exist.

Move from User to User

Mobile agents are also extremely useful in tasks that require input from many sources.

Scenario

This scenario uses user input, however other scenarios may rely on non user input based resources. In most enterprises, to purchase items, people must be involved. First the user must create a request with information about the vendor and product desired. From there, it must to go to their manager for approval. Next it must go to a purchasing agent. After the purchasing agent creates the purchase order, it is sent back to the user for confirmation. If the user approves it, it will then go to the mail room, else it will go back to the purchasing agent.

Because so many points are involved, most enterprises today have not automated such processes. Instead, they rely on Word, Excel and PDF documents relayed by e-mail. This process however is often very inconvenient for users, and because many processes are manual, steps in the process often break down.

Implementation

With mobile agents, the mobile agent can move from location to location, either by its normal transport or by email. When it reaches each machine, it notifies the user of the action required and prompts the user for input. When the input is satisfactory, it packs up and moves to the next user.

Business Logic

Mobile agents as I have demonstrated so far have many uses. However there is one that I believed has been overlooked and is of great use to enterprises, and that is in the client side validation of business logic.

In earlier articles, I discussed business logic and n-tier design. If you did not catch the hint the first time I mentioned them here, they are again:

• Dude, Where's my business logic?
• Tier Pressure and Isolationism.

As discussed in my earlier articles, an ultimate 3-tier application would distribute the business logic as follows:

In practice the above scenario is rare because it would require too many validations to be sent to the business layer or provide for a user interface that does not properly inform the user of when certain data fields are valid or not. Because of this a fair portion of the business logic which is related to validation, enabling and disabling of fields, etc., is copied to the client as shown next:

This is of course a practical decision and a known trade off. With mobile agents this could be addressed by sending down a mobile agent. The mobile agent not only brings the data, but advanced support for client side validation and input logic.

Let's consider an example. An input screen exists to input an address. In the address, there are fields for state, post code, and country. The state field however is required only if the country is USA or Canada. For other countries the state is optional.

If all logic is kept in the business layer, this requires a call to the server with an attempted post to insert the record. Only then can it return and tell the user that a field is missing. This is not only slower, but is not as clear for the user. The client application itself does not know that the state field can be required.

If this information is kept in the client application, when the requirements change, the client application must be redeployed.

Some frameworks are even rule based and send down with data certain rules about the data. However even simplistic and common rules such as this cannot be conveyed in their limited meta data.

With mobile agents, this validation logic can easily be built in to the data as validation code. Although it executes on the client, since it is stored on the business layer, it is still considered part of the business layer and the 0-100-0 ratio can easily be kept.

Let's consider a more complex yet common need, validation of post codes. Each country of course has its own format for post codes, and occasionally the rules change. With the validation of phone numbers, the rules very often change. Typically post codes are validated on the server against data tables. Certainly I am not proposing that these data tables be downloaded in each mobile agent, this would be very inefficient. However, with mobile agents, the format such as length and characters used in the post code can be easily validated in the client application. If the rules change for a given country, it's simply a matter of updating the mobile agent in the business layer.

Some frameworks attempt to download complete clients when changes occur. This however creates bandwidth as well as installation issues. Since mobile agents are small and internal, they can easily be changed and even cached, without creating large bandwidth requirements or installation conflicts.

In the future, I plan to write an article dedicated to this concept.