The Lounge - CodeProject

Walt Thiessen10-Apr-17 5:02

10-Apr-17 5:02

I've been giving some thought to the email spam/malware/adware delivery problem. Despite significant gains over the years toward combating email spam, it's still a big problem.

Gmail set the standard with their approach using aggregate community behavior data to decide what's ham and what's spam, but it only helps Google mail users. Bayesian and other spam-scoring schemes have produced various levels of success. Major anti-spam portals have also produced significant results.

Yet, email spam continues to be a growing problem, particularly now that it's a major delivery platform for malware and ransomware distributors.

Most solutions look at spam largely from the viewpoint of the content of emails. I've been thinking about it in terms of who the senders are.

Reputation is a big concept these days, yet for some reason it doesn't seem to be considered a big concept in combating email spam. Certainly, there are some systems that measure and use reputation to some degree, but it's nowhere near universal. I think perhaps that it should be.

I've been toying with the idea of starting an open source project from this perspective (unless someone can point me to one that already exists that does most of what I'm talking about here). My thinking is still in its early stages, but it runs along the following lines:

I. Provide plugins for Outlook, Apple Mail, Windows Mail, Thunderbird, and other email clients that do the following:

a. Provide users with a one-click way to whitelist favored correspondents. Build a button or link into each email thread in their client. If the user's correspondent is not whitelisted, the button or link offers "Whitelist" and "Blacklist" as clickable options. If the opposite is true, then it offers "Unwhitelist" and "Blacklist" as options. One click does the job.

b. Provide users with a simple, built-in way to set up email client encryption. Users find it daunting to set up local encryption, to say the least. It should be more widely and easily available.
1. Software negotiates most of the setup work with certificate providers like Comodo who offer individual certificates. User is only asked to provide personal data requested by the CA for issuing the certificate. Software does everything else.
2. Cloud tracks public keys.
3. Client optionally checks DMARC/DKIM/SPF for users whose email providers don't adequately check this stuff server-side.

c. Periodically (daily?) send data to a cloud that tracks simple ratings of user's interactions with their correspondents.
1. Whitelisted correspondents get a high score.
2. Correspondents who end up in the user's junk/spam folder get a low or negative score.
3. Others get a neutral score.
4. Data includes frequency of correspondence with each correspondent.

d. Three folder levels created and used: inbox, junk, and "new correspondents".
1. "New correspondents" would be senders that client hasn't seen before and that don't end up in the junk folder right off the bat because it already scored as likely spam.
2. Inbox would be for emails from whitelisted correspondents and possibly from correspondents with very high reputation scores.

e. Integrate local bayesian scoring already supplied by the client into this system as well.

II. Provide cloud or distributed network to track:

a. User scoring of correspondents

b. Public keys for client encryption

c. "Neighborhood" scoring ... who tends to correspond with clean correspondents, and who tends to correspond with shady correspondents?

d. All cloud accounts are scored on-going based upon their interactions with other correspondents. Scores are assigned based on their histories, with special weight toward their most recent histories.

e. Correspondent scores are available to clients for easy checking.

This isn't a complete outline, by any means, but it's a start.

Thoughts?

Welcome to the Lounge