HKB Project

• Download demo project - 3.5 MB

So how many of you out there have done small web site projects, and have launched them or made them public, only to wait many months for the website to start producing results such as a phone call or contact us messages. What if you could launch a website, and have it fully crawled and indexed by almost all search engine vendors in 7 days, and fully tuned in less than 30 days.

Such a task is possible if you have setup your project correctly, and have written the needed tools to monitor the results on a daily basis, to make your corrections.

I wish I code just paste some code snippets that will allow you to do this, but this challenge has been a work in progress for me over the last 10 years, and I would like to share my knowledge and technique with you the Code Project Member. I've have written many tools to aid in this, and have combined them all together to create the HKB Project. 

My project will help you confirm that your website is getting crawled, and that every page has been indexed. Then it will monitor both organic and paid landings, You can monitor paid advertising landings, examine the keywords used, to create a positive or negative keyword list. Negative keywords helps to narrow down your target demographic, with laser like accuracy. 

If you are struggling with promoting your website project, then read this complete article to pickup some tips and tricks that will help you with this task.

Introducing the HKB Project    

So what is the HKB Project? Well HKB stands for Harmonic Keyword Balancing, and was a complex thought that occurred to me back in 2003 when I was studying website construction and how they interact with search engines.   

The HKB Project will assist you in getting your new website projects and marketing campaign content  fine tuned within the first couple of months of operation. Do not confuse the HKB Project with analytic software.  

This is a ASP.net 4.0 server control that you place on the master pages that you want to keep track of. It's packaged as a .DLL file, so you simply drop the file in the bin, create the control element, set your folder permissions and off you go.  

Ive decided to share this program with the Code Project Community to see if the project is worth working on, and to see if there is any value to it. Plus I'm not able to complete the program because I can't figure out how to write the spider I need to collect existing words, and how to display the words collected for comparison, so in other words, my coding skills are still not sufficient to complete the project.

If your a qualified programmer that wishes to work on the project, I will be glad to share the source code with you. but you must have a at least 10K Code Project points to qualify.

HKB Project Conception    

I'm sure your rolling your eyes right now, and think I'm crazy but let me take the time to explain. This is a fun project that I have been working on for over 10 years, but I was limited in time and programming experience to write the complex code needed for full operation, so it's still a work in progress. But now at this point, it's almost ready for commercial purposes, and just needs a built-in spider to self crawl the website, a dictionary of words, and word comparer to finish the project.

When I first started creating websites, I thought I was the master of words. So I created very well written marketing content, grammatically correct, that was word art basically. I was proud of the work and the content, but I was not able to generate organic page landings. As a couple of years went by, I kept asking myself why am I having trouble generating organic search results, what's going on here. Then it hit me like a bolt of lightning one night drinking with friends at a party. I was listening to conversation from my friends, and realized that everybody has their own word dictionary in their head, and each one of my friends used a different phrase to describe something. Different nomenclature was used to ask for things, specify an object, describe an event, and few were the same. That was the answer; I wasn't using the same words that people use to look for things. So how does a person find the right words? Well I guess I need to start collecting the words and store them for later use. At that point I thought that the true master of words are the search engine users.

So that's how it started, I wrote a small program to collect the words used in search queries, on a per page basis, and used the data to correct words on the page. Then I tried creating a broader set of words that cover a wider range of people, and finally tried to obtain some sort of harmonic balance in the word sets on a per page basis.

As a sanity test later the following year, I had customers pick out keywords to use for paid advertising campaigns. Actually at first, they had to think of keywords, until keyword list were created for them to choose from. The results were interesting, in the fact that people ranging from those with PHD's to the high school graduate choose different sets of words that were important to them. Once again that little word dictionary in everybody’s head was different. So I came to the conclusion that no matter what area of the country you grew up in, your level of education, IQ score, age, we are all unique and use different words to describe things.  

HKB Dashboard - Main Interface

The Project's main interface has a dashboard for quick viewing of statistics. From here, you can look up Organic Landings, Crawler Statistics and so forth.  

More detailed information can be found here 

http://redcopper-online-marketing.com/EN-US/web-applications/HKB-Project/Default.aspx 

HKB Capture Module

The HKB Project has a front end on the client side called capture, and a back end report and management system.

Capture records data and stores it in a XML file format for later use or parsing. The back end reads the data, and allows you to investigate data points further, to sort of go back in time to the point of origin, where the landing was generated from.  

Capture records direct landings, organic search result landings, and paid advertising landings. It also captures crawler or bot visits, in which crawlers are separated from browsers visits

HKB Crawler Report 

The crawler reporting module will assist you in knowing which crawlers are visiting and indexing your new website project. Use this module to confirm that all pages have been crawled. If you have a database page that uses query string parameters or URL rewriting, confirm that each instance of that page have been registered in the crawler visit journal. For example, I have a part look up page that represents a 1000 items, so I should see 1000 visits to that single page with each instance of the query string parameter.

If you’re not seeing 1000 instances of that single page, then you have an error in the page construction or the way you composed the HTML for it. Best practice is to write a sitemap generator program in your database code that will produce all 1000 versions of the page automatically in a second sitemap.xml file to feed into the search engine. When you have honed your skills to the master level, you can have new projects fully crawled in 7 days by the Google Bot. 

Once you have solved or confirmed your crawler issues, you may now begin analyzing organic page landings, and the keywords used to find them using the Landing Report. The landing report will give you the data needed to make adjustments to your page content. You will now see keyword matches or mismatches, and the page they landed on. Use this data to adjust the actual keywords that are being used to find your page to the keywords you thought they were going to use on the page itself. One example is the keyword "tooth whitening", "teeth whitening" is actually the correct keyword, so you change that word to the latter. 

HKB Keyword Report

Now you’re ready to use the Keyword module. All the different versions of keywords or phrases used are listed here sorted by page. A correctly constructed page will only have keywords and phrases that match that page exactly. You should not have overlaps in keywords for example; if you have 5 marketing bullet points, then you should have 5 unique pages, 1 for each bullet point. Only the home page should contain overlaps of all 5 keywords.

Once you have sorted out your keywords, you’re now ready for Paid Advertising. Use the Keywords in the Keyword module as a starter base for planning your Google Adwords text marketing campaign. Adwords will make suggestions to you to start with, but the suggestions are usually not accurate or too broad in subject. The keyword module will present all words, in which you can pick out the positive ones, and kick out the negative ones to use as negative keywords in your marketing campaign.

So if I take the keyword phrase "solar powered radio", I will get landings for all variants of that phrase such as "windup powered radio". "Powered radio" is the positive word or phrase, while "windup" would be the negative word. The purpose of positive and negative keywords is to tighten your results into a smaller target like a laser beam, and to conserve your advertising cash on only the leads you really want.

In most cases, when you launch a new website project, nobody knows that it exists except for you and the customer. So how do you spread the word about it? The best method is to launch an image advertising campaign, in which you create display ads in various sizes, and post them on other people’s website. Sort of like those billboards along the interstate, which do grab your attention. In this case, were looking for impressions over clicks at first. But when you get your image campaign fine tuned, then you start getting allot of clicks, which generates leads or actual sales. The Landing module will help you fine tune your image campaign, by giving you the information you need to make critical adjustments on a daily basis until its perfect. 

Using the code  

Download the Zip file to examine the sample project and its folder and file structure.

The program is compiled as a DLL file that you drop in the Bin, and then you register the capture program on your masterpages, or home default page.

The back end or admin pages are in a folder that is static in location, but needs write permissions, and the data is stored in the App_Data folder.

<div style="width: 100%; height: 34px;">
    <Capture:redCopper.HKB_Capture_Control id="HKB_Capture_Control" 
        runat="server" HKB_ShowCaptureProgram="True"
        Page_Theme="Black" Powered_By="redCopper" Width="100%" />
</div> 

Points of Interest

The program uses XML files to store data, and is very fast in operation. It is very good at distinguishing between bot or crawlers and browsers, to help keep the data as accurate as possible. There is a Rampart module, that will block bad bots or crawlers such as email harvesters, in which you can add to, or simply delete from.

History 

• March 25, 2013:  HKB Project V2013

This is the latest version of the project, and is very stable. The dashboard has been added for a quick daily view, and you can them navigate for detailed reports. The search engine vendors have been updated to reflect that Google handles Search results for AOL, and Bing handles Yahoo.  

How to use the demo project - please read this

The demo project is an example of how to setup the HKB Project in your project. If you unzip the demo and run the project on your developer computer in Visual Studio, it will do absolutely nothing. I will explain below.

In the lower left hand corner of the default page, is the HKB Project logo, that you click on, which will load the back end management interface. Now you can login using the credentials provided in the read me PDF file within the project demo.

The project ships with no data. I was going to provide live data, but the data would expire after 7 days, so in order to get data, the project needs to be located in a place where crawlers and bots can access it, so the capture program can begin the process of capturing and storing. Once you have some data, you can then use the back end management to analyze the data, and get a true feel for how it works.