Hackers and their malicious programs may gain access to your network via Internet connections. This can be avoided with web filtering (through URL classification).

A web filter lets corporate managers and network administrators avoid employees from accessing URLs that contain content material that the organization finds objectionable. Decisions on objectionable content material are generally determined on the basis of legal or ethical concerns, employee productivity, network security, or any other issue. By using a URL classification web filter, you can ban URLs that are classified in everything from pornography to social networking.

Chris

One answer to this growing concern is automated URL classification. This method uses statistical techniques to uncover the properties of malicious and malware website URLs. These techniques would be capable of understanding and analyzing signs of potentially suspect URLs with very few false positives.

Obviously, URL classification has enormous benefits to the network adminstrator. As an adminstrator, it is always frustrating when a network user exposes the system to a threat that the administrator would never allow if he were present. URL classification allows the administrator to have a proxy presence and deny the suspicious website automatically.

 Chris

Well, since the Internet gains and loses hundreds of thousands of sites per day, I think this is just one task that my friend is not going to be able to tackle on his own.  His DIY (do it yourself) skills have certainly met their match.

When I called him last and asked him how he was doing, he sounded kind of frustrated.  He said he would call me back when he was done.  I am expecting that call on the first Monday of never!

Chris

The question I had the other day was about which portable camping grill was the best.  I received about a dozen answers and one of them sent my to a nice little portal site that specialized in portable camping grills.  I found good reviews and very low prices.

The problem arose when I tried to award the answer that I liked best as the “Best Answer.”  For some reason my security software had that page of Yahoo Answers classified as a computer threat.  When I scanned the page, there were no problems, but my security software was not to be persuaded.

That is one potential limitation of URL categorization:  URL mis-classification.  If the URL is classified incorrectly, then potentially every page of the site can be as well.

The other side of this coin is a similar problem.  If the site is fine when it is first classified but then becomes infected at a later date, then you will have an undetected malware site within your categorization as a clean site.

Chris

Well, empty URLs create the opposite (but equal) problem for URL classification.  How could a URL classifying software program correctly classify a site such as wbj.org or wsj.com as a business site based wholly on the URL.  It can’t.  And the reason is because the URL gives no evidence to its true content.  This is what is called an empty URL.

However, most of your empty URLs will still have other classifiable pages that are indexed.  wbj.org/business-in-washington is an example.  So, even your empty URLs can be classified eventually – once other pages from within the site are classified themselves. 

Still, it does warrant mentioning that empty URLs are a potential limitation of the URL classifying method.

Chris

Port blocking and URL categorization is a way to effectively accomplish this.  URL categorization creates a database of URL addresses and these can be classified in a multitude of ways.  Facebook, as an example, may be classified as “social networking.”   By using port blocking, the administrator could block all sites in the “social networking”category.  This would stop all visits to every site within that category (like MySpace) - and not just Facebook.

The limitation, of course, would be in any site that is not yet classified or is not classified within the “social networking” category.

Chris

What happens when a user on your network is the first to stumble upon a malicious or infect site?

Well, since there is no record of the site within the category of malware or malicious sites, the URL is allowed.  Your network has just become a guinea pig.  Now that you have suffered this attack, the bad URL is correctly classified under malware and every subsequent visit is blocked.

Thanks to your network’s sacrifice, all future visitors are forewarned and saved.  Unfortunately, that will not help you one single iota.

I am not suggesting that URL classification is flawed in the least, I just wanted to post information regarding the one particular limitation.

Chris

So much more can be done with URL classification today.  Web content can be classified as you go now.  This way you get URL classification that is immediate and that lives up to the required need of the individual.  It is important to have URL classification in current time.  This helps to find the right classification slot as well. 

You can get link analysis and classification.  You can get protection from unwanted content online also.  Site history can be checked to see if it is reputable.  Malware can be detected as part of this as can any software that could harm your computer. 

Do you want your URL classification and categorization system to build a database for you as you search in real time?  This is the kind of benefits you can now expect.

Chris

Certainly you can identify a web page  by its content.  You can use the actual text or hypertext or incoming/outgoing links or a combination of all these and others.  So if we follow the old adage: “If it aint broke, don’t fix it,” why would we want to use URL classification?

URL identification is the preferred classification when speed is a critical issue or when you want to alter website content before it is downloaded if it is of a questionable nature.  It also helps when website images can’t be seen for content analysis.  There are also several other reasons – depending on your needs.

URL classification is a great tool and will become more widely used in the future.

Chris

Some websites simply have too much content that is all over the spectrum.  There is so much text, so much content, that the categorization of the website is muddled.  This phenomena is known as word noise, too many words.

It is true that some form of text summarization techniques have proven to be useful in these types of situations; however, an equally effective (if not more so) and less expensive method for classification in this situation would be URL categorization.

You may wonder if an incredibly large URL address might be considered a form of word noise when employing this form of classification.  Yet, even your large and ungainly website URL addresses are generally broken up into your root and then either post names, categories, archives, pages, blog titles, and tags.  Rather than creating word noise, these website URL practically categorize themselves due in large part to modern SEO techniques.

Chris