How do I add custom URLs or recategorize existing URLs?
Web Security and Filter, Web Security Gateway, Web Security Gateway Anywhere, Websense Enterprise
6.3, 7.0, 7.1, 7.5, 7.6, 7.7, 7.8, 8.0
How do I recategorize existing URLs?
Custom URLs let you:
- Add sites to Websense software that are not in the Websense Master Database.
- Filter sites differently than their Master Database categories.
Websense Manager offers two custom URL lists:
Adding a custom URL:
- Custom URLs/Not Filtered lets you:
- Specify internet sites to be permitted for all users, regardless of how the site's category is filtered. (The exception is the Always Block category set, which blocks all Internet access.)
- Custom URLs/Recategorized lets you:
- Classify sites that are not in the Master Database by adding them to a database category. Websense software filters these sites according to the filtering setting for that category.)
- Reclassify sites that already exist in the Master Database. Single out sites to be filtered differently than their original categories. For example, block a site in an otherwise-permitted category.
- Select Recategorize URL from the Common Tasks pane.
- Create a category under User Defined.
- Click Add URLs.
- In the "add to" drop down list ,choose User-defined.
- Enter the URL that you want recategorized.
- Select a default filtering action, such as block, permit, etc.
- Click OK twice and select Save All.
- When recategorizing a URL, ensure that the URL had not been explicitly recategorized under its default category.
- If you need to get the URL for step 3 above, then run TestLogServer. For example:
- TestLogServer –file <file.txt> -onlyip <ip address>
- Visit the site in question, then cut from the TestLogServer output.
- Expand Custom URLs/Not Filtered or Custom URLs/Recategorized in the navigation tree.
- Select a category with which to associate the site.
- Click Add.
- Type the full URL for each site to filter under this category.
- Press Enter after each entry. Each URL must be on a separate line. Include the protocol and port for any non-HTTP site. For example:
If the protocol or port is omitted, Websense software filters the site as an HTTP
- Alternatively, enter URL patterns to be treated as custom URLs one at a time. URL patterns can include regular expressions. To verify that regular expressions match the desired URLs:
- Click Test URL Pattern. The Test URL Pattern dialog box appears.
- Enter the URL pattern to test.
- Type the intended (target) URLs. Press Enter after each entry.
- Click Match, and verify that the matched URLs are correct.
- Click Clear to clear the URLs and test another pattern.
- Click OK when finished.
- NOTE: Too many regular expressions can cause filtering to be processor intensive and cause the system to bog down. Tests have shown that with 100 regular expressions, the average CPU usage on the Filtering Service machine increased by 20%.
- Click OK.
- Click Save Changes above the navigation tree.
The new category now appears in the category list in the User-Defined tree.Note
: In version 7.7, the Not Filtered
Section has been removed. For more information, click here
Notes & Warnings
Websense software performs a string match against the Custom URLs exactly as they are entered. For example, if you enter http://www.yahoo.com as a recategorized Custom URL, then only if the entire URL is entered will there be a match, and the URL blocked.
Conversely, if you add just http://yahoo.com as a recategorized Custom URL, then any URL which includes "yahoo.com" in it will match and be blocked (i.e. mail.yahoo.com, games.yahoo.com, etc).
Also note, there is an implicit wild card after all re-categorized URLs. For example if you re-categorize http://www.yahoo.com then http://www.yahoo.com/additional_string/ will be re-categorized as well.
Note that many websites may host content on file servers (such as Akamai servers) and so additional URLs may need to be entered in order to block or permit a site as needed. The best way to capture this is to run testlogserver while accessing these sites which will give you visibility to any/all other sites that also need to be recategorized.
There is a difference in the entry when recategorizing https in a Limited Access Filter or custom category.
Additional Problems and Search Terms:
Allow or Block website access, Unable to recategorize a URL
Whenever possible, avoid using Regular Expressions, as their complexity increases load on the Filtering Service.
Using regular expressions as filtering criteria typically increases CPU usage. Tests have shown that with 100 regular expressions, the average CPU usage on the Filtering Service machine increased by 20%.
Websense Technical Support policies prevent technicians from assisting customers in the creation of regular expressions. It is a legal liability to provide customers with regular expressions if they do not work as intended or causes harm to the system. Therefore, regular expressions is a feature that is provided "as is" with no direct support that is intended to enhance the filtering capabilities and is the responsibility of the customers to learn and implement regular expressions and to thoroughly test the regular expression to avoid causing undue harm to the environment as to not overblock or underblock or cause the Filtering Service to max out the CPU utilization. The information contained here are for informational purposes only, and can cause filtering issues if used inappropriately such as not knowing *exactly* what you're doing.
Adding Regex Expressions to a proxy can incurs a dramatic impact on performance. A handful or few dozen Regex Expressions can cause the proxy to fail due to excessive load. As customer's networks are not generally the same, no rule of thumb is available for using Regex expressions. Every network should be analyzed individually to ensure an overload condition does not occur.
Using regular expressions
Avoid recategorizing sites that have been categorized as malicious sites. It is best to confirm the site has not been compromised by checking the Site Lookup tool and aceinsight.com. For more information, click here.