update.config

The update.config file controls how Websense Content Gateway performs a scheduled update of specific local cache content. The file contains a list of URLs specifying objects that you want to schedule for update.

A scheduled update performs a local HTTP GET on the objects at the specific time or interval. You can control the following parameters for each specified object:

The URL

URL-specific request headers, which overrides the default

The update time and interval

The recursion depth


	Important After you modify this file, run content_line -x from the Content Gateway bin directory (/opt/WCG/bin) to apply the changes. When you apply the changes to a node in a cluster, Content Gateway applies the changes to all nodes in the cluster.

Scheduled update supports the following tag/attribute pairs when performing recursive URL updates:

Scheduled update is designed to operate on URL sets consisting of hundreds of input URLs (expanded to thousands when recursive URLs are included); it is not intended to operate on massively large URL sets, such as those used by Internet crawlers.

Format

Each line in the update.config file uses the following format:

URL\request_headers\offset_hour\interval\recursion_depth\

The following table describes each field.


Field	Allowed Inputs
URL	HTTP and FTP-based URLs.
request_headers	(Optional.) A list of headers (separated by semi-colons) passed in each GET request. You can define any request header that conforms to the HTTP specification. The default is no request header.
offset_hour	The base hour used to derive the update periods. The range is 00-23 hours.
interval	The interval, in seconds, at which updates should occur, starting at offset hour.
recursion_depth	The depth to which referenced URLs are recursively updated, starting at the given URL.

Examples

The following example illustrates an HTTP scheduled update:

http://www.company.com\User-Agent: noname user agent\13\3600\5\

This example specifies the URL and request headers, an offset hour of 13 (1 p.m.), an interval of one hour, and a recursion depth of 5. This would result in updates at 13:00, 14:00, 15:00, and so on. To schedule for an update to occur only once a day, use an interval value of 24 hours x 60 minutes x 60 seconds = 86400.

The following example illustrates an FTP scheduled update:

ftp://anonymous@ftp.company.com/pub/misc/test_file.cc\\18\120\0\

This example specifies the FTP request, an offset hour of 18 (6 p.m.), and an interval of every two minutes. The user must be anonymous and the password must be specified by the variable proxy.config.http.ftp.anonymous_passwd in the records.config file.