How does the logging process work?

51131 | Reporting FAQ | TRITON AP-WEB and Web Filter & Security | 12-Oct-2015

When users browse the Web, their activity as recorded as log data:

Network Agent, Content Gateway, or a third-party integration forwards the Internet request to Filtering Service.

Filtering Service determines the appropriate response to the request.

By default, Filtering Service forwards a record of the transaction to Log Server.

If you have enabled integration with a third-party SIEM tool, Filtering Service forwards the request instead to Websense Multiplexer, which then sends the request to both Log Server and the SIEM tool. See Websense SIEM Solutions for more information.

Log Server stores the data in temporary cache or BCP files on the local hard disk.

When the load of incoming data is not too heavy, Log Server performs preprocessing on the cached files and forwards them to the Log Database. Log Server can:

Process multiple, similar log records into a single record in a process called log record consolidation

Combine the elements that make up a web page (like advertisements, graphics, and text) into a single record using visits processing

Information about Log Server preprocessing options can be found in the Administering Websense Databases paper.

The Log Database temporarily stores each log record in a table in the catalog database.

Database jobs move the data to various tables in the partition databases. For more information about how the jobs work, see Database jobs.

Data in the partition databases can be used in dashboard, investigative, and presentation reports (see Use Reports to Evaluate Internet Activity).

ODBC and BCP

Log Server can forward log records to the Log Database using either of 2 formats:

With ODBC (Open Database Connectivity), Log Server inserts records into the database individually. The data stored in Log Server cache files is moved directly to the database.

With BCP (Bulk Copy Program), Log Server inserts the data in batches. To do this, Log Server starts by moving the data from the cache files into a new set of storage files. From there, the data is moved to the database.

Configure data insertion options in the Web module of the TRITON Manager on the Settings > Reporting > Log Server page.

For more information about ODBC and BCP, see Specify how log records are processed into the database.