Creating a validation script

Administrator Help | Forcepoint DLP | Version 8.5.x

Fingerprinting cells with some values, such as multiple short values, can lead to multiple false-positive incidents. Forcepoint DLP includes a mechanism that forwards database data to an external script for processing before fingerprinting.

Validation script mechanism

Each database fingerprint classifier can use a validation script. The validation script receives an input file containing the raw database data in a CSV format, and returns CSV data containing the information that should be fingerprinted.

Validation scripts must be designed to receive at least two parameters: an input path name and an output path name. An additional parameter, the configuration file path name, is optional.

The input file is a CSV file with a header row containing the database column names. Each line is delimited by a valid windows line break (CRLF), and all values are double-quotes escaped. A sample package containing a sample input file, among other things, is available from Forcepoint Technical Support.

The output file has the same format as the input file, but instead of using CRLF as the line delimiter, it uses CRCRLF (2 carriage-return characters and one line-feed character). An output sample file is available in the same package as the sample input file.

Validating fingerprinting scans

To validate your fingerprinting scans:

Optionally, create a copy of the following files in the \ValidationScripts folder where Forcepoint DLP was installed (typically C:\Program Files\Websense\Data Security\ValidationScripts).

default_validation.bat.sample

default_validation.ini.sample

To create your script from scratch, skip this step.

Name your new validation script using the following convention:

<classifier-name>_validation.[bat|exe|py]

Here:

<classifier-name> is the name of the classifier on which the script will be run. Alternatively, use the word "default" for scripts that run on all classifiers that don't have specific scripts named after them.

bat is the extension for a batch file.

exe is the extension for an executable.

py is the extension for a python script.

If the script requires a configuration file, name the configuration file using the following convention:

<classifier-name>_validation.[xml|ini]

Place all files in the \ValidationScripts folder on the server where Forcepoint DLP is installed (typically C:\Program Files\Websense\Data Security\ValidationScripts).

Every validation script must be an executable or a batch file. If there is a need for an infrastructure element, for example the python interpreter, the operating system must be able to automatically initiate the element when the script is being called. To ensure the correct file association is configured, Forcepoint recommends running the script from the command line, without reference to any other executable.


	Note Pay attention not to leave more than one executable or configuration file with the same name and different extension in the validation scripts directory.

The script should receive 2 command-line parameters from Forcepoint DLP: the full path of a source file the system creates, and the full path where the system expects to find a destination file.

The first line of the source file includes the names of the columns that are available for fingerprinting. The remaining lines contain the data in those columns.

The script should read and perform validation on the source file.

The script should write the validated results to a destination file.

The destination file should be formatted in the same way as the source file—with the names of the columns that were fingerprinted on the first line. Note that the number of columns varies if your script adds or removes columns.

The destination file must use the name and path that received from Forcepoint DLP.

The script should return a return code of 0 if everything succeeded, and non-zero if there was a problem.

To have the script use a configuration file, place the configuration file in the same location as the script, and name it with the same name as the script file followed by .xml or .ini. If this file is found, it is supplied as a third parameter to the script.

Create and run the fingerprinting classifier as described in Creating a database fingerprint classifier. Name the classifier with the name given in step 2.

During the scan, if the crawler finds a script with the following name format, it runs that script:

<classifier-name>_validation.[bat|exe|py]

If it does not find a script with that naming format, it searches for a script named default_validation.[bat|exe|py] and runs that.

If the crawler receives a non-zero return code from the script, the fingerprinting process stops and an appropriate error is returned. In this case, you can either fix the script or remove it then refingerprint.

When the system finds a validation script, the Sample Data screen in the database fingerprinting wizard shows validated data, and not the raw data extracted from the database/CSV. (This is on the Field Selection page of the wizard, where you click View Sample Data.) You can use this to make sure that the validation script behaves as expected, and to see the exact information that is protected.

To run the script on subsequent fingerprint classifiers, copy the script and rename it.

Sample validation script

There is a sample validation script in the \Validation Scripts directory where Forcepoint DLP is installed. The script contains the basic abilities required for most customers, such as removing NULL or single-character values from being fingerprinted. You can modify it to suit your needs.

The sample package contains the following files:

default_validation.bat - Sample validation script

validation_logic.py - Used by the sample validation script.

default_validation.ini - Sample configuration file

default_validation.ini.sample - An additional configuration sample file

dictionary.txt - Sample dictionary file

in.csv - Sample input file

out.csv - Sample output file

The first 3 files are also included (with the .sample extension, for the batch and ini files) in the Forcepoint DLP installation package.

The sample validation script is a production grade script, which is suitable for many organizations.

Please note that although "default_validation.bat" and "default_validation.ini" files can be renamed according to the conventions mentioned above, do not rename the "validation_logic.py" file. This file must be present in the \ValidationScripts directory (typically C:\Program Files\Websense\Data Security\ValidationScripts) in its original form.

The validation script is predefined to make sure Forcepoint DLP ignores:

Numbers smaller than 10,000.

Text strings containing fewer than 4 characters.

Strings containing only zeros (i.e., "000000").

Empty strings.

Placeholders (NULL and similar values).

Invalid SSNs in columns named "ssn."

Invalid email addresses in columns named "email."

The following additions and changes can be configured through the "default_validation.ini" configuration file:

It is possible to create a dictionary file that contains a list of strings for the validation script to remove. The file should be a line delimited UTF-16 file, and its path name should be written in the IgnoredDictionary configuration option in regular file system format. (For example c:\directory\dictionary.txt.)

Administrators can create UTF-16 files in Windows Notepad by saving the text with "Unicode" encoding.

An example of this can be found in the "default_validation.ini.sample" file.

A sample dictionary file—"dictionary.txt"—is also provided.

Regular expressions can be used to validate any column. To use this feature:

Add the column name, in lower case, to the columns parameter. Separate column names by semicolons.

Add a configuration section for the column by appending [column-name] to the file (again, lower case). This is the section header.

Add a RegExp parameter under the relevant (newly added) section header. Its value is a regular expression.

The default_validation.ini sample file contains this type of validation for email addresses and social security numbers. These can be used as a reference.


	Note Additional configuration options are available. Contact Forcepoint Technical Support for further assistance.