Solutions

Proxy stops passing traffic, Client Workers (CWs) get maxed out and also SmartFilter content filter database is in use

Solutions ID:    KB3709
Version:    2.0
Status:    Published
Published date:    02/24/2010
Updated:    02/26/2010
 

Problem Description

All traffic through the Proxy halts and the client workers rise to the maximum possible on the Proxy.

Smartfilter is the Content Filter Database in use and Content Filter Memory Allocation is set to Normal

You are running an SGOS version below 5.4.3.*  

Observing the content filter download status page URL https://<IP_Address_of_SG>:8082/ContentFilter/Status gives output similar to the following:

Download log:
  SmartFilter download at: 2010/02/24 22:53:36 +0000
  Downloading from: list.smartfilter.com
  Checking incremental update
    Installed database version: 21840
    Current published version:  21840
    Database is up-to-date; no download required

Previous download:
  SmartFilter download at: 2010/02/24 21:49:48 +0000
  Downloading from: list.smartfilter.com
  Checking incremental update
    Installed database version: 21839
    Current published version:  21840
    Applying incremental updates
  Incremental download complete
  Download size:      365737920
  Building database
    Database version:   21840
    Database date:      Wed, 24 Feb 2010 21:25:36 UTC
    Database expires:   Wed, 31 Mar 2010 21:25:36 UTC

Memory Allocation:                      Normal

Using a memory allocation setting of normal (as seen above) allocates up to a maximum of 350MB of memory for the content filter database.

However, it can be seen in the above output that the database is larger than this setting:

  Download size:      365737920

You may also see High CPU when this problem occurs.  If so the majority of CPU consumption will be in HTTP policy Evaluation.

Resolution

Upgrade to SGOS 4.3.3.9 or above and set the Content Filter Memory Allocation setting to HIGH

This can be done using the Management GUI:

Configuration > Content Filtering > General

NOTE:

If you are already on 4.3.3.9 then after setting the memory allocation to HIGH you may have to reboot the proxy in order for traffic to start flowing correctly again.

The amount of memory it is possible to allocate by each setting varies in accordance with the following table:

 Total RAM   low   normal   high 
 512MB   1MB   50MB   50MB 
 1GB   1MB   120MB-200MB   120MB-200MB (Depending on platform) 
 1.5GB   1MB   200MB   350MB 
 2GB   1MB   250MB   600MB 
 3GB   1MB   250MB   1.1GB 
 4GB   1MB   350MB   1.6GB 

However, this setting is only a ceiling so the high setting won't result in the consumption of more memory than necessary for the database.

On a Proxy with less than 2GB of memory the current size of the Smartfilter Database is too large to fit into memory. Hence, you should consider upgrading your proxy, or if it is capable of running SGOS 5 upgrade it to SGOS 5.4.3.

5.4.3 allows better paging of the database to disk, but if this becomes necessary there is still the possibility of a performance degradation as a result.


Rate this Page

Please take a moment to complete this form to help us better serve you.

Did this document help answer your question?
 
 
If you are finished providing feedback, please click the RATE CONTENT button. Otherwise, please add more detail in the following text box and then click RATE CONTENT.
 
 

Your response will be used to improve our document content.

Ask a Question