Solutions

How to get a similar statistics through SNMP walk, similar to proxy SG built-in GUI > Statistics > Protocol Details > HTTP/FTP History.

Solutions ID:    KB4889
Version:    1.0
Status:    Published
Published date:    01/25/2012
 

Problem Description

The stats/graphs available through proxy SG built-in GUI > Statistics > Protocol Details > HTTP/FTP History are not available for arbitrary period of time. If we can same type of information for arbitary time periods, it is not possible through the built-in graphs.

We would like to get hourly graphs similar to what is available under GUI > Statistics > Protocol Details > HTTP/FTP History. Three types of graphs are available there, these are:
1.      HTTP/HTTPS/FTP Objects
2.      HTTP/HTTPS/FTP Bytes
3.      HTTP/HTTPS/FTP Clients

Resolution

 

We can emulate #1 and #2 of these graphs for any arbitrary time period through periodic SNMP GET requests. However, it should be noted that the same cannot be done for #3. Also #3 is a bit of a misleading statistics. So basically, not having this capability in SNMP is not a major issue. Further discussion on what this #3 shows is given in this KB:

 
ProxySG supports most of the MIB-2 (RFC 1213), the Proxy MIB and the RFC2594 MIB (wwwMIB). Here are some useful OID's from these MIBs that are pertinent to this discussion:
 
#a
proxyClientHttpRequests
Requests OID: .1.3.6.1.3.25.17.3.2.1.1.
The number of HTTP requests received from clients.
 
#b
Client HTTP Traffic
proxyClientHttpInKbs .1.3.6.1.3.25.17.3.2.1.4 The number of kilobits received from the clients by the proxy.
 
#c
proxyClientHttpOutKbs: .1.3.6.1.3.25.17.3.2.1.5.
The number of kilobits delivered to clients from the proxy.
 
Please note that, #a gives the same information as #1 or HTTP/HTTPS/FTP Objects, as mentioned above and #b and #c together give the same information as #2 or HTTP/HTTPS/FTP Bytes, as mentioned above. Kindly note that, the counter for #2 is showing the sum of #b and #c, in a cumulative manner. Also there is a difference in unit involved here. The ProxySG built-in graphs report these values in Bytes, whereas the SNMP values will be in kb or KiloBits. Also note that SNMP values are essentially integer values, so there is some loss of information but the deviation is not that major.
 
Test Cases to verify the above:
To verify this information, we can run two sets of tests through a proxy configured explicitly. There is only 1 client involved in two test cases. So setup was simple: Client machine -----Proxy----------Internet. While MIB browser or SNMP manager was talking to the proxy at (10.78.55.202)
Test case #1
Client machine makes only 1 request to example.com
Packet capture also shows only 1 request coming to the proxy and on the proxy we see the following graph:

 

Figure01: ProxySG HTTP/HTTPS/FTP Object graph with 1 request
As can be seen there was only 1 request in this graph.
Looking at the Byte graph we see this:

 

Figure02: ProxySG HTTP/HTTPS/FTP Bytes graph with 1 request
As can be seen there was a total of 11641 Bytes of information transferred. Please note that the tooltip value show is in Bytes whereas the graph scale is in KBytes. A packet capture during this transaction also confirms that together with the GET request and the corresponding response, there was a total of 11641 HTTP bytes (not TCP or IP or lower level headers), transferred.
Now if we also have a MIB browser poll for #a, #b, #c going on at the same time. Before the request was sent we polled the three above mentioned OIDs and received:
proxyClientHttpRequest.0 = 253, proxyClientHttpInKbps.0 = 960 and proxyClientHttpOutKbs.0=12306
 
These values are cumulative and are sustained from the beginning of the restart. Unlike trend statistics, these cannot be cleared.
Now after the above mentioned transaction has gone through, we poll the same SNMP OIDs again. We receive the following:
proxyClientHttpRequest.0 = 254, proxyClientHttpInKbps.0 = 962 and proxyClientHttpOutKbs.0=12396
We can come up with the following table with these two sets of values:
 
Table01: Test case #1 MIB values
 

OID
OLD value
New Value
Difference
Interpretation
proxyClientHttpRequest
253
254
1
This the object count of proxy graph
proxyClientHttpInKbps
960
962
2
Adding this to the next one and unit converting we get the Bytes as in proxy graph
proxyClientHttpOutKbs
12306
12396
90
 

 

 

So adding the differences of proxyClientHttpInKbps and proxyClientHttpOutKbs, we get 90 + 2 = 92. This is 92 KiloBits. Converting this to bytes we get (92 * 1024 )/8 = 11776 Bytes. As we can see this is close to 11641 Bytes as mentioned in proxy SG bytes graph. As noted above, there is some loss of precision due to integer reporting but for practical purposes and over a longer period of time this is insignificant. So, the general formula would be: For any time period T,
Byte Count =(((proxyClientHttpInKbpsnew -proxyClientHttpInKbpsold ) + (proxyClientHttpOutKbsnew – proxyClientHttpOutKbsold))*1024)/8
 
(Please note that we are dealing with 32 bit counters here, at some point, these might overflow, then we would  need to modify the difference calculation by (2^32 - old_value + new_value)
Test Case #2
In this case, we can use a script of some software to generate
10 requests to example.com in 200ms interval.
With this load on the proxy. We get the following graphs:

 

Figure03: ProxySG HTTP/HTTPS/FTP Object graph with 10 requests

 

Figure04: ProxySG HTTP/HTTPS/FTP Object graph with 10 requests
 
As can be seen object count is 10 and the Byte count is: 88907 Bytes.
Doing a similar SNMP poll as mentioned earlier we see the following:
Table02: Test case 2 MIB values
 

OID
OLD value
New Value
Difference
Interpretation
proxyClientHttpRequest
254
264
10
This the object count of proxy graph
proxyClientHttpInKbps
962
988
26
Adding this to the next one and unit converting we get the Bytes as in proxy graph
proxyClientHttpOutKbs
12396
13064
668
 

 
Doing similar arithmetic with the last two values here we get, (26 + 668)*1024/8 = 88832 Bytes. This is pretty close to SG graph reported 88907 Bytes. There is some loss of precision due to integer reporting again. But as can be appreciated, it is becoming more negligible with more traffic. In this very small test it is less than 0.1%...
So ultimately, we can generate similar graphs to the SG with polling these SNMP OIDs. To get per minute stats/graph, we’d need to poll every 60 seconds and manipulate the results thus received and graph it to a suitable scale.
We can use some freely available Perl scripts or other scripts to automate this.
Otherwise, we can use Open Source software Cacti, available here:
With this software it is possible to generate graphs for arbitrary OIDs.

Rate this Page

Please take a moment to complete this form to help us better serve you.

Did this document help answer your question?
 
 
If you are finished providing feedback, please click the RATE CONTENT button. Otherwise, please add more detail in the following text box and then click RATE CONTENT.
 
 

Your response will be used to improve our document content.

Ask a Question