Solutions

Troubleshooting web pages that do not load or load incompletely

Solutions ID:    KB4827
Version:    1.0
Status:    Published
Published date:    12/15/2011
 

Problem Description

A blank page loads when going through the proxy

Some web pages do not load

URL does not load

Web page object does not display

 

Resolution

 


Instead of making the assumption that the proxy is at fault for any issue involving a page or part content not displaying in the browser, the following method makes it possible to identify the point of failure and allow the correct line of investigation to be taken.  Simply assuming the proxy is at fault with no further investigation will lead in the wrong direction if, as is often the case, the proxy is not at fault and is working correctly within protocols.  These steps can be used to find the nature of the problem:

 

 

Task One:  Identify the exact link or resource (by URL) that is not loading

 

This could be an image, some Javascript or the whole page. This is to ensure it is possible to track the request in diagnostic traces, to see why they do not get a response.  This might require trying the following techniques:

 

 

- Usually a page that fails to load will still have the URL in the browser address bar - make a note of it or take a screenshot of the browser page.

- If the resource not loading is a link you click on the page, hover the mouse on the link first to check the full URL.  Take a screenshot or make a note of it.

- If the object not loading is an embedded object on the page, right click on the page - view html source.  The objects on the page should be visible in the source by filename or url. 

- if you are able to get the page to load properly by bypassing the proxy, right click on the page to view the source code on the correctly loaded page, and look for the resource.  This can be difficult with complex web pages, but by searching neighbouring text strings, it's usually possible to find them.

 

 

In this example we, will assume we identified a .gif image that does not load, that has the full URL  http://www.facebook.com/profile_images/686180626/logo-big_normal.gif

 

 

Task Two:  Gather diagnostic data

 

Run the following two diagnostic tools simultaneously, test the site until you see the error, and stop the traces:

 

 

POLICY TRACE

 

Add these lines in local policy (Configuration -> Policy -> Policy Files -> Install Local File from "Text Editor", and add the lines to the end):

 

<Proxy>

client.address=x.x.x.x trace.rules(yes) trace.request(yes)

 

(where x.x.x.x is the client address)

 

Download the policy trace via https://IP.IP.IP.IP:8082/Policy

 

 

PACKET CAPTURE LOG

 

It is best to have an unfiltered capture, but if your proxy receives a high volume of traffic during the capture, it might be necessary to prepare a filter Proxy GUI > Console > Maintenance > Service information > pcaps > filter.

 

In the filter field, enter the following without brackets:

 

host <clients ip> || host <server/website ip or hostname> || host <ip of the dns server used by the proxy>

 

Start a packet capture via Management Console > Maintenance > Service information > Packet Capture > Start

Test the internet site(s)

Stop the packet capture Log Management Console > Maintenance > Service information > Packet Capture > Stop

 

And download the file as *.pcap or *.cap

 

 

 

Task Three:  Analyse the policy trace data.

 

Using a text editor tool, eg Notepad++, open the policy trace and search for the URL string.  This will show whether the request reaches the proxy and how it is handled by policy.  For eg, you might see something similar to this format:

 

GET http://www.facebook.com/profile_images/686180626/logo-big_normal.gif

DNS lookup was unrestricted

User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SV1; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; InfoPath.3; MS-RTC LM 8)

  user: name="DOMAIN\facebook.surfer" realm=IWAEircom

  EXCEPTION(custom): Either 'deny' or 'exception' was matched in policy

    url.category: Social Networking@Blue Coat

 

Any exception would explain a policy rule blocking the content.  Typically, you would also expect to see the exception displayed on the page, which would give a clear indication there is a policy rule blocking the content.

 

 

 

Task Four:  Analyse the packet capture data.

 

If there is no “EXCEPTION” or “DENY” action being hit, then it is time to analyse the packet capture with the Wireshark tool.  The goal is to find the same request that

We identified in step (1) above, and trace it through both client and server side request/response pairs, to find the point of failure.  This can be done by applying the following Wireshark filter techniques, although some knowledge of the http protocol is required:

 

 

-    Take all or part of the URL string, and apply it to the Wireshark filter, eg:

 

http contains “686180626/logo-big_normal.gif” or tcp contains “686180626/logo-big_normal.gif”

 

-    Search alternatives are to use the string search function (useful when the traffic is not on standard http ports) in Wireshark under Edit > Find packet > Find – String

-    A good method to scan all the http requests, depending on how big the capture is, is the filter “http.request.method”

 

Once the request is found, the goal is to track both the client<>proxy side of the exchange and the proxy<>website side.  Since there is no built-in way of associating the two sides with a session table of some kind, it requires some careful analysis:

 

-    Typically the search methods above will reveal the pair of client and “duplicate” proxy requests with no further searching.  One can then right click on both of the requests, and follow the sequence of request/response to see whether the website responds.  A typical http request/response pair consists of a GET request, and a 200OK response.  Failure response codes of 4xx or 5xx generally indicate server side failures, while 3xx responses imply an instruction to carry out a further action, such as serve from cache, or redirect etc.  Any number of sites detail this info, such as http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

 

-    If the website is down, or any upstream device prevents the proxy from creating a TCP session with the site, no proxy-side request will be seen.  This is more difficult to find in the capture, since the only clue will be TCP SYN packets sent to the website IP.  If the capture is not too big, it’s usually possible to see a series of SYN packets sent shortly after the proxy receives the client request.  A check with a “whois” site for the destination IP, or pinging the hostname as seen in the client request will confim the destination.  If there is no response, the website could be down or not responding, or any upstream network device (such as a Firewall) may be blocking the SYN packet, or the returning SYN,ACK from the website. 

 

-    If the proxy does not receive a SYN,ACK packet from the website IP, then logic would direct the investigation to the upstream network.  Such question should be asked as: does the firewall receive and forward the SYN packets?; does the Firewall receive and return the SYN,ACK response from the site?

 

-    If it can be seen that the proxy does not attempt to contact the site, rule out a DNS issue.  If the capture is unfltered or includes the DNS server(s) IP address(es), the DNS request and response should be visible.

 

-    If this proxy is being used an ADN web gateway, it’s worth ruling out the issue as described here:

 

KB4771 - "Unknown" content in SYN packet causes some websites to refuse connections to ADN enabled web-gateway proxies.

 

 

Other related solutions:

 

KB4771 - "Unknown" content in SYN packet causes some websites to refuse connections to ADN enabled web-gateway proxies

KB3754 - Some web page load slowly, or a blank page loads when going through the proxy (RFC 1323)

KB3528 - www.google-analytics.com is causing web pages to load slowly or sometimes not load at all 

FAQ739 - Flash Video does not play when going via a Proxy

 


Rate this Page

Please take a moment to complete this form to help us better serve you.

Did this document help answer your question?
 
 
If you are finished providing feedback, please click the RATE CONTENT button. Otherwise, please add more detail in the following text box and then click RATE CONTENT.
 
 

Your response will be used to improve our document content.

Ask a Question