With both versions 8, and 9, of Reporter an Access Logging configuration can be setup, such that the data from the ProxySG can be streamed directly into the Reporter database. in this case, no acess log file is every kept on the Reporters file system, but the data is directly injected, live, into the Reporters database.
NOTE1: Using Reporter 9.1.x versions is recommended for all streaming connections, going forward. The steps, and troubleshooting tips, found in this article are intended for users with this version.
NOTE2: if your organization has concerns regarding disaster recovery of the Bluecoat Reporter 9 database, it is recommended you use the FTP client to upload access logs to either the Reporter server, or a separate FTP server. From here you can configure the Reporter server to procure and process these access log files off disk into the database. In this case, you can also use these files as a backup if your database does become corrupt.
Steps to set this up (Reporter side):
- Install Reporter.
- Once you log in you will be sent to the administration part of the User Interface (UI).
- Create a database by clicking on "General settings", and then "Data settings.
- Once here, click on the "New" button and choose these options:
Fill in your preferred database name.
Leave the log sources blank at this time, and click next.
Answer the note about not having any log sources.
Ensure you have set the right expiry schedule, and press next.
Ensure the database will be created in a place your happy with, then press Done.
Steps to setup up (ProxySG side):
NOTE: This streaming configuration is available on the latest SGOS versions of both the 4, and 5, series software.
- Login to the admin UI of your ProxySG.
- Go to Configuration menu options, or administration section.
- Choose "Access logging". and configure these things.
- Ensure that 'Enable Access logging is clicked on the general tab
- ON the "Logs" tab click on "Logs" option/tab, and ensure the log type you want to send to Reporter ( ordinarily it will be called Main) has the bcreportermain_v1 log type selected.
- On the "Upload client" tab choose the log type of "Main", and select the client type of "Blue Coat Reporter Client"
- Once selected, then click on the "settings" button to configure the IP address of the Primary Bluecoat Reporter Server. Unless you have changed the port on the reporter server, keep the port set at 9081.
- Do not use the "test upload" button, as this was intended for the FTP client.
- Keep the log file as "GZIP" format, however, if you've already been running it as a text file, Blue Coat does not recommend you change it. (The Bluecoat Reporter server can handle both, but breaks if you swap it mid-stream).
- Click on upload schedule and ensure you have selected the LOG called "Main". Here select "continuously" with the default parameters of "wait between connect attempts:" of 60 and "time between Keep-alive Log packets" of 300.
- On the same screen ensure you are rotating on at least a daily schedule of 2AM, but we recommend a weekly schedule of Sunday at 2 AM. Do not rotate it at frequencies of less than a day.
- Press Apply to save the changes made to your configuration above.
Final steps to setup (Reporter side):
Once you have configured the ProxySG to begin to stream the access log to the Reporter server, we should begin to see a log source show up as "unassigned" on the Administration screen.
Here's how to check this:
- To see this, login to the Reporter UI, and go the Administration section.
- From here click on "Data Settings" and then "Log sources"
- At the bottom of all your configured Log Sources, you will see a list of "unassigned log sources" where your ProxySG should be appearing.
NOTE1: We should begin to see a log source show up as "unassigned" on the Administration screen within ten minutes. Once it appears, you merely need to go click on the 'actions' tab, and assign it to the database you created above, and within minutes you should start seeing real data, from this SG, in your database.
NOTE2: If you are streaming multiple SG'S into this one Reporter server, you will be able to identify each by their IP address, and serial number, in the unassigned queue.
If a unassigned log source does not appear in the Reporter UI in about ten minutes, you might need to start using the SG as a cache, so it has some data to send. If it does not appear in about ten mins after that, then you can use the 'statistics' screen on the SG to determine if the SG is logging, and actually sending a any data.
Here's how on the SG side:
- Login to the Administration UI of the SG, and load the Admin console.
- Once here, click on the statistics screen., and then click on 'access logging'.
- Here you should see a live demonstration of what is actually being logged to the access log.
- If there is no data on this screen, you should re-visit the configuration steps above to determine why it is not logging data.
- If it is logging data to the screen, then click over on the 'upload status' screen to determine if it is sending this data to the reporter server.
- Clicking on the "log Size" tab here, also shows whether or not the log is actively logging.
- If it is not sending, go back to the configuration section and check that you are using the "Primary Bluecoat Reporter server" option for you IP address. Clicking on the settings button shows this.
- While not recomended in normal operations, pressing the button "rotate now" on the access logging configuration screen can often highlight connectivity issues too
- If it is still not sending the access log data, a PCAP may need to be taken to find out why it can't connect to the Reporter server.
- If a PCAP reveals that the SG is sending to the Reporter server, on port 9081, then you may want to find out who is holding that port up on the Reporter server by executing a netstat -a command on the command line window.
- In some cases, it is prudent to disable the streaming connection, while you delete the access log off of the SG cache. This would be advisable in cases where there is a huge backlog of data, and the SG will take days to stream all the data to the Reporter server. To do this, you need to access the CLI of the SG. Here's how.
- Blue Coat SG200 Series#config t
Blue Coat SG200 Series#(config)access-log
Blue Coat SG200 Series#(config access-log)edit log main
Blue Coat SG200 Series#(config log main)commands delete-logs
Note: Just rinse and repeat for all other log names. Don’t get confused between log names
and log types.
Here's how on the Reporter side:
- Running Reporter, version 8, on the same server may be blocking access to port 9081, on Reporter, version 9. If this was the case then Blue Coat recommends stopping both Bluecoat Reporter versions, and then only starting Blue Coat Reporter 9.1.x back up.
- Reporter, version 9.1.x, can show you the history of log source restarts in the "System overview" section of the UI. Here, simply click on "System diagnostics, and a navigate down to the Database overview section. From here click on the "History" link of the database you want to check on. Check here if you believe the SG is droping the connection to the Reporter server frequently. If you set the SG up, as instructed above, the connection should only rotated once a day, or once a week.