Almost from the day that the Web was invented, people wanted to track information about the pages delivered and the computers requesting that information. This information is stored on the Web server in generally one of two formats:
- Common Log Format (CLF)
- Combined Log Format
Common Log Format (CLF)
The Common Log Format (CLF) is a fairly basic form of Web server logging. It tracks seven different elements of the Web transaction. Each request is written to one line, with the different elements of the request separated by spaces (items in quotes or square brackets are considered one item), and items that aren't sent are listed as a hyphen or dash (-):
- The remote host
The hostname or IP address of the computer requesting the Web page. - The client user name
The name of the user on the client computer (defined by RFC 1413), assuming the Web server is snooping for that information. - Authenticated user name
If the client authenticated to get the page, the username will be written to the log (but not the password). - Date
The date and time the request was received. - The Request
This is the actual HTTP request line as it was sent from the client computer. - Status
The status refers to the HTTP status codes indicating if the request was successful and so on. - Bytes
The number of bytes that were sent in the server's response.
Here's how the log entry might look for this Web page in Common Log Format:
10.1.1.1 - - [08/Feb/2004:05:37:49 -0800] "GET /cs/loganalysistools/a/aaloganalysis.htm HTTP/1.1" 200 2758
Combined Log Format
However, it was quickly discovered that there were some items missing from the CLF that Web developers needed to know to provide better service to their customers. The combined log format uses the common log format but adds two items to the end:
- Referrer
The URL of the page that linked to the requested document. - User-agent
The name and version of the browser or other client software making the request.
Here's how the log entry might look for this Web page in Combined Log Format:
10.1.1.1 - - [08/Feb/2004:05:37:49 -0800] "GET /cs/loganalysistools/a/aaloganalysis.htm HTTP/1.1" 200 2758 "http://webdesign.about.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; YPC 3.0.2)"
Use of Web Logs
Most Web servers write Web logs in one of the above two formats. This means that many log analysis tools expect logs to be in these formats in order to work. While it is possible to configure your server to log in the format that you prefer, if you plan on using a tool to analyze your logs, you shouldn't change it.
How to Convert from Common Log Format to Combined Log Format in Apache
