How Do I CGI ?
BDL How To
BDL Log Tips
How It Works
A good deal of the functionality presented on this page refers to the Deluxe Edition of the Bestdam Logger script. However, given the level of functionality built into the Lite Edition most of the information also pertains to it.
The Bestdam Logger script is a combination counter script and logger script. In addition to maintaining site hit counts and logs of visitor information, it will periodically (daily, weekly, or monthly) e-mail a report to you. This e-mailed report contains the current hit counter reading, the contents of the hit history file, the current size of the logging history file, and the contents of the period log.
The concept of "periods" are central to the operation of Bestdam logger. During each period visitor information is logged to the period.log file. At the end of each period this log is cleared after the contents are included in the e-mailed report (and optionally added to the history.log file). There is also an option to reset hit counts to 0 at this point also.
The script three primary data files, all of which are ordinary ASCII text files:
The pagehits.cnt file contains the hit count number. The script reads in the number contained in this file, increments it with each page hit (if hit counting is enabled), and writes the incremented value to the file replacing the old number. Because it is an ASCII text file, you can change the hit count number simply by editing the file using any text editor. Just be sure to use ASCII mode when you transfer the file from and to the server. The hit count number can optionally be displayed on the page using text digits, which can be formatted using standard HTML text formatting tags, or graphics digits of our choice.
It is important to note that the counter is incremented every time the page is hit, or loaded. That even includes multiple hits from the same visitor. Multiple hits from the same visitor are common if you have a "Home" link on your sub-pages (and home page hits are whats being counted). This is called "counter bloat" because it does not accurately represent the number of unique visitors to your Web site.
The period.log file contains the visitor information that captured with each hit. Each line in the log represents one hit. The Log Tips page details the information that is collected and what it can tell you.
Like the counter file, the period.log file is updated with every hit. However, unlike the counter file, having every hit from an individual site visitor can be valuable as it allows you to somewhat track the path a visitor took through your site. If you are monitoring the home page, and the visitor goes to it from a sub-page, the "referrer" column of the log will contain the name of the sub-page. The Getting Fancy page shows you how to set up two installations of the script to track both home page and sub-page hits.
The period.log file will start out small and grow throughout the day as lines are added to the log. You can view the log entries at any time simply by FTPng the file to your local hard-drive (using ASCII mode) and opening the file in a text editor. It is helpful to turn off any "word wrapping" in the editor so that one line in the editor contains one line from the log.
The trigger.dat file contains a single numeric digit which represents the day of the week. This value is read in near the start of the script's execution and compared to the numeric value for the current day. If the numbers are different, the day has changed since the previous hit was received. This signals the script to perform any daily processing that is enabled in the user variables. The value for the new day is then written to this file. That's why you will normally see a date/time stamp on this file that is shortly after midnight (the first hit the page received after midnight). (In other words, this file is read with each page hit, but only written to once a day.)
The Deluxe Edition of the script has four additional data files:
The iptrack.num addresses the "counter bloat" issue outlined above. It keeps a list of the IP addresses of site visitors. You can adjust how far back this list goes (3, 5, 10, whatever hits). When a page is hit the IP address is captured and compared against the list of IP addresses in this file. If a match is found, the counter is not incremented. This helps ensure that you are only counting unique visitors to your site. It is a good rule of thumb to set the list to go back x hits, where x is the number of sub-pages your site has that contain a link to the home page. (This file is written to each a hit consists of a "new" IP address.)
The ipblock file does not actually come with the Deluxe version of the script. You can create it using any text editor and upload it to the server. This file contains a list of IP addresses that you want to block from counting or logging. These may be hits from you or those in your company. You can block an entire range (subnet) of IP addresses simply by entering only the first part (the network number) of the IP address. (This file is only read by the script, never written to. You must manually enter the data into it.)
The history.log file is utilized if the $keep_history variable is set to Yes (1). Whenever report processing occurs, the contents of the period.log file are added to the history.log file before the period.log file is cleared for the new period. Since most hosting services limit your amount of disk space, the size of the history.log file is given in each e-mailed report. This file can grow quite large and could result in your running out of disk space. (This file is written to by the script only at the end of a "period".)
The hithistory.txt file has periodic (daily, weekly, or monthly) entries which are "snapshots" of the count level at those points in time. It also contains the hit counts received during the period, and the percent increase/decrease of the hit counts received during the previous period. This is a tab-delimited text file that can be imported into a spreadsheet or other program for analysis, graphing, reporting, and presentation. The data from this file can also optionally be included in the e-mailed period reports. Utilizing this "hit history" file is covered in detail on the Hit History page. (This file is written to by the script only at the end of a "period".)
The bdlogger.pl file is the script itself. It too is simply an ASCII text file. If you've every had a computer programming class feel free to open the file in a text editor and have a look around. The script is written in Perl and Perl is a very easy language to learn. Now that you know what the script does, once you know what a few special symbols mean (and these are explained in the Messin' with Perl section of the How To Use Your CGI-BIN page), you'll be able to look at the script code to see how it does it. The script is liberally sprinkled with comments (preceeded by the pound sign (#)) to help you figure out what's going on.
What It Does
Bestdam Logger is a tool that allows you to tailor your Web site, and your promotional efforts, to maximize hits from an existing visitor base or judge the success of efforts to enhance or expand the functionality and appeal of your site.
The information contained in the pagehits.cnt, period.log, history.log, and hithistory.txt data (text) files can be FTPed to a local drive (using ASCII mode) and viewed using a text editor. Note that if your FTP program has a "View" function (as WS_FTP does) to view the contents of files that are stored on the server you don't even need to transfer these files to your local hard-drive to view them. With the Deluxe Edition, the information contained in most of these files (all except the history.log file) can also be viewed in the periodic e-mailed reports.
The script presents information about visitors in an easy-to-read columnar format. Presenting the information in this manner allows you to more easily spot patterns or trends. Information such as what domains your visitors are coming from can indicate the success of promotional efforts (not getting may hits from "...aol.com"?). If your site is targeted to graphics artists, it may be helpful to know how many of your visitors are using Macintosh systems. Given the variety of platforms and browsers out there, it may be helpful to optimize the appearance of your pages for the most popular browser.
The most important column in the period.log file is the referrer (last) column. This column will tell you which sites are sending you hits and how frequently. It will allow you to determine if most of your hits are coming from hard links on other sites or search results from search engines. In the case of search engine referrals, the search words the visitor entered to find your site are often contained in the referrer string. This will allow you to target those search engines which are not sending you hits.
The hit count data can also help you identify patterns or trends. Identifying when your site is the most popular (days, evenings, weekends, near the first of the month, etc) can help identify browsing habits of your target visitor segment(s).
It can also be useful to identify which of the pages on your site are the most popular, and the least. The multi-page support of the Bestdam Logger script allows you track the hits to multiple pages. If you have a third-tier or lower sub-page, you can see how many hits from the "parent" page flow down to the subordinate. If only a few hits flow down to this subordinate page you may want to work on the presentation of the material leading to it (and the appearance of the link or link graphic).
Th hit history table will summarize hit count data over the intermediate to long term. Search engines may "crawl" Web sites fairly often looking for dead links, etc. However, crawling only collects information about Web sites and pages. This information must be incorporated into the search engine's index. The rebuilding of indexes may not occur for six months. As a result, promotional efforts in the way of search engine submissions and directory entries may not result in increased site traffic for a considerable period of time. The hit history table will allow you to get a big picture view of your site's performance over these "long" periods of time. It also serves as a data source if you wish to graph your site's performance for reports and presentations.
Bestdam Logger will give you valuable information related to those who are, and are not, visiting your Web site and how they got there. However, like anything else, it is up to you to transform that information into knowledge. Study the logs. Analyze the information. Buried in the details may be opportunities you hadn't previously thought of.