AWStats logfile analyzer 7.4 Documentation

 


Plugins, contribs and related programs

You will find here description for different kinds of ressource files :
- AWStats plugins
- Other AWStats contribs/tools/package
- Other Non AWStats contribs/tools (not related to AWStats but related to log analysis)
- Documents on AWStats or related to log analysis



AWStats plugins

Plugins are .pm files you can put in your AWStats plugins directory to add new features.
Note that you must enable the plugin by adding a new line in your config files to make it works (See LoadPlugin parameter).

Standard free plugins : tooltips, decodeutfkeys, ipv6, hashfiles,  userinfo, hostinfo, clusterinfo, urlalias, timehires, timezone, rawlog, graphapplet, graphgooglechartsapi geoipfree
GeoIP Plugins (may require other licensed product to work) : geoip, geoip_city_maxmind, geoip_asn_maxmind, geoip_region_maxmind, geoip_isp_maxmind, geoip_org_maxmind,


List of standard plugins (Provided with AWStats) :

Tooltips
Required Modules: None
Parameters: None
Description: Add tooltips pop-up help boxes to HTML report pages.  
Provided with AWStats (5.4+).

DecodeUTFKeys
Required Modules: Encode and URI::Escape
Parameters: None
Description: Allow AWStats to show correctly (in language charset) keywords/keyphrases strings even if they were UTF8 coded by the referer search engine.
Plugin provided with AWStats (6.0+).

IPv6
Required Modules: Net::IP and Net::DNS
Parameters: None
Description: This plugin gives AWStats capability to make reverse DNS lookup on IPv6 addresses.
Note: IPv6 addresses are currently incompatible with the GeoIP databases
Plugin provided with AWStats (5.5+)

HashFiles
Required Modules: Storable
Parameters: None
Description: AWStats DNS cache files are read/saved as native hash files. This improves cache file loading speed, especially for very large web sites.
Plugin provided with AWStats (5.1+)

UserInfo
Required Modules: None
Parameters: None
Description:  Add text (Firtname, Lastname, Office Department, ...) to authenticated user reports for each login value. A text file called userinfo.myconfig.txt, with two fields (first is login, second is text to show, separated by a tab char) must be created in DirData directory.
Plugin provided with AWStats (5.5+)

ClusterInfo
Required Modules: None
Parameters: None
Description: Add text (for example a full hostname) to cluster reports for each cluster number. A text file called clusterinfo.myconfig.txt, with two fields (first is cluster number, second is text to show) separated by a tab character must be created in DirData directory.
Note: This plugin is useless if ShowClusterStats is set to 0 or if you don't use a personalized log format that contains %cluster tag.
Plugin provided with AWStats (6.2+)

HostInfo
Required Modules: Net::XWhois
Parameters: None
Description: Add a column into host chart with a link to open a popup window that shows info on host (like whois records).
Plugin provided with AWStats (6.0+)

UrlAliases
Required Modules: None
Parameters: None
Description: Add text (Page title, description...) to URL reports before URL value. A text file called urlalias.myconfig.txt, with two fields (first is URL, second is text to show, separated by a tab char) must be created in DirData directory.
You can build your urlalias file manually or use urlaliasbuilder.pl tool (provided with 5.4+)
Plugin provided with AWStats (5.2+)

TimeHiRes
Required Modules: Time::HiRes
Parameters: None
Description: Time reported by -showsteps option is in millisecond. For debug purpose.
Plugin provided with AWStats (5.1+)

TimeZone
Required Modules: Time::Local
Parameters: <offset or local timezone>
Description: Converts the timezone of log files before processing using the offset provided in the parameters. This is useful for some IIS users if logs are stored in GMT or for users who have servers in a different timezone. You must provide either the abreviated name of the timezone you wish to convert to or the offset value in hours. For example
LoadPlugin="timezone +2"  would add two hours to the log timestamps
LoadPlugin="timezone CET"  would convert the log from GMT to CET
Note: This plugin reduces AWStats speed by about 40%!
Plugin provided with AWStats (5.1+)

Rawlog
Required Modules: None
Parameters: None
Description: This plugin adds a form in AWStats main page to allow users to see raw content of current log files. A display filter will also be displayed.
Plugin provided with AWStats (5.7+)

GraphApplet
Required Modules: None
Parameters: None
Description: Supported charts are built by a 3D graphic applet.
LoadPlugin="graphapplet" # EXPERIMENTAL FEATURE
Plugin provided with AWStats (6.0+)

GraphGoogleChartAPI
Required Modules: None
Parameters: None
Description: Replaces the standard charts with free Google API generated images in HTML reports. If country data is available and more than one country has hits, a map will be generated using Google Visualizations.
Note: The machine where reports are displayed must have Internet access for the charts to be generated. The only data sent to Google includes the statistic numbers, legend names and country names.
Plugin provided with AWStats (6.0+)

GeoIPfree
Required Modules: Geo::IPfree version 0.2+ (from Graciliano M.P.)
Parameters: None
Description: Country chart is built from an Internet IP-Country database. This plugin is useless for intranet only log files. 
Note: You must choose between using this plugin (need Perl Geo::IPfree module, database is free but not up to date) or the GeoIP plugin (need Perl Geo::IP module from Maxmind, database is also free and up to date).
Note: Activestate provide a corrupted version of Geo::IPfree 0.2 Perl module, so install it from elsewhere (from www.cpan.org for example).
This plugin reduces AWStats speed by about 10% !
Plugin provided with AWStats (5.5+).
You can find Geo::IPfree Perl Module on :
- All OS: CPAN site
- RPM compatible OS (RedHat, Mandrake...): Lafraia rpm
- ActiveState: GeoIPfree module provided by Activestate is not working correctly so I removed the link. You can follow the following setup instead: Download from CPAN site and copy 'Geo' directory (found in 'lib') into AWStats plugins directory. This is a fast tip to install Geo-IPfree perl module to be used by AWStats geoipfree plugin.
Note: The Geo::IPfree database has not been updated in a long time, so using the geoip plugin (using the Geo::IP Perl module, now free for countries) is highly recommended.
You can use the following tool to test your geoipfree setup: testgeoipfree.pl. Just put this file into your plugins directory and run it.

 
Maxmind GeoIP Plugin Notes
Maxmind is a company that sells geolocation technology that matches IP addresses to countries, regions, cities and ISPs. All of their databses are available for an initial fee and a monthly update subscription and include greater accuracy and a higher frequency of updating. Some databases are available for free but they are not updated as often and may not be as accurate.

Install:
First choose a database. When downloading a database for AWStats, please download the Binary Format version. Current databases AWStats supports are:

Commercial
: Country, Region, City, Organization, ISP
Open Source Country, City, ASN

If you choose to purchase a commercial database for country information, you only need to purchase the one with the greatest level of detail you want to appear in your report. For example, if you only want country information, you can purchase just the Country database. If you want City detail, you do not need to purcahse the Country and Region databases, just the City database. (Organization and ISP databases do not include country or region info so you must purchase those in addition to Country, Region or City if you desire more information)

Next, install the Geo::IP Perl module. Then choose a Maxmind API to access the databse. Maxmind also has two Perl API's available for interacting with their databases. There is a a C/Perl library that will run  quickly and there is a pure Perl API that is a little slower but does not require any C depdencies. Choose one of the APIs and install it on your machine.

Parameters: Each GeoIP database can be loaded entirely into memory or lookups can scan through the file. When editing your AWStats configuration, you have to specify if you want to use memory or file lookups. Specify GEOIP_STANDARD for file lookups or GEOIP_MEMORY_CACHE for memory loading. Please note that the memory method may crash on some Linux distributions so if that happens, try the standard method. Also, using the C API with Activestate Perl may crash on Windows machines, so use the PurePerl module instead.

You may also override some databases by generating a comma seperated text file with IP addresses and associated data. This is useful if you are reporting on an Intranet site, have a mixture of internal and external users or the data for a specific IP is incorrect in the database. You can use the included geoip_generator tool to create override files. (Overrides only available in AWStats version 6.96 and later)
Override Formats:
GeoIP - ip_address,"countrycode"
GeoIP City - ip_address,"countrycode","regioncode","cityname","postalcode",latitude,longitude,"metrocode","areacode"
GeoIP ASN - ip_address,"asn ispname"
GeoIP Region - ip_address,"countrycode","regioncode"
GeoIP Org - ip_address,"organisation"
GeoIP ISP - ip_address,"ISP"

Note: AWStats does not currently support IPv6 for GeoIP lookups as Maxmind only offers the country DB with IPv6 at this time. As Maxmind includes IPv6 support in more DBs, we will update AWStats.
Enabling GeoIP plugins will reduce the speed of AWStats processing.

GeoIP
Required Modules: Geo::IP or Geo::IP::PurePerl (from Maxmind)
Parameters: [GEOIP_STANDARD | GEOIP_MEMORY_CACHE] [/pathto/geoip.dat] </pathto/override.txt>
Description: Country chart is built from an Internet IP-Country database.
Note: You must choose between using this plugin or the GeoIPfree plugin (need Perl Geo::IPfree module, database is free but not up to date).
Example LoadPlugin="geoip GEOIP_STANDARD /usr/local/geoip.dat /usr/local/geoip_override.txt"
Plugin provided with AWStats (5.2+)

GeoIP_City_Maxmind
Required Modules: Geo::IP or Geo::IP::PurePerl (from Maxmind)
Parameters: [GEOIP_STANDARD | GEOIP_MEMORY_CACHE] [/pathto/GeoIPCity.dat] </pathto/override.txt>
Description: This plugin adds a chart of hits by cities (with country and regions informations for major countries).
By enabling this plugin, you will see a new link called 'Cities' on reports menu.
Example LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /usr/local/geoip.dat /usr/local/GeoIPCity_override.txt"
Plugin provided with AWStats (6.2+)

GeoIP_ASN_Maxmind
Required Modules: Geo::IP or Geo::IP::PurePerl (from Maxmind)
Parameters: [GEOIP_STANDARD | GEOIP_MEMORY_CACHE] [/pathto/GeoIPASN.dat] </pathto/override.txt> <link to AS lookup>
Description: This plugin adds a chart of AS numbers where the host IP address is registered. This plugin can display some ISP information if included in the database. You can also provide a link that will be used to lookup additional registration data. Put the link at the end of the parameter string and the report page will include the link with the full AS number at the end.
Example LoadPlugin="geoip_asn_maxmind GEOIP_STANDARD /usr/local/geoip.dat /usr/local/GeoIPASN_override.txt http://enc.com.au/itools/aut-num.php?autnum="
Plugin provided with AWStats (6.9+)

GeoIP_Region_Maxmind (Commercial Only)
Required Modules: Geo::IP or Geo::IP::PurePerl (from Maxmind)
Parameters: [GEOIP_STANDARD | GEOIP_MEMORY_CACHE] [/pathto/GeoIPRegion.dat] </pathto/override.txt>
Description: This plugin adds a chart of hits by regions. Only regions for US and Canada can be detected at this time.
By enabling this plugin, you will see a new link called 'Regions' on reports menu. For the moment this link works only when AWStats reports are build as CGI (static built not yet supported).
Example LoadPlugin="geoip_region_maxmind GEOIP_STANDARD /usr/local/GeoIPRegion.dat /usr/local/geoip_override.txt"
Plugin provided with AWStats (6.2+)

GeoIP_ISP_Maxmind (Commercial Only)
Required Modules: Geo::IP or Geo::IP::PurePerl (from Maxmind)
Parameters: [GEOIP_STANDARD | GEOIP_MEMORY_CACHE] [/pathto/GeoIPISP.dat] </pathto/override.txt>
Description: This plugin adds a chart of hits by ISP.
By enabling this plugin, you will see a new link called 'ISP' on reports menu. For the moment this link works only when AWStats reports are build as CGI (static built not yet supported).
Example LoadPlugin="geoip_isp_maxmind GEOIP_STANDARD /usr/local/GeoIPISP.dat /usr/local/geoip_override.txt"
Plugin provided with AWStats (6.3+)

GeoIP_Org_Maxmind (Commercial Only)
Required Modules: Geo::IP or Geo::IP::PurePerl (from Maxmind)
Parameters: [GEOIP_STANDARD | GEOIP_MEMORY_CACHE] [/pathto/GeoIPOrg.dat] </pathto/override.txt>
Description: This plugin adds a chart of hits by Organization names.
By enabling this plugin, you will see a new link called 'Organizations' on reports menu. For the moment this link works only when AWStats reports are build as CGI (static built not yet supported).
Example LoadPlugin="geoip_org_maxmind GEOIP_STANDARD /usr/local/GeoIPOrg.dat /usr/local/geoip_override.txt"
Plugin provided with AWStats (6.3+)


Other AWStats contribs/tools/package


All following files/products were developped and submitted by contributors. I haven't tried them, so I can't tell you if they work and how they work...
They can enhance AWStats use or save you time giving you a start for a new development.

Other tools:

Replacement Frontend:

BetterAWStats is a PHP Frontend to show you AWStats data file differently.
AWStats PhpNuke module is a module for the open source CMS PHP Nuke.
Ravencore is an Open Source Hosting Control Panel that include AWStats as part of its statistics solutions.
summary4severaldomains.pl is a small perl script that build an index.html page with AWStats txt summary information for each domains on same page (number of pages, hits and bandwith for month).
AWStats Totals is a php page that use AWStats database to show a summary for several config files, on the same page.

Miscellanous:

A plugin for WordPress that allow WordPress users to add tags required by AWStats to report the "miscellanous chart".
Exim2Awstats is a tool to convert Exim MTA log file to a format readable by AWSTats.
Logprocess automate the collection, merging, processing and archiving of IIS log files in a hosting (mult-domain) environment using AWStats to generate and manage stats/reports.
XlencArlm is a tool to enhance use of AWStats with IIS (Catching log files from FTP, doing GMT correction on log files, ...). Don't know if it works correctly but seems to be interesting.
Configurador is a piece of CGI perl scripts to allow you to edit an AWStats config file from the web. Code comments are in Spanish but it may be a goo start to develop your own tool. Hector Garcia Alvarez
logtrans.py is a python small example on how to convert an old log file from IIS4.0 to a new format compatible with AWStats. Don't know if it works, just here to provide you a code sample if you like python.
isa2w3c.pl, is a convertor to have a more standard log file from an ISA log file.
go_awstats, is a tool to automate build of AWStats statics reports for all config files but also for all month.
Stats Letter, is a Perl tool to send AWStats reports to one or several user for one or several reports. Documentation in French only.


Other Non AWStats contribs/tools (not related to AWStats but related to log analysis)

maillog2commonlog.pl converts mail logs in smail or qmail format to the common log format. Note that if you want to analyze, with AWStats, mail log files from postfix, sendmail or qmail, a better solution is to use instead maillogconvert.pl preprocessor (See AWStats F.A.Q about analyzing mail logs).
Joey Hess, freeware.
maillogconv.pl is another mail log processor that can be used to change mail log files into a log file that AWStats can understand.
Note that a derivated and enhanced tool called maillogconvert.pl is provided with AWStats since 5.7. See F.A.Q about analyzing mail logs to know how to setup AWStats with maillogconv.pl or maillogconvert.pl to analyze Postfix, Sendmail, Qmail or MDaemon log files.
mlc is another postfix mail logs converter (written in C). Try it if the maillogconvert.pl tool provided with AWStats does not work on your postfix log file.
access_referer_agent2combined.pl is a perl script that converts a directory of common log format (CLF) access-files, referer-files and agent-files to a single combined.log This is usefull if you have seperate access/referer/agent logfiles and want to use all features of a loganalyzer that require a combined log file.
anonlog "anonymizes" your logfile by encoding sensitive information.
Stephen Turner, freeware.

If you don't like AWStats, you can try thoose other popular similar tools :
Analog. A very old log analyzer (written in C).
Stephen Turner, free software.
Webalizer. Another very old log analyzer (written in C).

There is also commercial products you can find doing a search on 'log file analyzer' with Google.



Documents on AWStats or related to log analysis

AWStats official documentation for last version can be found here: HTML On line or PDF file

Additional resources for AWStats provided by Antezeta (or in Italian)
An intro and install doc 'Using AWStats on GNU/Linux or Windows' in Chinese
A setup tutorial in Danish
A setup tutorial in French from Funix: How to install AWStats and Webalizer
A setup tutorial in French for Windows users
A setup tutorial in French from Fpoeserv
A setup tutorial for debian users with a logrotate tool in German
Another way to explain AWStats setup



Article written by .


Follow @awstats_project