awstats_updateall launches update process for all AWStats config files (except
awstats.model.conf) found in a particular directory, so you can easily setup a
cron/scheduler job. The scanned directory is by default /etc/awstats.
Usage: awstats_updateall.pl now [options]
Where options are:
-awstatsprog=pathtoawstatspl
-configdir=confdirtoscan
where awstats_options are any option known by AWStats
-config=configvalue is value for -config parameter (REQUIRED)
-update option used to update statistics before to generate pages
-lang=LL to output a HTML report in language LL (en,de,es,fr,...)
-month=MM to output a HTML report for an old month=MM
-year=YYYY to output a HTML report for an old year=YYYY
and awstatsbuildstaticpages_options can be
-awstatsprog=pathtoawstatspl gives AWStats software (awstats.pl) path
-dir=outputdir to set output directory for generated pages
-builddate=%YY%MM%DD Used to add build date in built pages filenames
-staticlinksext=xxx For pages with .xxx extension instead of .html
-buildpdf[=pathtohtmldoc] Build a PDF file after building HTML pages.
Output directory must contains icon directory
when this option is used (need 'htmldoc').
logresolvemerge allows you to get one unique output log file, sorted on date, built from particular sources: - It can read several input log files, - It can read .gz/.bz2 log files, - It can also makes a fast reverse DNS lookup to replace all IP addresses into host names in resulting log file. logresolvemerge comes with ABSOLUTELY NO WARRANTY. It's a free software distributed with a GNU General Public License (See COPYING.txt file). logresolvemerge is part of AWStats but can be used alone as a log merger or resolver before using any other log analyzer.
Usage: logresolvemerge.pl [options] file logresolvemerge.pl [options] file1 ... filen logresolvemerge.pl [options] *.* perl logresolvemerge.pl [options] *.* > newfile Options: -dnslookup make a reverse DNS lookup on IP adresses -dnslookup=n same with a n parallel threads instead of serial requests -dnscache=file make DNS lookup from cache file first before network lookup -showsteps print on stderr benchmark information every 8192 lines -addfilenum if used with several files, file number can be added in first -addfilename if used with several files, file name can be added in first
field of output file. This can be used to add a cluster id
when log files come from several load balanced computers. -stoponfirsteof Stop processing when any logfile reaches end-of-file. -printfields For IIS or W3C logs, prints the latest field header for
the currentlog file when switching between log file entries
so that the parsercan automatically determine which fields are avaiable. -ignoremissing will not fail if a log file is missing
This runs logresolvemerge in command line to open one or several server log files to merge them (sorted on date) and/or to make a reverse DNS lookup (if asked). The result log file is sent on standard output. Note: logresolvemerge is not a 'sort' tool to sort one file. It's a software able to output sorted log records (with a reverse DNS lookup included or not) even if log records are dispatched in several files. Each of thoose files must be already independently sorted itself (but that is the case in all web server log files). So you can use it for load balanced log files or to group several old log files.
Don't forget that the main goal of logresolvemerge is to send log records to a log analyzer in a sorted order without merging files on disk (NO NEED OF DISK SPACE AT ALL) and without loading files into memory (NO NEED OF MORE MEMORY). Choose of output records is done on the fly.
So logresolvemerge is particularly usefull when you want to output several and/or large log files in a fast process, with no use of disk or more memory, and in a chronological order through a pipe (to be used by a log analyzer).
Note: If input records are not 'exactly' sorted but 'nearly' sorted (this occurs with heavy servers), this is not a problem, the output will also be 'nearly' sorted but a few log analyzers (like AWStats) knowns how to deal with such logs.
WARNING: If log files are old MAC text files (lines ended with CR char), you can't run this tool on Win or Unix platforms.
WARNING: Because of memory holes in ActiveState Perl version, use another Perl interpreter if you need to process large log files.
Now supports/detects: Automatic detection of log format Files can be .gz/.bz2 files if zcat/bzcat tools are available in PATH. Multithreaded reverse DNS lookup (several parallel requests) with Perl 5.8+. New versions and FAQ at http://www.awstats.org
maillogconvert is mail log preprocessor that convert a mail log file (from
postfix, sendmail or qmail servers) into a human readable format.
The output format is also ready to be used by a log analyzer, like AWStats.
The first parameter specifies what format the mail logfile is :
standard - logfile is standard postfix,sendmail,qmail or mdaemon log format
vadmin - logfile is qmail log format with vadmin multi-host support
The second parameter specifies what year to timestamp logfile with, if current
year is not the correct one (ie. 2002). Always use 4 digits. If not specified,
current year is used.
If no output is specified, it goes to the console (stdout).
For example, the following sample from postfix mail log server:
# 1 Mail [email protected] -> [email protected], forward [email protected] -> [email protected] Jan 01 07:27:31 apollon postfix/smtpd[1684]: connect from remt30.cluster1.charter.net[209.225.8.40]
Jan 01 07:27:32 apollon postfix/smtpd[1684]: 2BC793B8A4: client=remt30.cluster1.charter.net[209.225.8.40]
Jan 01 07:27:32 apollon postfix/cleanup[1687]: 2BC793B8A4: message-id=<36027278@vneka>
Jan 01 07:27:32 apollon postfix/qmgr[13860]: 2BC793B8A4: from=, size=2130, nrcpt=1 (queue active)
Jan 01 07:27:32 apollon postfix/smtpd[1684]: disconnect from remt30.cluster1.charter.net[209.225.8.40]
Jan 01 07:27:38 apollon postfix/local[1689]: 2BC793B8A4: to=, orig_to=, relay=local, delay=6, status=sent ("|/usr/bin/procmail")
# 2 Reject: 450
Jan 01 14:05:44 apollon postfix/smtpd[2114]: connect from baby.mainframe.nl[81.29.4.2]
Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: client=baby.mainframe.nl[81.29.4.2]
Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: reject: RCPT from baby.mainframe.nl[81.29.4.2]: 450 : User unknown in local recipient table; from=<> to= proto=ESMTP helo= Jan 01 14:10:16 juni postfix/smtpd[2568]: C34ED1432B: reject: RCPT from relay2.tp2rc.edu.tw[163.28.32.177]: 450 : User unknown in local recipient table; from=<> proto=ESMTP helo= # 1 From unknown
Jan 01 15:17:05 apollon postfix/smtpd[29866]: connect from tomts12.bellnexxia.net[209.226.175.56]
Jan 01 15:17:05 apollon postfix/smtpd[29866]: 578093B8B5: client=tomts12.bellnexxia.net[209.226.175.56]
Jan 01 15:17:05 apollon postfix/cleanup[28931]: 578093B8B5:
message-id=<20030905131913.EOVH11393.tomts12-srv.bellnexxia.net@tomts12-srv>
Jan 01 15:17:06 apollon postfix/qmgr[965]: 578093B8B5: from=<>, size=109367, nrcpt=1 (queue active)
Jan 01 15:17:06 apollon postfix/local[32432]: 578093B8B5: to=, orig_to=, relay=local, delay=1, status=sent ("|/usr/bin/procmail")
Jan 01 15:17:06 apollon postfix/smtpd[29866]: disconnect from tomts12.bellnexxia.net[209.226.175.56]
Urlaliasbuilder generates an 'urlalias' file from an input file (an urlalias file
is a file with two columns: url and clear title of url).
The input file must contain a list of URLs (It can be an AWStats history file).
For each of thoose URLs, the script get the corresponding HTML page and catch the
header information (title), then it writes an output file that contains one line
for each URLs and several fields:
- The first field is the URL,
- The second is title caught from web page.
This resulting file can be used by AWStats urlalias plugin.
The site parameter contains the web server to get the page from.
Where options are:
-urllistfile=Input urllist file
If this file is an AWStats history file then urlaliasbuilder will use the
SIDER section of this file as its input URL's list.
-urlaliasfile=Output urlalias file to build
-overwrite Overwrite output file if exists (by default appends to file).
-secure Use https protocol
