Apache & Loging Web Farm

This is a discussion about Apache & Loging Web Farm in the Everything Linux category; Apache & Loging Web Farms I have multiple load-balanced servers and I want to generate one web report for the whole farm, but each server generates its own log file Webalizer will not work the log files are 500Gb's each I get about 160 hits a sec Is there some other software out there what will work with very big logs?

Everything Linux 1800 This topic was started by , . Last reply by ,


data/avatar/default/avatar22.webp

3 Posts
Location -
Joined 2004-01-01
Apache & Loging Web Farms
 
I have multiple load-balanced servers and I want to generate one web report for the whole farm, but each server generates its own log file
 
Webalizer will not work the log files are 500Gb's each I get about 160 hits a sec
 
Is there some other software out there what will work with very big logs?

Participate in our website and join the conversation

You already have an account on our website? To log in, use the link provided below.
Login
Create a new user account. Registration is free and takes only a few seconds.
Register
This subject has been archived. New comments and votes cannot be submitted.
Aug 9
Created
Aug 10
Last Response
0
Likes
1 minute
Read Time
User User
Users

Responses to this topic


data/avatar/default/avatar39.webp

336 Posts
Location -
Joined 2004-07-09
first off, you should read this document written by an admin of a site getting
about 45 million hits a day.
 
next, read this document written by an admin who has set up a multi-system-one-logfile
configuration. Note the use of a pipe as the log destination.
 
The general Apache documentation section on logs will round out your knowledge.
 
Basically, you'll need to have Apache pipe all of the logging to the syslog daemon.
 
With the logging all going to syslog, you set each system in the farm to send logging
to the one log host you designate, whose own syslog receives the log data via a LAN.