[syslog-ng]making syslog_NG redundant / NFS / HA possibilities

[Dave Johnson]

Oliver--

Thats an interesting setup, a couple comments:
* I'd recommend running TCP syslog (-ng) wherever you can...
* If you setup an active/active syslog-ng server in your loadbalancer,
my first reaction is that its probably not safe to write to the same
file constantly through nfs...

Another case scenario, which maybe large enough to consider--=20
What happens if there is network failure to the netapp from the
syslog-ng servers.  (ethernet cable on one of the servers gets yanked
; or ethernet card goes bad on one of the servers, etc...)
 - Are you going to configure your syslog-ng servers to sync(0) or
maybe buffer some data in the event that this happens?
 - And when the server joins will it dump a large amount of logs to disk?

--------

So with all this said, there are a couple other overall design
approaches, which you may consider:

1) Setup a passive/active configuration in the load balancer=20
    (Use the load balancer to failover to a secondary logging host)
    - You may need to run a script on the load balancer and/or
system(s) to check for nfs
      availablility.

2) Just have one central syslog-ng host, and run syslog-ng on the all
your systems creating a large enough buffer so if the central server
becomes unavailable, buffer.  All those remote machines "should" log
to local disk anyways.  Your network equipment may need to still have
a "primary" and "secondary" syslog host for the central server
failing.

-------

And finally the most straightforward approach--

3) If absolute message integrity is required, setup an active / active
(raid 1) approach--
    Send messages to two servers at the same time logging to two
different files, and make
    sure that no infrastructure on each machine is shared.

On Mon, 31 Jan 2005 13:13:57 +0100, Oliver T=F6nsmann
<oliver.toensmann@uni-bielefeld.de> wrote:
> Hi,
>=20
> we like to setup a central logging service for our distributed
> datacenter environment. Our idea is to use syslog_ng as an HA (high
> available) service running on two separate servers.  The syslog clients
> (Solaris machines, network components, firewall appliances) should post
> the log data via UDP to a loadbalancer which will forward the log data
> to the two syslog_ng hosts.
>=20
> Now we have the problem how to merge the log data from the two syslog_ng
> hosts to one location. Most of our data is stored on centralised storage
> subsystems and this storage is distributed via NFS (NetApp file server).
> We like to store the logdata also into nfs directories.  Following
> figure describes a possible dataflow:
>=20
> syslog clients --> LB -->
>=20
> --> syslog_ng host1
> --> syslog_ng host2
>=20
> -->
> NFS:/.../log/$HOST/$FACILITY/$PRIORITY/one_logfile_for_both_syslog_hosts
>=20
> Performance and nfs overhead is not a problem in our environment and nfs
> works very reliable due to our HA structures;  logdata per day ~2-3 GB.
> Is there a possibilty to let syslog_ng hosts write the logdata
> simultaenously into the same logfile (NFS)? What are the disadvantages?
>=20
> What are your experiences in using syslog_ng as an HA service? What
> would be a good practice to setup syslog_ng and to make it high available=
?
>=20
> Ideas and helpful suggestions are welcome - Thanxx.
> Oliver
>=20
> ------------------------------------------------
> Oliver Toensmann
> Universitaet Bielefeld - Hochschulrechenzentrum
> Email:  Oliver.Toensmann@uni-bielefeld.de
>=20
> _______________________________________________
> syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
>=20
>