For environments where the use of a load balancer is not feasible because of say cost or complexity I often use Corosync to provide similar functionality. Corosync is a piece of software that allows for clustering of an application to provide high availability. One issue I have often experienced with the tool is that the error messages are not descriptive making troubleshooting difficult.
As an example, I have used Corosync to cluster syslog servers in the past. In one such environment I had a pair of syslog servers in an active-standby configuration with a VIP. While the VIP came up as expected the syslog server reported an unknown error as shown below.
test@log01:/home/test$ sudo crm status ============ Last updated: Wed Jan 23 00:00:47 2013 Last change: Tue Jan 22 23:49:38 2013 via cibadmin on log01 Stack: openais Current DC: log01 - partition WITHOUT quorum Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c 2 Nodes configured, 2 expected votes 4 Resources configured. ============ Online: [ log01 ] OFFLINE: [ log02 ] Resource Group: log_svr vip (ocf::heartbeat:IPaddr2): Started log01 Failed actions: log_svc:0_start_0 (node=log01, call=6, rc=1, status=complete): unknown error
So what was causing the error and how can you clear it up?
First, let’s check the syslog configuration file for syntax errors:
test@log01:/home/test$ sudo syslog-ng -s test@log01:/home/test$
Good, none found. Next, let’s ensure syslog is running:
test@log01:/home/test$ sudo /etc/init.d/syslog-ng status * syslog-ng is not running
Hmm, it is not. Let’s start it:
test@log01:/home/test$ sudo /etc/init.d/syslog-ng start * Starting system logging syslog-ng Error resolving hostname; host='logrelay' Error initializing dest driver; dest='d_loghost', id='d_loghost#0' Error initializing message pipeline;
Well there you go, logrelay cannot be resolved and as such syslog cannot start.
It would be nice if Corosync could relay messages on behalf of the applications it supports to ease administration of the system…
© 2013, Steve Flanders. All rights reserved.
