接上文 学习日记4：nginx负载均衡（二）2012.01.08
......
到此，整体工作还差一个备用nginx proxy，其中涉及到keepalived，做心跳。
黄线以上做完了。
本次主要做的是1加一台从nginx_proxy ，暂且命令为front2（为从）。之前的nginx_proxy命令为front（为主）
1 在主nginx上配置keepalived。
cd /usr/local/src/
wget
http:
//
keepalived.org/software/keepalived-1.2.7.tar.gz
tar
-xzvf keepalived-
1.2
.
7
.
tar
.gz
cd keepalived
-
1.2
.
7
/
.
/configure --prefix=/usr/local/keepalived
这时候会报错的！
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!
缺少gcc，安装下面扩展
yum
install
gcc
gcc
-c++ autoconf automake
继续编译，下面还会报错，类似这样的错误。
make
: *** No targets specified and no makefile found. Stop.
configure: error: ! OpenSSL is not properly installed on your system. configure: error: !
对此，我们安装以下扩展即可解决。
yum
-y
install
openssl-devel
yum
-y
install
popt-devel
在编译
./configure --prefix=/usr/local/keepalived
这时候，会出现下面的结果，这个结果就表明是ok的。到此你的keepalived安装是成功的！
Keepalived configuration
------------------------
Keepalived version :
1.2
.
7
Compiler :
gcc
Compiler flags :
-g -
O2
Extra Lib :
-lpopt -lssl -
lcrypto
Use IPVS Framework : Yes
IPVS
sync
daemon support : Yes
IPVS use libnl : No
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
SNMP support : No
Use Debug flags : No
继续下面的操作
make
make
install
cp
/usr/local/keepalived/sbin/keepalived /usr/sbin/
cp
/usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp
/usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir
/etc/
keepalived
cd
/etc/
keepalived
vim keepalived.conf
keepalived.conf配置文件比较重要
front1 ：主nginx的配置文件如下：
! Configuration File
for
keepalived
global_defs {
router_id LVS_01
}
vrrp_instance VI_1 {
state MASTER #master 主nginx
interface eth1
virtual_router_id
51
mcast_src_ip
192.168
.
65.133 #本机ip地址也即主nginx地址
priority
100
advert_int
1
authentication {
auth_type PASS
auth_pass
1111
}
virtual_ipaddress {
192.168
.
65.128
#vip 虚拟出来的ip地址 要求在内网中未被使用
}
}
到此，主nginx配置完成。同样的操作，在从nginx上也配置一遍。
不过从nginx的配置文件如下：
[root@localhost nginx]# vim /etc/keepalived/
keepalived.conf
! Configuration File
for
keepalived
global_defs {
router_id LVS_01
}
vrrp_instance VI_1 {
state BACKUP #slave
interface eth2
virtual_router_id
51
mcast_src_ip
192.168
.
65.137 #slave ip
priority
80
advert_int
1
authentication {
auth_type PASS
auth_pass
1111
}
virtual_ipaddress {
192.168
.
65.128
#vip
}
}
重启两边的keepalived服务：
[root@localhost conf.d]# service keepalived restart
Stopping keepalived: [FAILED]
Starting keepalived: [ OK ]
主nginx上，用
[root@localhost conf.d]# ip a
1
: lo: <LOOPBACK,UP,LOWER_UP> mtu
16436
qdisc noqueue state UNKNOWN
link
/loopback
00
:
00
:
00
:
00
:
00
:
00
brd
00
:
00
:
00
:
00
:
00
:
00
inet
127.0
.
0.1
/
8
scope host lo
inet6 ::
1
/
128
scope host
valid_lft forever preferred_lft forever
2
: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
1500
qdisc pfifo_fast state UP qlen
1000
link
/ether
00
:0c:
29
:
20
:e9:
20
brd ff:ff:ff:ff:ff:ff
inet
192.168
.
65.133
/
24
brd
192.168
.
65.255
scope global eth1
inet
192.168
.
65.128
/
32
scope global eth1
inet6 fe80::20c:29ff:fe20:e920
/
64
scope link
valid_lft forever preferred_lft forever
到此，我们的vip算是真正的绑定上了。
查看keepalived的日志：
root@localhost conf.d]#
tail
-f /var/log/
messages
Jan
9
18
:
53
:
06
localhost Keepalived_healthcheckers[
3355
]: Using LinkWatch kernel netlink reflector...
Jan
9
18
:
53
:
06
localhost Keepalived_vrrp[
3356
]: Configuration is using :
34824
Bytes
Jan
9
18
:
53
:
06
localhost Keepalived_vrrp[
3356
]: Using LinkWatch kernel netlink reflector...
Jan
9
18
:
53
:
06
localhost Keepalived_vrrp[
3356
]: VRRP sockpool: [ifindex(
2
), proto(
112
), fd(
11
,
12
)]
Jan
9
18
:
53
:
06
localhost Keepalived_vrrp[
3356
]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan
9
18
:
53
:
07
localhost Keepalived_vrrp[
3356
]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan
9
18
:
53
:
07
localhost Keepalived_vrrp[
3356
]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan
9
18
:
53
:
07
localhost avahi-daemon[
1765
]: Registering new address record
for
192.168
.
65.128
on eth1.IPv4.
Jan
9
18
:
53
:
07
localhost Keepalived_healthcheckers[
3355
]: Netlink reflector reports IP
192.168
.
65.128
added
Jan
9
18
:
53
:
07
localhost Keepalived_vrrp[
3356
]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1
for
192.168
.
65.128
Jan
9
18
:
53
:
12
localhost Keepalived_vrrp[
3356
]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1
for
192.168
.
65.128
从nginx上，同样
[root@localhost nginx]# ip a
1
: lo: <LOOPBACK,UP,LOWER_UP> mtu
16436
qdisc noqueue state UNKNOWN
link
/loopback
00
:
00
:
00
:
00
:
00
:
00
brd
00
:
00
:
00
:
00
:
00
:
00
inet
127.0
.
0.1
/
8
scope host lo
inet6 ::
1
/
128
scope host
valid_lft forever preferred_lft forever
2
: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
1500
qdisc pfifo_fast state UP qlen
1000
link
/ether
00
:0c:
29
:a7:
81
:c7 brd ff:ff:ff:ff:ff:ff
inet
192.168
.
65.137
/
24
brd
192.168
.
65.255
scope global eth2
inet6 fe80::20c:29ff:fea7:81c7
/
64
scope link
valid_lft forever preferred_lft forever
我们发现并没有绑定vip 128
同时；
[root@localhost nginx]#
tail
-f /var/log/
messages
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Registering Kernel netlink reflector
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Registering Kernel netlink command channel
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: Opening
file
'
/etc/keepalived/keepalived.conf
'
.
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Opening
file
'
/etc/keepalived/keepalived.conf
'
.
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: Configuration is using :
34822
Bytes
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Configuration is using :
4821
Bytes
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: Using LinkWatch kernel netlink reflector...
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: VRRP sockpool: [ifindex(
2
), proto(
112
), fd(
11
,
12
)]
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Using LinkWatch kernel netlink reflector...
貌似从nginx似乎出了什么问题，先不着急。
这时候我们停止掉主nginx的服务，再看看效果怎么样！
killall
nginx
再观察从nginx的状态变化
[root@localhost nginx]#
tail
-f /var/log/
messages
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Registering Kernel netlink reflector
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Registering Kernel netlink command channel
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: Opening
file
'
/etc/keepalived/keepalived.conf
'
.
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Opening
file
'
/etc/keepalived/keepalived.conf
'
.
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: Configuration is using :
34822
Bytes
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Configuration is using :
4821
Bytes
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: Using LinkWatch kernel netlink reflector...
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan
9
19
:
16
:
25
localhost Keepalived_vrrp[
22421
]: VRRP sockpool: [ifindex(
2
), proto(
112
), fd(
11
,
12
)]
Jan
9
19
:
16
:
25
localhost Keepalived_healthcheckers[
22420
]: Using LinkWatch kernel netlink reflector...
Jan
9
19
:
17
:
33
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan
9
19
:
17
:
34
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan
9
19
:
17
:
34
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan
9
19
:
17
:
34
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth2
for
192.168
.
65.128
Jan
9
19
:
17
:
34
localhost avahi-daemon[
1810
]: Registering new address record
for
192.168
.
65.128
on eth2.IPv4.
Jan
9
19
:
17
:
34
localhost Keepalived_healthcheckers[
22420
]: Netlink reflector reports IP
192.168
.
65.128
added
Jan
9
19
:
17
:
39
localhost Keepalived_vrrp[
22421
]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth2
for
192.168
.
65.128
从第33秒后发现，备用nginx已经转为master状态。
再看
[root@localhost nginx]# ip a
1
: lo: <LOOPBACK,UP,LOWER_UP> mtu
16436
qdisc noqueue state UNKNOWN
link
/loopback
00
:
00
:
00
:
00
:
00
:
00
brd
00
:
00
:
00
:
00
:
00
:
00
inet
127.0
.
0.1
/
8
scope host lo
inet6 ::
1
/
128
scope host
valid_lft forever preferred_lft forever
2
: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
1500
qdisc pfifo_fast state UP qlen
1000
link
/ether
00
:0c:
29
:a7:
81
:c7 brd ff:ff:ff:ff:ff:ff
inet
192.168
.
65.137
/
24
brd
192.168
.
65.255
scope global eth2
inet
192.168
.
65.128
/
32
scope global eth2
inet6 fe80::20c:29ff:fea7:81c7
/
64
scope link
valid_lft forever preferred_lft forever
发现没有？vip飘过来了！
针对Keepalived的不足,用check_nginx.sh来监控nginx进程，实现真正意义上的负载均衡高可用.
此脚本思路其实也很简单，即放置在后台一直监控nginx进程；如进程消失，尝试重启nginx，
如是失败则立即停掉本机的keepalived服务，让另一台负载均衡器接手.
vi
/shell/nginx_pid.
sh
###################################################
#
!/bin/
bash
while
:
do
nginxpid
=`
ps
-C nginx --no-header |
wc
-
l`
if
[ $nginxpid -eq
0
]
then
/usr/local/nginx-
0.8
.
53
/sbin/
nginx
sleep
5
if
[ $nginxpid -eq
0
]
then
/etc/init.d/
keepalived stop
fi
fi
sleep
5
done
###################################################
我们仍然可以继续访问我们的站点。至此。故障点切换宣告结束。
【原创】学习日记5：nginx负载均衡（三）2012.01.10 nginx+keepalievd，实现负载均衡和故障点切换。keepalived双机热备。
