Redis集群主備緩存區滿(mǎn)了導致主備頻繁倒換
問(wèn)題現象
Redis 頻繁進(jìn)行主備倒換,通過(guò)查看主實(shí)例的日志:redis.log發(fā)現下面報錯:
Client id=1317049445 addr=192.168.2.45:8004 fd=40 name= age=314 idle=0 flags=S db=0 sub=0
psub=0 multi=-1 qbuf=0 qbuf-free=32568 obl=0 oll=4430 omem=761143439 events=rw cmd=psync
scheduled to be closed ASAP for overcoming of output buffer limits
其中:psync scheduled to be closed ASAP for overcoming of output buffer limits 明顯就是問(wèn)題所在,那是什么問(wèn)題呢。
解決思路
于是我在源碼中搜索了scheduled to be closed ASAP for overcoming of output buffer limits(psync明顯是一個(gè)命令,就不用在代碼里面搜索了)。
于是,我找到了下面代碼:
/* If the source client contains a partial response due to client output
* buffer limits, propagate that to the dest rather than copy a partial
* reply. We don't wanna run the risk of copying partial response in case
* for some reason the output limits don't reach the same decision (maybe
* they changed) */
if (src->flags & CLIENT_CLOSE_ASAP) {
sds client = catClientInfoString(sdsempty(),dst);
freeClientAsync(dst);
serverLog(LL_WARNING,"Client %s scheduled to be \
closed ASAP for overcoming of output buffer limits.", client);
sdsfree(client);
return;
}
單從代碼來(lái)看,看不出啥,但是代碼上面存在注釋?zhuān)沂褂梦夷撬募壦椒g了下,可以看出緩存區滿(mǎn)了,于是可以想到可能主備同步的時(shí)候可能會(huì )限制緩存區大小,并且這個(gè)緩存區被占滿(mǎn)了。
于是我又在redis.conf中找了緩沖區的相關(guān)配置,找到了下面是三個(gè):
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
配置格式為:
client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
具體含義:
-
class:我理解就是緩沖區的類(lèi)型,目前Redis的緩沖區分為三個(gè):
- 普通客戶(hù)端:
normal - 主備同步:
replica - 發(fā)布訂閱:
pubsub
- 普通客戶(hù)端:
- hard limit: 緩沖區大小的硬性限制。當達到這個(gè)限制之后,連接就會(huì )斷開(kāi)。
- soft limit: 緩沖去大小的軟性限制。
- soft seconds: 緩沖區大小達到了(超過(guò))soft limit值的持續時(shí)間。
因此我們可以將client-output-buffer-limit replica 256mb 64mb 60作出簡(jiǎn)單調整,重啟Redis服務(wù)之后解決這個(gè)問(wèn)題。
評論
0 評論