Redis AOF刷新策略分析(转载)
- - 互联网 - ITeye博客此文为转载,原文: http://afei2.sinaapp.com/?p=536. redis支持使用aof来进行持久化,防止数据丢失,aof的刷新策略通过参数appendfsync控制,有三个值:always、everysec、no,默认是everysec. 下面从源码的角度剖析一下aof的刷新策略.
void aeMain(aeEventLoop *eventLoop) { eventLoop->stop = 0; while (!eventLoop->stop) { if (eventLoop->beforesleep != NULL) eventLoop->beforesleep(eventLoop); aeProcessEvents(eventLoop, AE_ALL_EVENTS); } } /* This function gets called every time Redis is entering the * main loop of the event driven library, that is, before to sleep * for ready file descriptors. */ void beforeSleep(struct aeEventLoop *eventLoop) { ...... /* Write the AOF buffer on disk */ flushAppendOnlyFile(0); ...... }上面的代码中的flushAppendOnlyFile(int force)进行实际的执行。
src/aof.c
void flushAppendOnlyFile(int force) { ...... /* Perform the fsync if needed. */ if (server.aof_fsync == AOF_FSYNC_ALWAYS) { /* aof_fsync is defined as fdatasync() for Linux in order to avoid * flushing metadata. */ latencyStartMonitor(latency); aof_fsync(server.aof_fd); /* Let's try to get this data on the disk */ latencyEndMonitor(latency); latencyAddSampleIfNeeded("aof-fsync-always",latency); server.aof_last_fsync = server.unixtime; } else if ((server.aof_fsync == AOF_FSYNC_EVERYSEC && server.unixtime > server.aof_last_fsync)) { if (!sync_in_progress) aof_background_fsync(server.aof_fd); server.aof_last_fsync = server.unixtime; } }AOF_FSYNC_ALWAYS会调用aof_fsync进行同步写入,而aof_fsync在linux下就是fdatasync,
/* Define aof_fsync to fdatasync() in Linux and fsync() for all the rest */ #ifdef __linux__ #define aof_fsync fdatasync #else #define aof_fsync fsync #endif /* Starts a background task that performs fsync() against the specified * file descriptor (the one of the AOF file) in another thread. */ void aof_background_fsync(int fd) { bioCreateBackgroundJob(REDIS_BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL); }其中everysec是通过下面的逻辑来进行的,检测后台是否fsync任务在进行,如果有的话,判断上次的fsync距离现在的时间,如果大于2s, 则阻塞,否则直接进行后台队列。
如果上一次的fsync执行了2s多,则会阻塞执行,直到写入成功,这个时候日志中会记录下面一条记录,并且增加info中对应的aof_delayed_fsync值
/* * When the fsync policy is set to 'everysec' we may delay the flush if there * is still an fsync() going on in the background thread, since for instance * on Linux write(2) will be blocked by the background fsync anyway. * When this happens we remember that there is some aof buffer to be * flushed ASAP, and will try to do that in the serverCron() function. * * However if force is set to 1 we'll write regardless of the background * fsync. * * 但是如果上一次的fsync执行了2s多,则会阻塞执行,直到写入成功 */ /* With this append fsync policy we do background fsyncing. * If the fsync is still in progress we can try to delay * the write for a couple of seconds. */ if (sync_in_progress) { if (server.aof_flush_postponed_start == 0) { /* No previous write postponinig, remember that we are * postponing the flush and return. */ server.aof_flush_postponed_start = server.unixtime; return; } else if (server.unixtime - server.aof_flush_postponed_start < 2) { /* We were already waiting for fsync to finish, but for less * than two seconds this is still ok. Postpone again. */ return; } /* Otherwise fall trough, and go write since we can't wait * over two seconds. */
aof_pending_bio_fsync /* Return the number of pending jobs of the specified type. */ bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC), serverCron中检查server.aof_flush_postponed_start,如果有的话,就追加一次flush,但是只有在上面的情况下会导致阻塞,其他情况下都会很快返回; /* AOF postponed flush: Try at every cron cycle if the slow fsync * completed. */ if (server.aof_flush_postponed_start) flushAppendOnlyFile(0);