Apa yang dilakukan Redis jika kehabisan memori?

Ini mungkin pertanyaan yang mudah tetapi saya kesulitan menemukan jawabannya. Bagaimana Redis 2.0 menangani kehabisan memori yang dialokasikan maksimum? Bagaimana cara memutuskan data mana yang akan dihapus atau data mana yang disimpan dalam memori?

Jika Anda mengaktifkan fungsi memori virtual (baru di versi 2.0 atau 2.2, menurut saya), maka Redis mulai menyimpan data yang "tidak terlalu sering digunakan" ke disk saat memori habis.

Jika memori virtual di Redis dinonaktifkan, tampaknya memori virtual sistem operasi mulai digunakan (yaitu swap), dan kinerja turun drastis.

Sekarang, Anda juga dapat mengkonfigurasi Redis dengan parameter maxmemory, yang mencegah Redis menggunakan lebih banyak memori (default).

Versi Redis yang lebih baru memiliki berbagai kebijakan saat memori maksimum tercapai:

• volatile-lru menghapus kunci di antara yang memiliki set kedaluwarsa, mencoba untuk menghapus kunci yang tidak digunakan baru-baru ini.
• volatile-ttl menghapus kunci di antara yang memiliki set kedaluwarsa, mencoba menghapus kunci dengan sisa waktu singkat untuk hidup.
• volatile-random menghapus kunci acak di antara yang memiliki set kedaluwarsa.
• allkeys-lru seperti volatile-lru, tetapi akan menghapus semua jenis kunci, baik kunci normal maupun kunci dengan satu set kedaluwarsa.
• allkeys-random seperti volatile-random, tetapi akan menghapus semua jenis kunci, baik kunci normal maupun kunci dengan set kedaluwarsa.

Jika Anda memilih kebijakan yang hanya menghapus kunci dengan set EXPIRE, maka saat Redis kehabisan memori, sepertinya program baru saja membatalkan operasi malloc (). Artinya, jika Anda mencoba menyimpan lebih banyak data, operasi itu gagal total.

Beberapa tautan untuk info lebih lanjut (karena Anda seharusnya tidak hanya mengambil kata-kata saya):

• http://antirez.com/post/redis-as-LRU-cache.html
• http://eli.thegreenplace.net/2009/10/30/handling-out-of-memory-conditions-in-c/

Dari redis.conf , versi 2.8

# Don't use more memory than the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys
# according to the eviction policy selected (see maxmemory-policy).
#
# If Redis can't remove keys according to the policy, or if the policy is
# set to 'noeviction', Redis will start to reply with errors to commands
# that would use more memory, like SET, LPUSH, and so on, and will continue
# to reply to read-only commands like GET.
#
# This option is usually useful when using Redis as an LRU cache, or to set
# a hard memory limit for an instance (using the 'noeviction' policy).
#
# WARNING: If you have slaves attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the slaves are subtracted
# from the used memory count, so that network problems / resyncs will
# not trigger a loop where keys are evicted, and in turn the output
# buffer of slaves is full with DELs of keys evicted triggering the deletion
# of more keys, and so forth until the database is completely emptied.
#
# In short... if you have slaves attached it is suggested that you set a lower
# limit for maxmemory so that there is some free RAM on the system for slave
# output buffers (but this is not needed if the policy is 'noeviction').
#
# maxmemory <bytes>

# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select among five behaviors:
#
# volatile-lru -> remove the key with an expire set using an LRU algorithm
# allkeys-lru -> remove any key according to the LRU algorithm
# volatile-random -> remove a random key with an expire set
# allkeys-random -> remove a random key, any key
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
# noeviction -> don't expire at all, just return an error on write operations
#
# Note: with any of the above policies, Redis will return an error on write
#       operations, when there are no suitable keys for eviction.
#
#       At the date of writing these commands are: set setnx setex append
#       incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
#       sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
#       zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
#       getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy volatile-lru

Perbarui redis 4.0

127.0.0.1:6379> MEMORY HELP
1) "MEMORY DOCTOR                        - Outputs memory problems report"
2) "MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key"
3) "MEMORY STATS                         - Show memory usage details"
4) "MEMORY PURGE                         - Ask the allocator to release memory"
5) "MEMORY MALLOC-STATS                  - Show allocator internal stats"

/usr/local/etc/redis.conf

############################## MEMORY MANAGEMENT ################################

# Set a memory usage limit to the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys
# according to the eviction policy selected (see maxmemory-policy).
#
# If Redis can't remove keys according to the policy, or if the policy is
# set to 'noeviction', Redis will start to reply with errors to commands
# that would use more memory, like SET, LPUSH, and so on, and will continue
# to reply to read-only commands like GET.
#
# This option is usually useful when using Redis as an LRU or LFU cache, or to
# set a hard memory limit for an instance (using the 'noeviction' policy).
#
# WARNING: If you have slaves attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the slaves are subtracted
# from the used memory count, so that network problems / resyncs will
# not trigger a loop where keys are evicted, and in turn the output
# buffer of slaves is full with DELs of keys evicted triggering the deletion
# of more keys, and so forth until the database is completely emptied.
#
# In short... if you have slaves attached it is suggested that you set a lower
# limit for maxmemory so that there is some free RAM on the system for slave
# output buffers (but this is not needed if the policy is 'noeviction').
#
# maxmemory <bytes>

# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select among five behaviors:
#
# volatile-lru -> Evict using approximated LRU among the keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key among the ones with an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.
#
# LRU means Least Recently Used
# LFU means Least Frequently Used
#
# Both LRU, LFU and volatile-ttl are implemented using approximated
# randomized algorithms.
#
# Note: with any of the above policies, Redis will return an error on write
#       operations, when there are no suitable keys for eviction.
#
#       At the date of writing these commands are: set setnx setex append
#       incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
#       sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
#       zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
#       getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy noeviction

# LRU, LFU and minimal TTL algorithms are not precise algorithms but approximated
# algorithms (in order to save memory), so you can tune it for speed or
# accuracy. For default Redis will check five keys and pick the one that was
# used less recently, you can change the sample size using the following
# configuration directive.
#
# The default of 5 produces good enough results. 10 Approximates very closely
# true LRU but costs more CPU. 3 is faster but not very accurate.
#
# maxmemory-samples 5

Saya baru saja mulai membaca tentang Redis, jadi saya tidak yakin. Tapi, saya menemukan beberapa informasi yang mungkin berguna.

Berikut cuplikan dari http://antirez.com/post/redis-as-LRU-cache.html :

Cara lain untuk menggunakan Redis sebagai cache adalah direktif maxmemory, fitur yang memungkinkan menentukan jumlah memori maksimum yang akan digunakan. Ketika data baru ditambahkan ke server, dan batas memori telah tercapai, server akan menghapus beberapa data lama menghapus kunci volatil, yaitu kunci dengan set EXPIRE (batas waktu), meskipun kuncinya masih jauh dari kedaluwarsa secara otomatis.

Selain itu, Redis 2.0 memiliki mode VM di mana semua kunci harus masuk ke dalam memori, tetapi nilai untuk kunci yang jarang digunakan dapat berada di disk:

• http://redis.io/topics/virtual-memory
• http://antirez.com/post/redis-virtual-memory-story.html

Jika Anda bertanya-tanya apa yang sebenarnya ditanggapi oleh Redis (2.8) ketika mencapai maksimum yang ditentukan oleh konfigurasinya, tampilannya seperti ini:

$ redis-cli
127.0.0.1:6379> GET 5
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
127.0.0.1:6379> SET 5 a
(error) OOM command not allowed when used memory > 'maxmemory'.

Saya baru-baru ini mengalami situasi tanpa memori bebas dan aplikasi saya terhenti (penulisan tidak mungkin, pembacaan dimungkinkan), menjalankan skrip PHP berhenti mati di jalurnya di tengah jalan dan harus dilakukan kill -9secara manual (bahkan setelah memori tersedia).

Saya berasumsi kehilangan data (atau ketidakkonsistenan data) telah terjadi, jadi saya melakukan flushdbdan memulihkan dari cadangan. Pelajaran yang dipelajari? Cadangan adalah teman Anda.

Redis bukanlah cache seperti memcache, secara default ( maxmemory-policyparameter disetel ke noeviction) semua data yang Anda masukkan ke redis tidak akan dihapus, satu-satunya pengecualian adalah menggunakan EXPIRE.