é常ï¼ä¸ºäºæé«ç½ç«ååºé度ï¼æ»æ¯æçç¹æ°æ®ä¿åå¨å
åä¸èä¸æ¯ç´æ¥ä»å端æ°æ®åºä¸è¯»åãRedisæ¯ä¸ä¸ªå¾å¥½çCacheå·¥å
·ã大åç½ç«åºç¨ï¼çç¹æ°æ®éå¾å¾å·¨å¤§ï¼å åGä¸ç¾Gæ¯å¾æ£å¸¸çäºå¿ï¼å¨è¿ç§æ
åµä¸ï¼å¦ä½æ£ç¡®æ¶æRediså¢ï¼
é¦å
ï¼æ 论æ们æ¯ä½¿ç¨èªå·±çç©ç主æºï¼è¿æ¯ä½¿ç¨äºæå¡ä¸»æºï¼å
åèµæºå¾å¾æ¯æéå¶çï¼scale upä¸æ¯ä¸ä¸ªå¥½åæ³ï¼æ们éè¦scale out横åå¯ä¼¸ç¼©æ©å±ï¼è¿éè¦ç±å¤å°ä¸»æºååæä¾æå¡ï¼å³åå¸å¼å¤ä¸ªRediså®ä¾ååè¿è¡ã
å
¶æ¬¡ï¼ç®å硬件èµæºææ¬éä½ï¼å¤æ ¸CPUï¼å åGå
åç主æºå¾æ®éï¼å¯¹äºä¸»è¿ç¨æ¯å线ç¨å·¥ä½çRedisï¼åªè¿è¡ä¸ä¸ªå®ä¾å°±æ¾å¾æäºæµªè´¹ãåæ¶ï¼ç®¡çä¸ä¸ªå·¨å¤§å
åä¸å¦ç®¡çç¸å¯¹è¾å°çå
åé«æãå æ¤ï¼å®é
使ç¨ä¸ï¼é常ä¸å°æºå¨ä¸åæ¶è·å¤ä¸ªRediså®ä¾ã
æ¹æ¡
1.Rediså®æ¹é群æ¹æ¡ Redis Cluster
Redis Clusteræ¯ä¸ç§æå¡å¨Shardingææ¯ï¼3.0çæ¬å¼å§æ£å¼æä¾ã
Redis
Clusterä¸ï¼Shardingéç¨slot(槽)çæ¦å¿µï¼ä¸å
±åæ16384个槽ï¼è¿æç¹å¿ç±»ä¼¼åé¢è®²çpre
shardingæè·¯ã对äºæ¯ä¸ªè¿å
¥Redisçé®å¼å¯¹ï¼æ ¹æ®keyè¿è¡æ£åï¼åé
å°è¿16384个slotä¸çæä¸ä¸ªä¸ã使ç¨çhashç®æ³ä¹æ¯è¾ç®
åï¼å°±æ¯CRC16å16384å模ã
Redisé群ä¸çæ¯ä¸ªnode(èç¹)è´è´£åæè¿16384个slotä¸çä¸é¨åï¼ä¹å°±æ¯è¯´ï¼æ¯ä¸ª
sloté½å¯¹åºä¸ä¸ªnodeè´è´£å¤çãå½å¨ææ·»å æåå°nodeèç¹æ¶ï¼éè¦å°16384个槽å个ååé
ï¼æ§½ä¸çé®å¼ä¹è¦è¿ç§»ãå½ç¶ï¼è¿ä¸è¿ç¨ï¼å¨ç®åå®
ç°ä¸ï¼è¿å¤äºåèªå¨ç¶æï¼éè¦äººå·¥ä»å
¥ã
Redisé群ï¼è¦ä¿è¯16384个槽对åºçnodeé½æ£å¸¸å·¥ä½ï¼å¦ææ个nodeåçæ
éï¼é£å®è´è´£çslotsä¹å°±å¤±æï¼æ´ä¸ªé群å°ä¸è½å·¥ä½ã
为
äºå¢å é群çå¯è®¿é®æ§ï¼å®æ¹æ¨èçæ¹æ¡æ¯å°nodeé
ç½®æ主ä»ç»æï¼å³ä¸ä¸ªmaster主èç¹ï¼æn个slaveä»èç¹ãè¿æ¶ï¼å¦æ主èç¹å¤±
æï¼Redis Clusterä¼æ ¹æ®é举ç®æ³ä»slaveèç¹ä¸éæ©ä¸ä¸ªä¸å为主èç¹ï¼æ´ä¸ªé群继ç»å¯¹å¤æä¾æå¡ãè¿é常类似åç¯æç« æå°çRedis
Shardingåºæ¯ä¸æå¡å¨èç¹éè¿Sentinelçæ§æ¶ææ主ä»ç»æï¼åªæ¯Redis Clusteræ¬èº«æä¾äºæ
é转移容éçè½åã
Redis
Clusterçæ°èç¹è¯å«è½åãæ
éå¤æåæ
é转移è½åæ¯éè¿é群ä¸çæ¯ä¸ªnodeé½å¨åå
¶å®nodesè¿è¡éä¿¡ï¼è¿è¢«ç§°ä¸ºé群æ»çº¿(cluster
bus)ãå®ä»¬ä½¿ç¨ç¹æ®ç端å£å·ï¼å³å¯¹å¤æå¡ç«¯å£å·å 10000ãä¾å¦å¦ææ个nodeç端å£å·æ¯6379ï¼é£ä¹å®ä¸å
¶å®nodeséä¿¡ç端å£å·æ¯
16379ãnodesä¹é´çéä¿¡éç¨ç¹æ®çäºè¿å¶åè®®ã
对客æ·ç«¯æ¥è¯´ï¼æ´ä¸ªcluster被çåæ¯ä¸ä¸ªæ´ä½ï¼å®¢æ·ç«¯å¯ä»¥è¿æ¥ä»»æä¸ä¸ª
nodeè¿è¡æä½ï¼å°±åæä½åä¸Rediså®ä¾ä¸æ ·ï¼å½å®¢æ·ç«¯æä½çkey没æåé
å°è¯¥nodeä¸æ¶ï¼Redisä¼è¿å转åæ令ï¼æåæ£ç¡®çnodeï¼è¿
æç¹å¿åæµè§å¨é¡µé¢ç302 redirect跳转ã
Redis Clusteræ¯Redis 3.0以åææ£å¼æ¨åºï¼æ¶é´è¾æï¼ç®åè½è¯æå¨å¤§è§æ¨¡ç产ç¯å¢ä¸æåçæ¡ä¾è¿ä¸æ¯å¾å¤ï¼éè¦æ¶é´æ£éªã
2.Redis Shardingé群
Redis 3æ£å¼æ¨åºäºå®æ¹é群ææ¯ï¼è§£å³äºå¤Rediså®ä¾ååæå¡é®é¢ãRedis Clusterå¯ä»¥è¯´æ¯æå¡ç«¯Shardingåçææ¯çä½ç°ï¼å³å°é®å¼æç
§ä¸å®ç®æ³åçåé
å°å个å®ä¾åçä¸ï¼åæ¶å个å®ä¾èç¹åè°æ²éï¼å
±å对å¤æ¿æ
ä¸è´æå¡ã
å¤Rediså®ä¾æå¡ï¼æ¯åRediså®ä¾è¦å¤æçå¤ï¼è¿æ¶åå°å®ä½ãååã容éãæ©å®¹çææ¯é¾é¢ãè¿éï¼æ们ä»ç»ä¸ç§è½»é级ç客æ·ç«¯Redis Shardingææ¯ã
Redis
Shardingå¯ä»¥è¯´æ¯Redis
Clusteråºæ¥ä¹åï¼ä¸çæ®é使ç¨çå¤Rediså®ä¾é群æ¹æ³ãå
¶ä¸»è¦ææ³æ¯éç¨åå¸ç®æ³å°Redisæ°æ®çkeyè¿è¡æ£åï¼éè¿hashå½æ°ï¼ç¹å®
çkeyä¼æ å°å°ç¹å®çRedisèç¹ä¸ãè¿æ ·ï¼å®¢æ·ç«¯å°±ç¥é该ååªä¸ªRedisèç¹æä½æ°æ®ãShardingæ¶æå¦å¾ï¼
åºå¹¸çæ¯ï¼java redis客æ·ç«¯é©±å¨jedisï¼å·²æ¯æRedis Shardingåè½ï¼å³ShardedJedis以åç»åç¼åæ± çShardedJedisPoolã
JedisçRedis Shardingå®ç°å
·æå¦ä¸ç¹ç¹ï¼
é
ç¨ä¸è´æ§åå¸ç®æ³(consistent
hashing)ï¼å°keyåèç¹nameåæ¶hashingï¼ç¶åè¿è¡æ å°å¹é
ï¼éç¨çç®æ³æ¯MURMUR_HASHãéç¨ä¸è´æ§åå¸èä¸æ¯éç¨ç®åç±»
ä¼¼åå¸æ±æ¨¡æ å°ç主è¦åå æ¯å½å¢å æåå°èç¹æ¶ï¼ä¸ä¼äº§çç±äºéæ°å¹é
é æçrehashingãä¸è´æ§åå¸åªå½±åç¸é»èç¹keyåé
ï¼å½±åéå°ã
2.
为äºé¿å
ä¸è´æ§åå¸åªå½±åç¸é»èç¹é æèç¹åé
ååï¼ShardedJedisä¼å¯¹æ¯ä¸ªRedisèç¹æ ¹æ®åå(没æï¼Jedisä¼èµäºç¼ºçåå)ä¼èæ
ååº160个èæèç¹è¿è¡æ£åãæ ¹æ®æéweightï¼ä¹å¯èæååº160åæ°çèæèç¹ãç¨èæèç¹åæ å°å¹é
ï¼å¯ä»¥å¨å¢å æåå°Redisèç¹
æ¶ï¼keyå¨åRedisèç¹ç§»å¨ååé
æ´ååï¼èä¸æ¯åªæç¸é»èç¹åå½±åã
3.ShardedJedisæ¯ækeyTagPattern模å¼ï¼å³æ½åkeyçä¸é¨åkeyTagåshardingï¼è¿æ ·éè¿åçå½åkeyï¼å¯ä»¥å°ä¸ç»ç¸å
³èçkeyæ¾å
¥åä¸ä¸ªRedisèç¹ï¼è¿å¨é¿å
è·¨èç¹è®¿é®ç¸å
³æ°æ®æ¶å¾éè¦ã
Redis Shardingéç¨å®¢æ·ç«¯Shardingæ¹å¼ï¼æå¡ç«¯Redisè¿æ¯ä¸ä¸ªä¸ªç¸å¯¹ç¬ç«çRediså®ä¾èç¹ï¼æ²¡æåä»»ä½åå¨ãåæ¶ï¼æ们ä¹ä¸éè¦å¢å é¢å¤çä¸é´å¤çç»ä»¶ï¼è¿æ¯ä¸ç§é常轻éãçµæ´»çRediså¤å®ä¾é群æ¹æ³ã
å½ç¶ï¼Redis Shardingè¿ç§è½»éçµæ´»æ¹å¼å¿
ç¶å¨é群å
¶å®è½åæ¹é¢ååºå¦¥åãæ¯å¦æ©å®¹ï¼å½æ³è¦å¢å Redisèç¹æ¶ï¼å°½ç®¡éç¨ä¸è´æ§åå¸ï¼æ¯ç«è¿æ¯ä¼ækeyå¹é
ä¸å°è丢失ï¼è¿æ¶éè¦é®å¼è¿ç§»ã
ä½ä¸ºè½»é级客æ·ç«¯shardingï¼å¤çRedisé®å¼è¿ç§»æ¯ä¸ç°å®çï¼è¿å°±è¦æ±åºç¨å±é¢å
许Redisä¸æ°æ®ä¸¢å¤±æä»å端æ°æ®åºéæ°å è½½æ°æ®ãä½æäºæ¶åï¼å»ç©¿ç¼åå±ï¼ç´æ¥è®¿é®æ°æ®åºå±ï¼ä¼å¯¹ç³»ç»è®¿é®é æå¾å¤§ååãæ没æå
¶å®æ段æ¹åè¿ç§æ
åµï¼
Redis
ä½è
ç»åºäºä¸ä¸ªæ¯è¾è®¨å·§çåæ³--preshardingï¼å³é¢å
æ ¹æ®ç³»ç»è§æ¨¡å°½éé¨ç½²å¥½å¤ä¸ªRediså®ä¾ï¼è¿äºå®ä¾å ç¨ç³»ç»èµæºå¾å°ï¼ä¸å°ç©çæºå¯é¨
ç½²å¤ä¸ªï¼è®©ä»ä»¬é½åä¸shardingï¼å½éè¦æ©å®¹æ¶ï¼éä¸ä¸ä¸ªå®ä¾ä½ä¸ºä¸»èç¹ï¼æ°å å
¥çRedisèç¹ä½ä¸ºä»èç¹è¿è¡æ°æ®å¤å¶ãæ°æ®åæ¥åï¼ä¿®æ¹
shardingé
ç½®ï¼è®©æååå®ä¾çShardæåæ°æºå¨ä¸æ©å®¹åçRedisèç¹ï¼åæ¶è°æ´æ°Redisèç¹ä¸ºä¸»èç¹ï¼åå®ä¾å¯ä¸å使ç¨ã
presharding
æ¯é¢å
åé
好足å¤çåçï¼æ©å®¹æ¶åªæ¯å°å±äºæä¸åççåRediså®ä¾æ¿æ¢ææ°ç容éæ´å¤§çRediså®ä¾ãåä¸shardingçåç没ææ¹åï¼æ以ä¹
å°±ä¸åå¨keyå¼ä»ä¸ä¸ªåºè½¬ç§»å°å¦ä¸ä¸ªåçåºçç°è±¡ï¼åªæ¯å°å±äºååçåºçé®å¼ä»åRediså®ä¾åæ¥å°æ°Rediså®ä¾ã
并ä¸æ¯åªæå¢
å Redisèç¹å¼èµ·é®å¼ä¸¢å¤±é®é¢ï¼æ´å¤§çéç¢æ¥èªRedisèç¹çªç¶å®æºãå¨ãRedisæä¹
åãä¸æä¸å·²æå°ï¼ä¸ºä¸å½±åRedisæ§è½ï¼å°½éä¸å¼å¯
AOFåRDBæ件ä¿ååè½ï¼å¯æ¶æRedis主å¤æ¨¡å¼ï¼ä¸»Rediså®æºï¼æ°æ®ä¸ä¼ä¸¢å¤±ï¼å¤Redisçæå¤ä»½ã
è¿æ ·ï¼æ们çæ¶æ模å¼å
æä¸ä¸ªRedisèç¹åçå
å«ä¸ä¸ªä¸»Redisåä¸ä¸ªå¤Redisãå¨ä¸»Rediså®æºæ¶ï¼å¤Redisæ¥ç®¡è¿æ¥ï¼ä¸å为主Redisï¼ç»§ç»æä¾æå¡ã主
å¤å
±åç»æä¸ä¸ªRedisèç¹ï¼éè¿èªå¨æ
é转移ï¼ä¿è¯äºèç¹çé«å¯ç¨æ§ãåShardingæ¶ææ¼åæï¼
Redis Sentinelæä¾äºä¸»å¤æ¨¡å¼ä¸Redisçæ§ãæ
é转移åè½è¾¾å°ç³»ç»çé«å¯ç¨æ§ã
é«è®¿é®éä¸ï¼å³ä½¿éç¨Shardingåçï¼ä¸ä¸ªåç¬èç¹è¿æ¯æ¿æ
äºå¾å¤§ç访é®ååï¼è¿æ¶æ们è¿éè¦è¿ä¸æ¥å解ãé常æ
åµä¸ï¼åºç¨è®¿é®Redis读æä½éååæä½éå·®å¼å¾å¤§ï¼è¯»å¸¸å¸¸æ¯åçæ°åï¼è¿æ¶æ们å¯ä»¥å°è¯»åå离ï¼èä¸è¯»æä¾æ´å¤çå®ä¾æ°ã
å¯ä»¥å©ç¨ä¸»ä»æ¨¡å¼å®ç°è¯»åå离ï¼ä¸»è´è´£åï¼ä»è´è´£åªè¯»ï¼åæ¶ä¸ä¸»æå¤ä¸ªä»ãå¨Sentinelçæ§ä¸ï¼è¿å¯ä»¥ä¿éèç¹æ
éçèªå¨çæµã
3.å©ç¨ä»£çä¸é´ä»¶å®ç°å¤§è§æ¨¡Redisé群
ä¸é¢åå«ä»ç»äºå¤Redisæå¡å¨é群ç两ç§æ¹å¼ï¼å®ä»¬æ¯åºäºå®¢æ·ç«¯shardingçRedis Shardingååºäºæå¡ç«¯shardingçRedis Clusterã
客æ·ç«¯shardingææ¯å
¶ä¼å¿å¨äºæå¡ç«¯çRediså®ä¾å½¼æ¤ç¬ç«ï¼ç¸äºæ å
³èï¼æ¯ä¸ªRediså®ä¾ååæå¡å¨ä¸æ ·è¿è¡ï¼é常容æ线æ§æ©å±ï¼ç³»ç»ççµæ´»æ§å¾å¼ºãå
¶ä¸è¶³ä¹å¤å¨äºï¼
ç±äºshardingå¤çæ¾å°å®¢æ·ç«¯ï¼è§æ¨¡è¿æ¥æ©å¤§æ¶ç»è¿ç»´å¸¦æ¥ææã
æå¡ç«¯Rediså®ä¾ç¾¤ææç»ææååæ¶ï¼æ¯ä¸ªå®¢æ·ç«¯é½éè¦æ´æ°è°æ´ã
è¿æ¥ä¸è½å
±äº«ï¼å½åºç¨è§æ¨¡å¢å¤§æ¶ï¼èµæºæµªè´¹å¶çº¦ä¼åã
æå¡ç«¯shardingçRedis Clusterå
¶ä¼å¿å¨äºæå¡ç«¯Redisé群ææç»æååæ¶ï¼å®¢æ·ç«¯ä¸éè¦æç¥ï¼å®¢æ·ç«¯å使ç¨åRedisæå¡å¨ä¸æ ·ä½¿ç¨Redisé群ï¼è¿ç»´ç®¡çä¹æ¯è¾æ¹ä¾¿ã
ä¸è¿Redis Clusteræ£å¼çæ¨åºæ¶é´ä¸é¿ï¼ç³»ç»ç¨³å®æ§ãæ§è½çé½éè¦æ¶é´æ£éªï¼å°¤å
¶å¨å¤§è§æ¨¡ä½¿ç¨åºåã
è½ä¸è½ç»åäºè
ä¼å¿ï¼å³è½ä½¿æå¡ç«¯åå®ä¾å½¼æ¤ç¬ç«ï¼æ¯æ线æ§å¯ä¼¸ç¼©ï¼åæ¶shardingåè½éä¸å¤çï¼æ¹ä¾¿ç»ä¸ç®¡çï¼æ¬ç¯ä»ç»çRedis代çä¸é´ä»¶twemproxyå°±æ¯è¿æ ·ä¸ç§å©ç¨ä¸é´ä»¶åshardingçææ¯ã
twemproxyå¤äºå®¢æ·ç«¯åæå¡å¨çä¸é´ï¼å°å®¢æ·ç«¯åæ¥ç请æ±ï¼è¿è¡ä¸å®çå¤çå(å¦sharding)ï¼å转åç»å端çæ£çRedisæå¡å¨ãä¹å°±æ¯è¯´ï¼å®¢æ·ç«¯ä¸ç´æ¥è®¿é®Redisæå¡å¨ï¼èæ¯éè¿twemproxy代çä¸é´ä»¶é´æ¥è®¿é®ã
åç
§Redis Shardingæ¶æï¼å¢å 代çä¸é´ä»¶çRedisé群æ¶æå¦ä¸ï¼
twemproxyä¸é´ä»¶çå
é¨å¤çæ¯æ ç¶æçï¼å®æ¬èº«å¯ä»¥å¾è½»æ¾å°é群ï¼è¿æ ·å¯é¿å
åç¹ååææ
éã
twemproxyåå«nutcrackerï¼èµ·æºäºtwitterç³»ç»ä¸redis/memcachedé群å¼åå®è·µï¼è¿è¡ææè¯å¥½ï¼å代ç å¥ç®ç»å¼æºç¤¾åºãå
¶è½»éé«æï¼éç¨Cè¯è¨å¼åï¼å·¥ç¨ç½åæ¯ï¼GitHub - twitter/twemproxy: A fast, light-weight proxy for memcached and redis
twemproxyå端ä¸ä»
æ¯æredisï¼åæ¶ä¹æ¯æmemcachedï¼è¿æ¯twitterç³»ç»å
·ä½ç¯å¢é æçã
ç±äºä½¿ç¨äºä¸é´ä»¶ï¼twemproxyå¯ä»¥éè¿å
±äº«ä¸å端系ç»çè¿æ¥ï¼éä½å®¢æ·ç«¯ç´æ¥è¿æ¥å端æå¡å¨çè¿æ¥æ°éãåæ¶ï¼å®ä¹æä¾shardingåè½ï¼æ¯æå端æå¡å¨é群水平æ©å±ãç»ä¸è¿ç»´ç®¡çä¹å¸¦æ¥äºæ¹ä¾¿ã
å½ç¶ï¼ä¹æ¯ç±äºä½¿ç¨äºä¸é´ä»¶ä»£çï¼ç¸æ¯å®¢æ·ç«¯ç´è¿æå¡å¨æ¹å¼ï¼æ§è½ä¸ä¼æææèï¼å®æµç»æ大约éä½äº20%å·¦å³ã
温馨提示:答案为网友推荐,仅供参考