听说Redis 3以后安装集群比较简单，几个命令就可以完全搞定，在自己的电脑上面安装试了下，果然比较爽。
今天准备把Redis 3的集群安装在K8s上面，通常的做法都是在github上面查找是不是有人已经做过，果然很多。下面的代码基本上来自于github上面。不过就算用这个面的代码，在自己部署的时候也存在不少的问题。因为github上面的都是大神，很比比较小的问题不会写的很清楚。
所以： 如果这个文档（blog）如何解决不了问题，就到github上面去查找别的方法。
1>>>>>>>>>>>镜像说明
使用的镜像名: ausov/redis:3.2-alpine
先使用下面命令下载镜像:
修改保存，重启一下docker或kubelet.
[html]
view plain
copy
1. docker pull ausov/redis:3.2-alpine
这个镜像如果无法下载，则需要自己手动build一下.
这个是编译需要使用的文件
[html]
view plain
copy
1. [root@centos-master alpine]# pwd
2. /root/k8s-redis/3.2/alpine
3. [root@centos-master alpine]# ls
4. docker-entrypoint.sh Dockerfile redis.conf
[html]
view plain
copy
1. cat docker-entrypoint.sh
[html]
view plain
copy
1. #!/bin/sh
2.
3. set -eox pipefail
4. #shopt -s nullglob
5.
6. REDIS_CONF=${REDIS_CONF:-"/opt/k8s-redis/redis.conf"}
7.
8. if [ "${1#-}" != "$1" ] || [ "${1%.conf}" != "$1" ]; then
9. set -- redis-server "$@"
10. fi
11.
12. if [ "$1" = 'redis-server' ] && [ -n "$SLAVEOF" ] && [ -z "$SENTINEL" ]; then
13. echo "Starting Redis replica"
14. set -- $@ "$REDIS_CONF" --slaveof "$SLAVEOF" 6379
15.
16. elif [ "$1" = 'redis-server' ] && [ -n "$SENTINEL" ]; then
17. echo "Starting Redis sentinel"
18.
19. while true; do
20. redis-cli -h $SENTINEL INFO
21. if [[ "$?" == "0" ]]; then
22. break
23. fi
24. echo "Connecting to master failed. Waiting..."
25. sleep 10
26. done
27.
28. >> "$REDIS_CONF"
29. >> "$REDIS_CONF"
30. >> "$REDIS_CONF"
31. >> "$REDIS_CONF"
32.
33. set -- $@ "$REDIS_CONF" --port 26379 --sentinel --protected-mode no
34.
35. elif [ "$1" = 'redis-server' ]; then
36. echo "Starting Redis master"
37. set -- $@ "$REDIS_CONF"
38. fi
39.
40. exec "$@"
这个文件内容太多，我们在这里看下可用的就OK。
[html]
view plain
copy
1. cat redis.conf | grep -Ev '(^#|^$)'
[html]
view plain
copy
1. protected-mode no
2. port 6379
3. tcp-backlog 511
4. timeout 0
5. tcp-keepalive 300
6. daemonize no
7. supervised no
8. pidfile /var/run/redis_6379.pid
9. loglevel notice
10. logfile ""
11. databases 16
12. save 900 1
13. save 300 10
14. save 60 10000
15. stop-writes-on-bgsave-error yes
16. rdbcompression yes
17. rdbchecksum yes
18. dbfilename dump.rdb
19. dir ./
20. slave-serve-stale-data yes
21. slave-read-only yes
22. repl-diskless-sync no
23. repl-diskless-sync-delay 5
24. repl-disable-tcp-nodelay no
25. slave-priority 100
26. appendonly yes
27. appendfilename "appendonly.aof"
28. appendfsync everysec
29. no-appendfsync-on-rewrite no
30. auto-aof-rewrite-percentage 100
31. auto-aof-rewrite-min-size 64mb
32. aof-load-truncated yes
33. lua-time-limit 5000
34. slowlog-log-slower-than 10000
35. slowlog-max-len 128
36. latency-monitor-threshold 0
37. notify-keyspace-events ""
38. hash-max-ziplist-entries 512
39. hash-max-ziplist-value 64
40. list-max-ziplist-size -2
41. list-compress-depth 0
42. set-max-intset-entries 512
43. zset-max-ziplist-entries 128
44. zset-max-ziplist-value 64
45. hll-sparse-max-bytes 3000
46. activerehashing yes
47. client-output-buffer-limit normal 0 0 0
48. client-output-buffer-limit slave 256mb 64mb 60
49. client-output-buffer-limit pubsub 32mb 8mb 60
50. hz 10
51. aof-rewrite-incremental-fsync yes
[html]
view plain
copy
1. cat Dockerfile
[html]
view plain
copy
1. FROM redis:3.2-alpine
2.
3. COPY ["redis.conf", "/opt/k8s-redis/"]
4. COPY ["docker-entrypoint.sh", "/usr/local/bin/"]
以上就是编译需要使用到的文件。cd到这个目录，执行下面的命令。
[html]
view plain
copy
1. docker build -t "ausov/redis:3.2-alpine" .
镜像编译完成以后，需要确保所有的节点都有这个镜像。
2>>>>>>>>>>>部署安装
部署安装我们需要使用三个文件，分别是:
primary.yml secondary.yml sentinel.yml
下面是文件内容:
[html]
view plain
copy
1. [root@centos-master example]# cat primary.yml
2. apiVersion: v1
3. kind: Service
4. metadata:
5. name: redis-primary
6. labels:
7. app: redis-primary
8. spec:
9. ports:
10. - port: 6379
11. name: redis-primary
12. clusterIP: None
13. selector:
14. app: redis-primary
15. ---
16. apiVersion: apps/v1beta1
17. kind: StatefulSet
18. metadata:
19. name: redis-primary
20. spec:
21. serviceName: redis-primary
22. replicas: 1
23. template:
24. metadata:
25. labels:
26. app: redis-primary
27. spec:
28. terminationGracePeriodSeconds: 10
29. containers:
30. - name: redis-primary
31. image: ausov/redis:3.2-alpine
32. imagePullPolicy: IfNotPresent
33. ports:
34. - containerPort: 6379
35. name: redis-primary
36. volumeMounts:
37. - name: data
38. mountPath: /data
39. volumeClaimTemplates:
40. - metadata:
41. name: data
42. annotations:
43. /storage-class: anything
44. spec:
45. accessModes: [ "ReadWriteOnce" ]
46. resources:
47. requests:
48. storage: 5Gi
[html]
view plain
copy
1. [root@centos-master example]# cat secondary.yml
2. apiVersion: v1
3. kind: Service
4. metadata:
5. name: redis-secondary
6. labels:
7. app: redis-secondary
8. spec:
9. ports:
10. - port: 6379
11. name: redis-secondary
12. clusterIP: None
13. selector:
14. app: redis-secondary
15. ---
16. apiVersion: apps/v1beta1
17. kind: StatefulSet
18. metadata:
19. name: redis-secondary
20. spec:
21. serviceName: redis-secondary
22. replicas: 2
23. template:
24. metadata:
25. labels:
26. app: redis-secondary
27. spec:
28. terminationGracePeriodSeconds: 10
29. containers:
30. - name: redis-secondary
31. image: ausov/redis:3.2-alpine
32. imagePullPolicy: IfNotPresent
33. env:
34. - name: SLAVEOF
35. value: redis-primary-0.redis-primary
36. ports:
37. - containerPort: 6379
38. name: redis-secondary
39. volumeMounts:
40. - name: data
41. mountPath: /data
42. volumeClaimTemplates:
43. - metadata:
44. name: data
45. annotations:
46. /storage-class: anything
47. spec:
48. accessModes: [ "ReadWriteOnce" ]
49. resources:
50. requests:
51. storage: 5Gi
[html]
view plain
copy
1. [root@centos-master example]# cat sentinel.yml
2. apiVersion: v1
3. kind: Service
4. metadata:
5. name: redis-sentinel
6. labels:
7. app: redis-sentinel
8. spec:
9. ports:
10. - port: 26379
11. name: redis-sentinel
12. clusterIP: None
13. selector:
14. app: redis-sentinel
15. ---
16. apiVersion: apps/v1beta1
17. kind: StatefulSet
18. metadata:
19. name: redis-sentinel
20. spec:
21. serviceName: redis-sentinel
22. replicas: 3
23. template:
24. metadata:
25. labels:
26. app: redis-sentinel
27. spec:
28. terminationGracePeriodSeconds: 10
29. containers:
30. - name: redis-sentinel
31. image: ausov/redis:3.2-alpine
32. imagePullPolicy: IfNotPresent
33. env:
34. - name: SENTINEL
35. value: redis-primary-0.redis-primary
36. ports:
37. - containerPort: 26379
38. name: redis-sentinel
三个文件都存在以后:
[html]
view plain
copy
1. kubectl create -f primary.yml -f secondary.yml -f sentinel.yml
创建完成后，查看一下pod
[html]
view plain
copy
1. kubectl get pod
会得到以下信息:
[html]
view plain
copy
1. NAME READY STATUS RESTARTS AGE
[html]
view plain
copy
1. redis-primary-0 1/1 Running 0 2d
2. redis-secondary-0 1/1 Running 0 2d
3. redis-secondary-1 1/1 Running 0 2d
4. redis-sentinel-0 1/1 Running 0 2d
5. redis-sentinel-1 1/1 Running 0 2d
6. redis-sentinel-2 1/1 Running 0 2d
下面是查看StatefulSet的信息：
[html]
view plain
copy
1. [root@centos-master example]# kubectl get statefulset
2. NAME DESIRED CURRENT AGE
3. redis-primary 1 1 2d
4. redis-secondary 2 2 2d
5. redis-sentinel 3 3 2d
必须要确保DESIRED和CURRENT的值是一样的。
查看持久存储的信息:
[html]
view plain
copy
1. [root@centos-master example]# kubectl get pvc
[html]
view plain
copy
1. NAME STATUS VOLUME CAPACITY ACCESSMODES AGE
[html]
view plain
copy
1. data-redis-primary-0 Bound pvc-94534255-d614-11e6-955d-1866daed1b04 5Gi RWO 2d
2. data-redis-secondary-0 Bound pvc-9a8f2080-d614-11e6-955d-1866daed1b04 5Gi RWO 2d
3. data-redis-secondary-1 Bound pvc-9a9029b4-d614-11e6-955d-1866daed1b04 5Gi RWO 2d
我们在注意这里的STATUS，是Bound。
如果不是，用命令
[html]
view plain
copy
1. describe pvc data-redis-primary-0
如果出现下面的问题：
cannot find volume plugin for alpha provisioning
则需要修改文件:
[html]
view plain
copy
1. /etc/kubernetes/manifests/kube-controller-manager.json
增加参数: --enable-hostpath-provisioner=true
以下是截取了这个文件的一部分:
[html]
view plain
copy
1. "name": "kube-controller-manager",
2. "image": "/google_containers/kube-controller-manager-amd64:v1.5.1",
3. "command": [
4. "kube-controller-manager",
5. --address=127.0.0.1",
6. "--leader-elect",
7. --master=127.0.0.1:8080",
8. --cluster-name=kubernetes",
9. --root-ca-file=/etc/kubernetes/pki/ca.pem",
10. --service-account-private-key-file=/etc/kubernetes/pki/apiserver-key.pem",
11. --cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem",
12. --cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem",
13. --insecure-experimental-approve-all-kubelet-csrs-for-group=system:kubelet-bootstrap",
14. --allocate-node-cidrs=true",
15. --enable-hostpath-provisioner=true",
16. --cluster-cidr=10.245.0.0/16"
17. ],
通过上面的配置，Redis基本就配置完成:
