956237586 – 第 6 页 – HYL Studio

视频版

https://www.bilibili.com/video/BV1u84y1y7Mu/

1.安装redis

环境信息，按官方文档准备三个节点


redis1
192.168.2.104
ssh redis1@192.168.2.104

redis2
192.168.2.166
ssh redis2@192.168.2.166

redis3
192.168.2.139
ssh redis3@192.168.2.139

redis1

192.168.2.104

ssh redis1@192.168.2.104

redis2

192.168.2.166

ssh redis2@192.168.2.166

redis3

192.168.2.139

ssh redis3@192.168.2.139

执行安装过程


curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list

sudo apt-get update
sudo apt-get install redis redis-sentinel

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list

sudo apt-get update

sudo apt-get install redis redis-sentinel

2.配置集群复制配置

复制的配置，假设第一个是主节点。这里测试没做安全防护，生产记得加密码和网段限制


sudo vim /etc/redis/redis.conf
#config all nodes
bind * -::1
protected-mode no

#only in redis2 and redis3
replicaof 192.168.2.104 6379

sudo vim /etc/redis/redis.conf

#config all nodes

bind * -::1

protected-mode no

#only in redis2 and redis3

replicaof 192.168.2.104 6379

记得重启，之后测试集群复制情况，正常后继续配置sentinel


redis1@redis1:~$ redis-cli -p 6379
127.0.0.1:6379> get test
"2"
127.0.0.1:6379> set test 1
OK
127.0.0.1:6379> get test
"1"
127.0.0.1:6379> exit
redis1@redis1:~$ redis-cli -h 192.168.2.166 -p 6379
192.168.2.166:6379> get test
"1"
192.168.2.166:6379>
redis1@redis1:~$ ^C
redis1@redis1:~$ redis-cli -h 192.168.2.139 -p 6379
192.168.2.139:6379> get test
"1"
192.168.2.139:6379>

redis1@redis1:~$ redis-cli -p 6379

127.0.0.1:6379> get test

"2"

127.0.0.1:6379> set test 1

127.0.0.1:6379> get test

"1"

127.0.0.1:6379> exit

redis1@redis1:~$ redis-cli -h 192.168.2.166 -p 6379

192.168.2.166:6379> get test

"1"

192.168.2.166:6379>

redis1@redis1:~$ ^C

redis1@redis1:~$ redis-cli -h 192.168.2.139 -p 6379

192.168.2.139:6379> get test

"1"

192.168.2.139:6379>

3.配置sentinel

新建配置文件sentinel.conf 这里测试的是自己新建的，记得关掉系统的service防止干扰。实际应该修改/etc/redis下的配置


port 5000
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1

#with ip or port mapping(docker or nat)
sentinel announce-ip <ip>
sentinel announce-port <port>

port 5000

sentinel monitor mymaster 127.0.0.1 6379 2

sentinel down-after-milliseconds mymaster 5000

sentinel failover-timeout mymaster 60000

sentinel parallel-syncs mymaster 1

#with ip or port mapping(docker or nat)

sentinel announce-ip <ip>

sentinel announce-port <port>

启动成功效果


redis1@redis1:~$ redis-sentinel sentinel.conf
3385:X 27 Nov 2022 12:11:35.380 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3385:X 27 Nov 2022 12:11:35.380 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=3385, just started
3385:X 27 Nov 2022 12:11:35.380 # Configuration loaded
3385:X 27 Nov 2022 12:11:35.381 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3385:X 27 Nov 2022 12:11:35.381 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 7.0.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 5000
 |    `-._   `._    /     _.-'    |     PID: 3385
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

3385:X 27 Nov 2022 12:11:35.387 * Sentinel new configuration saved on disk
3385:X 27 Nov 2022 12:11:35.387 # Sentinel ID is 9a7a4e3269530706fd1002c3018a3f0c525c25d6
3385:X 27 Nov 2022 12:11:35.387 # +monitor master mymaster 192.168.2.104 6379 quorum 2
3385:X 27 Nov 2022 12:11:52.666 * +sentinel sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379
3385:X 27 Nov 2022 12:11:52.670 * Sentinel new configuration saved on disk
3385:X 27 Nov 2022 12:11:59.182 * +sentinel sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379
3385:X 27 Nov 2022 12:11:59.186 * Sentinel new configuration saved on disk

redis1@redis1:~$ redis-sentinel sentinel.conf

3385:X 27 Nov 2022 12:11:35.380 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3385:X 27 Nov 2022 12:11:35.380 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=3385, just started

3385:X 27 Nov 2022 12:11:35.380 # Configuration loaded

3385:X 27 Nov 2022 12:11:35.381 * Increased maximum number of open files to 10032 (it was originally set to 1024).

3385:X 27 Nov 2022 12:11:35.381 * monotonic clock: POSIX clock_gettime

_._

_.-``__ ''-._

_.-`` `. `_. ''-._ Redis 7.0.5 (00000000/0) 64 bit

.-`` .-```. ```\/ _.,_ ''-._

( ' , .-` | `, ) Running in sentinel mode

|`-._`-...-` __...-.``-._|'` _.-'| Port: 5000

| `-._ `._ / _.-' | PID: 3385

`-._ `-._ `-./ _.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' | https://redis.io

`-._ `-._`-.__.-'_.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' |

`-._ `-._`-.__.-'_.-' _.-'

`-._ `-.__.-' _.-'

`-._ _.-'

`-.__.-'

3385:X 27 Nov 2022 12:11:35.387 * Sentinel new configuration saved on disk

3385:X 27 Nov 2022 12:11:35.387 # Sentinel ID is 9a7a4e3269530706fd1002c3018a3f0c525c25d6

3385:X 27 Nov 2022 12:11:35.387 # +monitor master mymaster 192.168.2.104 6379 quorum 2

3385:X 27 Nov 2022 12:11:52.666 * +sentinel sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379

3385:X 27 Nov 2022 12:11:52.670 * Sentinel new configuration saved on disk

3385:X 27 Nov 2022 12:11:59.182 * +sentinel sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379

3385:X 27 Nov 2022 12:11:59.186 * Sentinel new configuration saved on disk

观察端口监听5000


redis1@redis1:~$ netstat -anp|grep 5000
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:5000            0.0.0.0:*               LISTEN      3385/redis-sentinel
tcp        0      0 192.168.2.104:55560     192.168.2.139:5000      ESTABLISHED 3385/redis-sentinel
tcp        0      0 192.168.2.104:44506     192.168.2.166:5000      ESTABLISHED 3385/redis-sentinel
tcp        0      0 192.168.2.104:5000      192.168.2.166:36802     ESTABLISHED 3385/redis-sentinel
tcp        0      0 192.168.2.104:5000      192.168.2.139:33910     ESTABLISHED 3385/redis-sentinel
tcp6       0      0 :::5000                 :::*                    LISTEN      3385/redis-sentinel

redis1@redis1:~$ netstat -anp|grep 5000

(Not all processes could be identified, non-owned process info

will not be shown, you would have to be root to see it all.)

tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 3385/redis-sentinel

tcp 0 0 192.168.2.104:55560 192.168.2.139:5000 ESTABLISHED 3385/redis-sentinel

tcp 0 0 192.168.2.104:44506 192.168.2.166:5000 ESTABLISHED 3385/redis-sentinel

tcp 0 0 192.168.2.104:5000 192.168.2.166:36802 ESTABLISHED 3385/redis-sentinel

tcp 0 0 192.168.2.104:5000 192.168.2.139:33910 ESTABLISHED 3385/redis-sentinel

tcp6 0 0 :::5000 :::* LISTEN 3385/redis-sentinel

查看集群状态


sentinel master mymaster
SENTINEL get-master-addr-by-name mymaster
SENTINEL replicas mymaster
SENTINEL sentinels mymaster

sentinel master mymaster

SENTINEL get-master-addr-by-name mymaster

SENTINEL replicas mymaster

SENTINEL sentinels mymaster

redis1


redis1@redis1:~$ redis-cli -p 5000
127.0.0.1:5000> sentinel master mymaster
 1) "name"
 2) "mymaster"
 3) "ip"
 4) "192.168.2.104"
 5) "port"
 6) "6379"
 7) "runid"
 8) "4097e9a96cdb74dd4239aede54843e33104e6913"
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "244"
19) "last-ping-reply"
20) "244"
21) "down-after-milliseconds"
22) "5000"
23) "info-refresh"
24) "3457"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "103880"
29) "config-epoch"
30) "0"
31) "num-slaves"
32) "0"
33) "num-other-sentinels"
34) "2"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "60000"
39) "parallel-syncs"
40) "1"

127.0.0.1:5000> SENTINEL get-master-addr-by-name mymaster
1) "192.168.2.104"
2) "6379"

127.0.0.1:5000> SENTINEL sentinels mymaster
1)  1) "name"
    2) "ea6cbec52f9c04c6c325414a05562a69b503129c"
    3) "ip"
    4) "192.168.2.166"
    5) "port"
    6) "5000"
    7) "runid"
    8) "ea6cbec52f9c04c6c325414a05562a69b503129c"
    9) "flags"
   10) "sentinel"
   11) "link-pending-commands"
   12) "0"
   13) "link-refcount"
   14) "1"
   15) "last-ping-sent"
   16) "0"
   17) "last-ok-ping-reply"
   18) "241"
   19) "last-ping-reply"
   20) "241"
   21) "down-after-milliseconds"
   22) "5000"
   23) "last-hello-message"
   24) "763"
   25) "voted-leader"
   26) "?"
   27) "voted-leader-epoch"
   28) "0"
2)  1) "name"
    2) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"
    3) "ip"
    4) "192.168.2.139"
    5) "port"
    6) "5000"
    7) "runid"
    8) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"
    9) "flags"
   10) "sentinel"
   11) "link-pending-commands"
   12) "0"
   13) "link-refcount"
   14) "1"
   15) "last-ping-sent"
   16) "0"
   17) "last-ok-ping-reply"
   18) "241"
   19) "last-ping-reply"
   20) "241"
   21) "down-after-milliseconds"
   22) "5000"
   23) "last-hello-message"
   24) "425"
   25) "voted-leader"
   26) "?"
   27) "voted-leader-epoch"
   28) "0"

100

101

102

103

104

redis1@redis1:~$ redis-cli -p 5000

127.0.0.1:5000> sentinel master mymaster

1) "name"

2) "mymaster"

3) "ip"

4) "192.168.2.104"

5) "port"

6) "6379"

7) "runid"

8) "4097e9a96cdb74dd4239aede54843e33104e6913"

9) "flags"

10) "master"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "244"

19) "last-ping-reply"

20) "244"

21) "down-after-milliseconds"

22) "5000"

23) "info-refresh"

24) "3457"

25) "role-reported"

26) "master"

27) "role-reported-time"

28) "103880"

29) "config-epoch"

30) "0"

31) "num-slaves"

32) "0"

33) "num-other-sentinels"

34) "2"

35) "quorum"

36) "2"

37) "failover-timeout"

38) "60000"

39) "parallel-syncs"

40) "1"

127.0.0.1:5000> SENTINEL get-master-addr-by-name mymaster

1) "192.168.2.104"

2) "6379"

127.0.0.1:5000> SENTINEL sentinels mymaster

1) 1) "name"

2) "ea6cbec52f9c04c6c325414a05562a69b503129c"

3) "ip"

4) "192.168.2.166"

5) "port"

6) "5000"

7) "runid"

8) "ea6cbec52f9c04c6c325414a05562a69b503129c"

9) "flags"

10) "sentinel"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "241"

19) "last-ping-reply"

20) "241"

21) "down-after-milliseconds"

22) "5000"

23) "last-hello-message"

24) "763"

25) "voted-leader"

26) "?"

27) "voted-leader-epoch"

28) "0"

2) 1) "name"

2) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"

3) "ip"

4) "192.168.2.139"

5) "port"

6) "5000"

7) "runid"

8) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"

9) "flags"

10) "sentinel"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "241"

19) "last-ping-reply"

20) "241"

21) "down-after-milliseconds"

22) "5000"

23) "last-hello-message"

24) "425"

25) "voted-leader"

26) "?"

27) "voted-leader-epoch"

28) "0"

redis2


redis1@redis1:~$ redis-cli -h 192.168.2.166 -p 5000
192.168.2.166:5000> sentinel master mymaster
 1) "name"
 2) "mymaster"
 3) "ip"
 4) "192.168.2.104"
 5) "port"
 6) "6379"
 7) "runid"
 8) "4097e9a96cdb74dd4239aede54843e33104e6913"
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "979"
19) "last-ping-reply"
20) "979"
21) "down-after-milliseconds"
22) "5000"
23) "info-refresh"
24) "8165"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "188616"
29) "config-epoch"
30) "0"
31) "num-slaves"
32) "0"
33) "num-other-sentinels"
34) "2"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "60000"
39) "parallel-syncs"
40) "1"
192.168.2.166:5000> SENTINEL sentinels mymaster
1)  1) "name"
    2) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"
    3) "ip"
    4) "192.168.2.139"
    5) "port"
    6) "5000"
    7) "runid"
    8) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"
    9) "flags"
   10) "sentinel"
   11) "link-pending-commands"
   12) "0"
   13) "link-refcount"
   14) "1"
   15) "last-ping-sent"
   16) "0"
   17) "last-ok-ping-reply"
   18) "855"
   19) "last-ping-reply"
   20) "855"
   21) "down-after-milliseconds"
   22) "5000"
   23) "last-hello-message"
   24) "166"
   25) "voted-leader"
   26) "?"
   27) "voted-leader-epoch"
   28) "0"
2)  1) "name"
    2) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"
    3) "ip"
    4) "192.168.2.104"
    5) "port"
    6) "5000"
    7) "runid"
    8) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"
    9) "flags"
   10) "sentinel"
   11) "link-pending-commands"
   12) "0"
   13) "link-refcount"
   14) "1"
   15) "last-ping-sent"
   16) "0"
   17) "last-ok-ping-reply"
   18) "855"
   19) "last-ping-reply"
   20) "855"
   21) "down-after-milliseconds"
   22) "5000"
   23) "last-hello-message"
   24) "64"
   25) "voted-leader"
   26) "?"
   27) "voted-leader-epoch"
   28) "0"
192.168.2.166:5000>

100

redis1@redis1:~$ redis-cli -h 192.168.2.166 -p 5000

192.168.2.166:5000> sentinel master mymaster

1) "name"

2) "mymaster"

3) "ip"

4) "192.168.2.104"

5) "port"

6) "6379"

7) "runid"

8) "4097e9a96cdb74dd4239aede54843e33104e6913"

9) "flags"

10) "master"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "979"

19) "last-ping-reply"

20) "979"

21) "down-after-milliseconds"

22) "5000"

23) "info-refresh"

24) "8165"

25) "role-reported"

26) "master"

27) "role-reported-time"

28) "188616"

29) "config-epoch"

30) "0"

31) "num-slaves"

32) "0"

33) "num-other-sentinels"

34) "2"

35) "quorum"

36) "2"

37) "failover-timeout"

38) "60000"

39) "parallel-syncs"

40) "1"

192.168.2.166:5000> SENTINEL sentinels mymaster

1) 1) "name"

2) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"

3) "ip"

4) "192.168.2.139"

5) "port"

6) "5000"

7) "runid"

8) "095fc33d13e733d7f974faf0c2b2e6aba015fc3f"

9) "flags"

10) "sentinel"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "855"

19) "last-ping-reply"

20) "855"

21) "down-after-milliseconds"

22) "5000"

23) "last-hello-message"

24) "166"

25) "voted-leader"

26) "?"

27) "voted-leader-epoch"

28) "0"

2) 1) "name"

2) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"

3) "ip"

4) "192.168.2.104"

5) "port"

6) "5000"

7) "runid"

8) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"

9) "flags"

10) "sentinel"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "855"

19) "last-ping-reply"

20) "855"

21) "down-after-milliseconds"

22) "5000"

23) "last-hello-message"

24) "64"

25) "voted-leader"

26) "?"

27) "voted-leader-epoch"

28) "0"

192.168.2.166:5000>

redis3


redis1@redis1:~$ redis-cli -h 192.168.2.139 -p 5000
192.168.2.139:5000> sentinel master mymaster
 1) "name"
 2) "mymaster"
 3) "ip"
 4) "192.168.2.104"
 5) "port"
 6) "6379"
 7) "runid"
 8) "4097e9a96cdb74dd4239aede54843e33104e6913"
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "571"
19) "last-ping-reply"
20) "571"
21) "down-after-milliseconds"
22) "5000"
23) "info-refresh"
24) "8363"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "239355"
29) "config-epoch"
30) "0"
31) "num-slaves"
32) "0"
33) "num-other-sentinels"
34) "2"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "60000"
39) "parallel-syncs"
40) "1"
192.168.2.139:5000> SENTINEL sentinels mymaster
1)  1) "name"
    2) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"
    3) "ip"
    4) "192.168.2.104"
    5) "port"
    6) "5000"
    7) "runid"
    8) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"
    9) "flags"
   10) "sentinel"
   11) "link-pending-commands"
   12) "0"
   13) "link-refcount"
   14) "1"
   15) "last-ping-sent"
   16) "0"
   17) "last-ok-ping-reply"
   18) "232"
   19) "last-ping-reply"
   20) "232"
   21) "down-after-milliseconds"
   22) "5000"
   23) "last-hello-message"
   24) "753"
   25) "voted-leader"
   26) "?"
   27) "voted-leader-epoch"
   28) "0"
2)  1) "name"
    2) "ea6cbec52f9c04c6c325414a05562a69b503129c"
    3) "ip"
    4) "192.168.2.166"
    5) "port"
    6) "5000"
    7) "runid"
    8) "ea6cbec52f9c04c6c325414a05562a69b503129c"
    9) "flags"
   10) "sentinel"
   11) "link-pending-commands"
   12) "0"
   13) "link-refcount"
   14) "1"
   15) "last-ping-sent"
   16) "0"
   17) "last-ok-ping-reply"
   18) "232"
   19) "last-ping-reply"
   20) "232"
   21) "down-after-milliseconds"
   22) "5000"
   23) "last-hello-message"
   24) "1740"
   25) "voted-leader"
   26) "?"
   27) "voted-leader-epoch"
   28) "0"

redis1@redis1:~$ redis-cli -h 192.168.2.139 -p 5000

192.168.2.139:5000> sentinel master mymaster

1) "name"

2) "mymaster"

3) "ip"

4) "192.168.2.104"

5) "port"

6) "6379"

7) "runid"

8) "4097e9a96cdb74dd4239aede54843e33104e6913"

9) "flags"

10) "master"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "571"

19) "last-ping-reply"

20) "571"

21) "down-after-milliseconds"

22) "5000"

23) "info-refresh"

24) "8363"

25) "role-reported"

26) "master"

27) "role-reported-time"

28) "239355"

29) "config-epoch"

30) "0"

31) "num-slaves"

32) "0"

33) "num-other-sentinels"

34) "2"

35) "quorum"

36) "2"

37) "failover-timeout"

38) "60000"

39) "parallel-syncs"

40) "1"

192.168.2.139:5000> SENTINEL sentinels mymaster

1) 1) "name"

2) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"

3) "ip"

4) "192.168.2.104"

5) "port"

6) "5000"

7) "runid"

8) "9a7a4e3269530706fd1002c3018a3f0c525c25d6"

9) "flags"

10) "sentinel"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "232"

19) "last-ping-reply"

20) "232"

21) "down-after-milliseconds"

22) "5000"

23) "last-hello-message"

24) "753"

25) "voted-leader"

26) "?"

27) "voted-leader-epoch"

28) "0"

2) 1) "name"

2) "ea6cbec52f9c04c6c325414a05562a69b503129c"

3) "ip"

4) "192.168.2.166"

5) "port"

6) "5000"

7) "runid"

8) "ea6cbec52f9c04c6c325414a05562a69b503129c"

9) "flags"

10) "sentinel"

11) "link-pending-commands"

12) "0"

13) "link-refcount"

14) "1"

15) "last-ping-sent"

16) "0"

17) "last-ok-ping-reply"

18) "232"

19) "last-ping-reply"

20) "232"

21) "down-after-milliseconds"

22) "5000"

23) "last-hello-message"

24) "1740"

25) "voted-leader"

26) "?"

27) "voted-leader-epoch"

28) "0"

3.测试故障迁移

模拟主节点挂掉30s或者直接关掉服务,可以看到自动选主，并且修改slave的指向(redis2日志)

见加粗部分


sudo service redis-server stop
redis-cli -p 6379 DEBUG sleep 30

sudo service redis-server stop

redis-cli -p 6379 DEBUG sleep 30


redis1@redis1:~$ redis-sentinel sentinel.conf
3450:X 27 Nov 2022 12:23:13.821 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3450:X 27 Nov 2022 12:23:13.821 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=3450, just started
3450:X 27 Nov 2022 12:23:13.821 # Configuration loaded
3450:X 27 Nov 2022 12:23:13.822 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3450:X 27 Nov 2022 12:23:13.823 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 7.0.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 5000
 |    `-._   `._    /     _.-'    |     PID: 3450
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

3450:X 27 Nov 2022 12:23:13.826 # Sentinel ID is 9a7a4e3269530706fd1002c3018a3f0c525c25d6
3450:X 27 Nov 2022 12:23:13.826 # +monitor master mymaster 192.168.2.104 6379 quorum 2
3450:X 27 Nov 2022 12:23:13.829 * +slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:23:13.832 * Sentinel new configuration saved on disk
3450:X 27 Nov 2022 12:23:13.832 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:23:13.836 * Sentinel new configuration saved on disk
3450:X 27 Nov 2022 12:23:18.844 # +sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:23:18.844 # +sdown sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:23:25.263 # -sdown sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:23:37.694 # -sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:26:23.114 # +sdown master mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:26:23.191 # +odown master mymaster 192.168.2.104 6379 #quorum 3/2
3450:X 27 Nov 2022 12:26:23.192 # +new-epoch 4
<strong>3450:X 27 Nov 2022 12:26:23.192 # +try-failover master mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:26:23.197 * Sentinel new configuration saved on disk
3450:X 27 Nov 2022 12:26:23.197 # +vote-for-leader 9a7a4e3269530706fd1002c3018a3f0c525c25d6 4
3450:X 27 Nov 2022 12:26:23.200 # ea6cbec52f9c04c6c325414a05562a69b503129c voted for ea6cbec52f9c04c6c325414a05562a69b503129c 4
3450:X 27 Nov 2022 12:26:23.204 # 095fc33d13e733d7f974faf0c2b2e6aba015fc3f voted for ea6cbec52f9c04c6c325414a05562a69b503129c 4
3450:X 27 Nov 2022 12:26:24.346 # +config-update-from sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379
3450:X 27 Nov 2022 12:26:24.346 # +switch-master mymaster 192.168.2.104 6379 192.168.2.139 6379
3450:X 27 Nov 2022 12:26:24.348 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.139 6379
3450:X 27 Nov 2022 12:26:24.349 * +slave slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379
3450:X 27 Nov 2022 12:26:24.357 * Sentinel new configuration saved on disk
3450:X 27 Nov 2022 12:26:29.371 # +sdown slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379</strong>

redis1@redis1:~$ redis-sentinel sentinel.conf

3450:X 27 Nov 2022 12:23:13.821 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3450:X 27 Nov 2022 12:23:13.821 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=3450, just started

3450:X 27 Nov 2022 12:23:13.821 # Configuration loaded

3450:X 27 Nov 2022 12:23:13.822 * Increased maximum number of open files to 10032 (it was originally set to 1024).

3450:X 27 Nov 2022 12:23:13.823 * monotonic clock: POSIX clock_gettime

_._

_.-``__ ''-._

_.-`` `. `_. ''-._ Redis 7.0.5 (00000000/0) 64 bit

.-`` .-```. ```\/ _.,_ ''-._

( ' , .-` | `, ) Running in sentinel mode

|`-._`-...-` __...-.``-._|'` _.-'| Port: 5000

| `-._ `._ / _.-' | PID: 3450

`-._ `-._ `-./ _.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' | https://redis.io

`-._ `-._`-.__.-'_.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' |

`-._ `-._`-.__.-'_.-' _.-'

`-._ `-.__.-' _.-'

`-._ _.-'

`-.__.-'

3450:X 27 Nov 2022 12:23:13.826 # Sentinel ID is 9a7a4e3269530706fd1002c3018a3f0c525c25d6

3450:X 27 Nov 2022 12:23:13.826 # +monitor master mymaster 192.168.2.104 6379 quorum 2

3450:X 27 Nov 2022 12:23:13.829 * +slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:23:13.832 * Sentinel new configuration saved on disk

3450:X 27 Nov 2022 12:23:13.832 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:23:13.836 * Sentinel new configuration saved on disk

3450:X 27 Nov 2022 12:23:18.844 # +sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:23:18.844 # +sdown sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:23:25.263 # -sdown sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:23:37.694 # -sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:26:23.114 # +sdown master mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:26:23.191 # +odown master mymaster 192.168.2.104 6379 #quorum 3/2

3450:X 27 Nov 2022 12:26:23.192 # +new-epoch 4

<strong>3450:X 27 Nov 2022 12:26:23.192 # +try-failover master mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:26:23.197 * Sentinel new configuration saved on disk

3450:X 27 Nov 2022 12:26:23.197 # +vote-for-leader 9a7a4e3269530706fd1002c3018a3f0c525c25d6 4

3450:X 27 Nov 2022 12:26:23.200 # ea6cbec52f9c04c6c325414a05562a69b503129c voted for ea6cbec52f9c04c6c325414a05562a69b503129c 4

3450:X 27 Nov 2022 12:26:23.204 # 095fc33d13e733d7f974faf0c2b2e6aba015fc3f voted for ea6cbec52f9c04c6c325414a05562a69b503129c 4

3450:X 27 Nov 2022 12:26:24.346 # +config-update-from sentinel ea6cbec52f9c04c6c325414a05562a69b503129c 192.168.2.166 5000 @ mymaster 192.168.2.104 6379

3450:X 27 Nov 2022 12:26:24.346 # +switch-master mymaster 192.168.2.104 6379 192.168.2.139 6379

3450:X 27 Nov 2022 12:26:24.348 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.139 6379

3450:X 27 Nov 2022 12:26:24.349 * +slave slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379

3450:X 27 Nov 2022 12:26:24.357 * Sentinel new configuration saved on disk

3450:X 27 Nov 2022 12:26:29.371 # +sdown slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379</strong>


redis2@redis2:~$ redis-sentinel sentinel.conf
3098:X 27 Nov 2022 12:23:24.963 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3098:X 27 Nov 2022 12:23:24.963 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=3098, just started
3098:X 27 Nov 2022 12:23:24.963 # Configuration loaded
3098:X 27 Nov 2022 12:23:24.965 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3098:X 27 Nov 2022 12:23:24.965 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 7.0.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 5000
 |    `-._   `._    /     _.-'    |     PID: 3098
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

3098:X 27 Nov 2022 12:23:24.968 # Sentinel ID is ea6cbec52f9c04c6c325414a05562a69b503129c
3098:X 27 Nov 2022 12:23:24.968 # +monitor master mymaster 192.168.2.104 6379 quorum 2
3098:X 27 Nov 2022 12:23:24.972 * +slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:23:24.975 * Sentinel new configuration saved on disk
3098:X 27 Nov 2022 12:23:24.975 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:23:24.979 * Sentinel new configuration saved on disk
3098:X 27 Nov 2022 12:23:29.998 # +sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:23:37.449 # -sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.136 # +sdown master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.237 # +odown master mymaster 192.168.2.104 6379 #quorum 2/2
3098:X 27 Nov 2022 12:26:23.237 # +new-epoch 4
3098:X 27 Nov 2022 12:26:23.237 # +try-failover master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.242 * Sentinel new configuration saved on disk
3098:X 27 Nov 2022 12:26:23.242 # +vote-for-leader ea6cbec52f9c04c6c325414a05562a69b503129c 4
3098:X 27 Nov 2022 12:26:23.248 # 9a7a4e3269530706fd1002c3018a3f0c525c25d6 voted for 9a7a4e3269530706fd1002c3018a3f0c525c25d6 4
3098:X 27 Nov 2022 12:26:23.253 # 095fc33d13e733d7f974faf0c2b2e6aba015fc3f voted for ea6cbec52f9c04c6c325414a05562a69b503129c 4
3098:X 27 Nov 2022 12:26:23.333 # +elected-leader master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.333 # +failover-state-select-slave master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.417 # +selected-slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.417 * +failover-state-send-slaveof-noone slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:23.483 * +failover-state-wait-promotion slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:24.307 * Sentinel new configuration saved on disk
<strong>3098:X 27 Nov 2022 12:26:24.307 # +promoted-slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:24.307 # +failover-state-reconf-slaves master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:24.394 * +slave-reconf-sent slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:25.322 * +slave-reconf-inprog slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:25.323 * +slave-reconf-done slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:25.399 # -odown master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:25.399 # +failover-end master mymaster 192.168.2.104 6379
3098:X 27 Nov 2022 12:26:25.399 # +switch-master mymaster 192.168.2.104 6379 192.168.2.139 6379
3098:X 27 Nov 2022 12:26:25.400 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.139 6379
3098:X 27 Nov 2022 12:26:25.400 * +slave slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379
3098:X 27 Nov 2022 12:26:25.403 * Sentinel new configuration saved on disk
3098:X 27 Nov 2022 12:26:30.447 # +sdown slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379
3098:X 27 Nov 2022 12:29:48.267 # -sdown slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379
3098:X 27 Nov 2022 12:29:58.242 * +convert-to-slave slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379</strong>

redis2@redis2:~$ redis-sentinel sentinel.conf

3098:X 27 Nov 2022 12:23:24.963 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3098:X 27 Nov 2022 12:23:24.963 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=3098, just started

3098:X 27 Nov 2022 12:23:24.963 # Configuration loaded

3098:X 27 Nov 2022 12:23:24.965 * Increased maximum number of open files to 10032 (it was originally set to 1024).

3098:X 27 Nov 2022 12:23:24.965 * monotonic clock: POSIX clock_gettime

_._

_.-``__ ''-._

_.-`` `. `_. ''-._ Redis 7.0.5 (00000000/0) 64 bit

.-`` .-```. ```\/ _.,_ ''-._

( ' , .-` | `, ) Running in sentinel mode

|`-._`-...-` __...-.``-._|'` _.-'| Port: 5000

| `-._ `._ / _.-' | PID: 3098

`-._ `-._ `-./ _.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' | https://redis.io

`-._ `-._`-.__.-'_.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' |

`-._ `-._`-.__.-'_.-' _.-'

`-._ `-.__.-' _.-'

`-._ _.-'

`-.__.-'

3098:X 27 Nov 2022 12:23:24.968 # Sentinel ID is ea6cbec52f9c04c6c325414a05562a69b503129c

3098:X 27 Nov 2022 12:23:24.968 # +monitor master mymaster 192.168.2.104 6379 quorum 2

3098:X 27 Nov 2022 12:23:24.972 * +slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:23:24.975 * Sentinel new configuration saved on disk

3098:X 27 Nov 2022 12:23:24.975 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:23:24.979 * Sentinel new configuration saved on disk

3098:X 27 Nov 2022 12:23:29.998 # +sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:23:37.449 # -sdown sentinel 095fc33d13e733d7f974faf0c2b2e6aba015fc3f 192.168.2.139 5000 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.136 # +sdown master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.237 # +odown master mymaster 192.168.2.104 6379 #quorum 2/2

3098:X 27 Nov 2022 12:26:23.237 # +new-epoch 4

3098:X 27 Nov 2022 12:26:23.237 # +try-failover master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.242 * Sentinel new configuration saved on disk

3098:X 27 Nov 2022 12:26:23.242 # +vote-for-leader ea6cbec52f9c04c6c325414a05562a69b503129c 4

3098:X 27 Nov 2022 12:26:23.248 # 9a7a4e3269530706fd1002c3018a3f0c525c25d6 voted for 9a7a4e3269530706fd1002c3018a3f0c525c25d6 4

3098:X 27 Nov 2022 12:26:23.253 # 095fc33d13e733d7f974faf0c2b2e6aba015fc3f voted for ea6cbec52f9c04c6c325414a05562a69b503129c 4

3098:X 27 Nov 2022 12:26:23.333 # +elected-leader master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.333 # +failover-state-select-slave master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.417 # +selected-slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.417 * +failover-state-send-slaveof-noone slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:23.483 * +failover-state-wait-promotion slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:24.307 * Sentinel new configuration saved on disk

<strong>3098:X 27 Nov 2022 12:26:24.307 # +promoted-slave slave 192.168.2.139:6379 192.168.2.139 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:24.307 # +failover-state-reconf-slaves master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:24.394 * +slave-reconf-sent slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:25.322 * +slave-reconf-inprog slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:25.323 * +slave-reconf-done slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:25.399 # -odown master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:25.399 # +failover-end master mymaster 192.168.2.104 6379

3098:X 27 Nov 2022 12:26:25.399 # +switch-master mymaster 192.168.2.104 6379 192.168.2.139 6379

3098:X 27 Nov 2022 12:26:25.400 * +slave slave 192.168.2.166:6379 192.168.2.166 6379 @ mymaster 192.168.2.139 6379

3098:X 27 Nov 2022 12:26:25.400 * +slave slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379

3098:X 27 Nov 2022 12:26:25.403 * Sentinel new configuration saved on disk

3098:X 27 Nov 2022 12:26:30.447 # +sdown slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379

3098:X 27 Nov 2022 12:29:48.267 # -sdown slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379

3098:X 27 Nov 2022 12:29:58.242 * +convert-to-slave slave 192.168.2.104:6379 192.168.2.104 6379 @ mymaster 192.168.2.139 6379</strong>

这时候再看master已经变成了redis3


redis1@redis1:~$ redis-cli -p 5000
127.0.0.1:5000> SENTINEL get-master-addr-by-name mymaster
1) "192.168.2.139"
2) "6379"

redis1@redis1:~$ redis-cli -p 5000

127.0.0.1:5000> SENTINEL get-master-addr-by-name mymaster

1) "192.168.2.139"

2) "6379"

去redis3改点数据，去redis2看变不变


redis1@redis1:~$ redis-cli -h 192.168.2.139 -p 6379
192.168.2.139:6379> get test
"1"
192.168.2.139:6379> set test 2
OK
192.168.2.139:6379> get test
"2"
192.168.2.139:6379>
redis1@redis1:~$ redis-cli -h 192.168.2.166 -p 6379
192.168.2.166:6379> get test
"2"
192.168.2.166:6379>

redis1@redis1:~$ redis-cli -h 192.168.2.139 -p 6379

192.168.2.139:6379> get test

"1"

192.168.2.139:6379> set test 2

192.168.2.139:6379> get test

"2"

192.168.2.139:6379>

redis1@redis1:~$ redis-cli -h 192.168.2.166 -p 6379

192.168.2.166:6379> get test

"2"

192.168.2.166:6379>

显然redis3改的值可以从redis2读到，说明复制正常，继续尝试恢复redis1看看能不能读到test=2


redis1@redis1:~$ redis-cli -p 6379
127.0.0.1:6379> get test
"2"
127.0.0.1:6379>

redis1@redis1:~$ redis-cli -p 6379

127.0.0.1:6379> get test

"2"

127.0.0.1:6379>

也可以读到，高可用集群搭建完成

附用到的命令


https://mirrors.tuna.tsinghua.edu.cn

redis1
192.168.2.104
ssh redis1@192.168.2.104

redis2
192.168.2.166
ssh redis2@192.168.2.166

redis3
192.168.2.139
ssh redis3@192.168.2.139

6379

sudo vim /etc/redis/redis.conf
#config all nodes
bind * -::1
protected-mode no

#only in redis2 and redis3
replicaof 192.168.2.104 6379

sudo service redis-server restart


vim sentinel.conf
port 5000 
sentinel monitor mymaster 192.168.2.104 6379 2 
sentinel down-after-milliseconds mymaster 5000 
sentinel failover-timeout mymaster 60000 
sentinel parallel-syncs mymaster 1

redis-sentinel sentinel.conf
redis-cli -p 5000
redis-cli -h 192.168.2.166 -p 5000
redis-cli -h 192.168.2.139 -p 5000

SENTINEL master mymaster 
SENTINEL get-master-addr-by-name mymaster 
SENTINEL replicas mymaster 
SENTINEL sentinels mymaster

redis-cli -p 6379
redis-cli -h 192.168.2.166 -p 6379
redis-cli -h 192.168.2.139 -p 6379

https://mirrors.tuna.tsinghua.edu.cn

redis1

192.168.2.104

ssh redis1@192.168.2.104

redis2

192.168.2.166

ssh redis2@192.168.2.166

redis3

192.168.2.139

ssh redis3@192.168.2.139

6379

sudo vim /etc/redis/redis.conf

#config all nodes

bind * -::1

protected-mode no

#only in redis2 and redis3

replicaof 192.168.2.104 6379

sudo service redis-server restart

vim sentinel.conf

port 5000

sentinel monitor mymaster 192.168.2.104 6379 2

sentinel down-after-milliseconds mymaster 5000

sentinel failover-timeout mymaster 60000

sentinel parallel-syncs mymaster 1

redis-sentinel sentinel.conf

redis-cli -p 5000

redis-cli -h 192.168.2.166 -p 5000

redis-cli -h 192.168.2.139 -p 5000

SENTINEL master mymaster

SENTINEL get-master-addr-by-name mymaster

SENTINEL replicas mymaster

SENTINEL sentinels mymaster

redis-cli -p 6379

redis-cli -h 192.168.2.166 -p 6379

redis-cli -h 192.168.2.139 -p 6379

参考

https://redis.io/docs/

https://redis.io/docs/getting-started/installation/install-redis-on-linux/

https://redis.io/docs/management/replication/

https://redis.io/docs/management/sentinel/

视频版

https://www.bilibili.com/video/BV1H24y117uh/

1.安装mysql-apt-config并准备基础环境

channel选mysql8, apt update后就能安装 mysql-shell了

获取debian-sys-maint的密码备用

使用/etc/hosts尝试模拟域名或者自行配置DNS

mysqld下的配置增加report_host=mysql1.db,


#/etc/mysql/mysql.conf.d/
report_host=mysql1.db 
#/etc/hosts 192.168.2.190 mysql1.db

#/etc/mysql/mysql.conf.d/

report_host=mysql1.db

#/etc/hosts 192.168.2.190 mysql1.db

2.创建账户

2.1.一个InnoDB Cluster server configuration账户用于配置集群的实例

使用dba.configureInstance()创建

2.2.一个或多个InnoDB Cluster administrator账户可以在集群安装后被用来管理服务器实例

在所有节点加入集群后，使用cluster.setupAdminAccount(),这个命令会创建指定的账号并赋予所需的权限，创建这个账号的事务会写入binlog并发送给集群的其他服务器

2.3一个或多个MySQL Router账户可以被MySQL Router用来连接集群

以上每个账户必须在所有InnoDB Cluster节点同时存在, 并使用相同的用户名和相同的密码

mysqlsh进入终端，在三个机器重复这个过程


mysqlsh>
dba.configureInstance('debian-sys-maint@localhost:3306',{clusterAdmin:"'icadmin'@'localhost'"})
dba.configureInstance('debian-sys-maint@localhost:3306',{clusterAdmin:"'icadmin'@'192.168.2.%'"})

mysqlsh>

dba.configureInstance('debian-sys-maint@localhost:3306',{clusterAdmin:"'icadmin'@'localhost'"})

dba.configureInstance('debian-sys-maint@localhost:3306',{clusterAdmin:"'icadmin'@'192.168.2.%'"})


 MySQL  JS > dba.configureInstance('root@localhost:3306',{clusterAdmin:"'icadmin'@'localhost'"})
Configuring local MySQL instance listening at port 3306 for use in an InnoDB cluster...

This instance reports its own address as mysql1.db:3306
Password for new account: ********
Confirm password: ********

applierWorkerThreads will be set to the default value of 4.

NOTE: Some configuration options need to be fixed:
+----------------------------------------+---------------+----------------+--------------------------------------------------+
| Variable                               | Current Value | Required Value | Note
                 |
+----------------------------------------+---------------+----------------+--------------------------------------------------+
| binlog_transaction_dependency_tracking | COMMIT_ORDER  | WRITESET       | Update the server variable                       |
| enforce_gtid_consistency               | OFF           | ON             | Update read-only variable and restart the server |
| gtid_mode                              | OFF           | ON             | Update read-only variable and restart the server |
| server_id                              | 1             | <unique ID>    | Update read-only variable and restart the server |
+----------------------------------------+---------------+----------------+--------------------------------------------------+

Some variables need to be changed, but cannot be done dynamically on the server.
Do you want to perform the required configuration changes? [y/n]: y
Do you want to restart the instance after configuring it? [y/n]: y

Cluster admin user 'icadmin'@'localhost' created.
Configuring instance...
The instance 'mysql1.db:3306' was configured to be used in an InnoDB cluster.
Restarting MySQL...
NOTE: MySQL server at mysql1.db:3306 was restarted.

MySQL JS > dba.configureInstance('root@localhost:3306',{clusterAdmin:"'icadmin'@'localhost'"})

Configuring local MySQL instance listening at port 3306 for use in an InnoDB cluster...

This instance reports its own address as mysql1.db:3306

Password for new account: ********

Confirm password: ********

applierWorkerThreads will be set to the default value of 4.

NOTE: Some configuration options need to be fixed:

+----------------------------------------+---------------+----------------+--------------------------------------------------+

| Variable | Current Value | Required Value | Note

+----------------------------------------+---------------+----------------+--------------------------------------------------+

| enforce_gtid_consistency | OFF | ON | Update read-only variable and restart the server |

| gtid_mode | OFF | ON | Update read-only variable and restart the server |

+----------------------------------------+---------------+----------------+--------------------------------------------------+

Some variables need to be changed, but cannot be done dynamically on the server.

Do you want to perform the required configuration changes? [y/n]: y

Do you want to restart the instance after configuring it? [y/n]: y

Cluster admin user 'icadmin'@'localhost' created.

Configuring instance...

The instance 'mysql1.db:3306' was configured to be used in an InnoDB cluster.

Restarting MySQL...

NOTE: MySQL server at mysql1.db:3306 was restarted.

3.检查机器

使用dba.checkInstanceConfiguration(instance) 可以检查节点是否满足 Section 7.1, “InnoDB Cluster Requirements” 提到的前置条件，详见 Checking Instance State

使用dba.configureInstance() 来自动配置节点

连接服务器所用的账户必须有这里要求的权限 Configuring InnoDB Cluster Administrator Accounts Manually

三个机器分别检查自己本地和对方


mysqlsh>
dba.checkInstanceConfiguration('icadmin@localhost:3306')
dba.checkInstanceConfiguration('icadmin@mysql1.db:3306')
dba.checkInstanceConfiguration('icadmin@mysql2.db:3306')
dba.checkInstanceConfiguration('icadmin@mysql3.db:3306')
换官方源安装mysql就不报错了，ubuntu的源会有错。。。。

mysqlsh>

dba.checkInstanceConfiguration('icadmin@localhost:3306')

dba.checkInstanceConfiguration('icadmin@mysql1.db:3306')

dba.checkInstanceConfiguration('icadmin@mysql2.db:3306')

dba.checkInstanceConfiguration('icadmin@mysql3.db:3306')

换官方源安装mysql就不报错了，ubuntu的源会有错。。。。

成功会输出这样的


 MySQL  JS > dba.checkInstanceConfiguration('icadmin@mysql3.db:3306')
Please provide the password for 'icadmin@mysql3.db:3306': ********
Save password for 'icadmin@mysql3.db:3306'? [Y]es/[N]o/Ne[v]er (default No): Y
Validating MySQL instance at mysql3.db:3306 for use in an InnoDB cluster...

This instance reports its own address as mysql3.db:3306

Checking whether existing tables comply with Group Replication requirements...
No incompatible tables detected

Checking instance configuration...
Instance configuration is compatible with InnoDB cluster

The instance 'mysql3.db:3306' is valid to be used in an InnoDB cluster.

{
    "status": "ok"
}

MySQL JS > dba.checkInstanceConfiguration('icadmin@mysql3.db:3306')

Please provide the password for 'icadmin@mysql3.db:3306': ********

Save password for 'icadmin@mysql3.db:3306'? [Y]es/[N]o/Ne[v]er (default No): Y

Validating MySQL instance at mysql3.db:3306 for use in an InnoDB cluster...

This instance reports its own address as mysql3.db:3306

Checking whether existing tables comply with Group Replication requirements...

No incompatible tables detected

Checking instance configuration...

Instance configuration is compatible with InnoDB cluster

The instance 'mysql3.db:3306' is valid to be used in an InnoDB cluster.

{

"status": "ok"

}

4.创建集群并添加节点

使用\connect 连接主节点


MySQL  JS > \connect icadmin@mysql1.db:3306
Creating a session to 'icadmin@mysql1.db:3306'
Fetching schema names for auto-completion... Press ^C to stop.
Your MySQL connection id is 18
Server version: 8.0.31 MySQL Community Server - GPL
No default schema selected; type \use <schema> to set one.
MySQL  mysql1.db:3306 ssl  JS >

MySQL JS > \connect icadmin@mysql1.db:3306

Creating a session to 'icadmin@mysql1.db:3306'

Fetching schema names for auto-completion... Press ^C to stop.

Your MySQL connection id is 18

Server version: 8.0.31 MySQL Community Server - GPL

No default schema selected; type \use <schema> to set one.

MySQL mysql1.db:3306 ssl JS >

使用dba.createCluster() 使用当前 MySQL Shell 连接的节点作为集群的种子


 MySQL  mysql1.db:3306 ssl  JS > dba.createCluster('testCluster', {replicationAllowedHost:'192.168.2.0/24'})

A new InnoDB Cluster will be created on instance 'mysql1.db:3306'.

Validating instance configuration at mysql1.db:3306...

This instance reports its own address as mysql1.db:3306

Instance configuration is suitable.
NOTE: Group Replication will communicate with other members using 'mysql1.db:3306'. Use the localAddress option to override.

Creating InnoDB Cluster 'testCluster' on 'mysql1.db:3306'...

Adding Seed Instance...
Cluster successfully created. Use Cluster.addInstance() to add MySQL instances.
At least 3 instances are needed for the cluster to be able to withstand up to
one server failure.

<Cluster:testCluster>

MySQL mysql1.db:3306 ssl JS > dba.createCluster('testCluster', {replicationAllowedHost:'192.168.2.0/24'})

A new InnoDB Cluster will be created on instance 'mysql1.db:3306'.

Validating instance configuration at mysql1.db:3306...

This instance reports its own address as mysql1.db:3306

Instance configuration is suitable.

NOTE: Group Replication will communicate with other members using 'mysql1.db:3306'. Use the localAddress option to override.

Creating InnoDB Cluster 'testCluster' on 'mysql1.db:3306'...

Adding Seed Instance...

Cluster successfully created. Use Cluster.addInstance() to add MySQL instances.

At least 3 instances are needed for the cluster to be able to withstand up to

one server failure.

<Cluster:testCluster>

使用setupAdminAccount广播管理账号,生产环境的话要注意下安全，这里测试没限定网段


 MySQL  mysql1.db:3306 ssl  JS > var cluster = dba.getCluster()
 MySQL  mysql1.db:3306 ssl  JS > cluster
<Cluster:testCluster>
 MySQL  mysql1.db:3306 ssl  JS > cluster.setupAdminAccount("icadmin")

Missing the password for new account icadmin@%. Please provide one.
Password for new account: ********
Confirm password: ********

Creating user icadmin@%.
Account icadmin@% was successfully created.

 MySQL  mysql1.db:3306 ssl  JS >

MySQL mysql1.db:3306 ssl JS > var cluster = dba.getCluster()

MySQL mysql1.db:3306 ssl JS > cluster

<Cluster:testCluster>

MySQL mysql1.db:3306 ssl JS > cluster.setupAdminAccount("icadmin")

Missing the password for new account icadmin@%. Please provide one.

Password for new account: ********

Confirm password: ********

Creating user icadmin@%.

Account icadmin@% was successfully created.

MySQL mysql1.db:3306 ssl JS >

使用cluster.addInstance()向当前集群内增加节点,C选择完全量复制，I选择增量复制，默认是全量


MySQL  mysql1.db:3306 ssl  JS > cluster.addInstance('icadmin@mysql2.db:3306')

NOTE: The target instance 'mysql2.db:3306' has not been pre-provisioned (GTID set is empty). The Shell is unable to decide whether incremental state recovery can correctly provision it.
The safest and most convenient way to provision a new instance is through automatic clone provisioning, which will completely overwrite the state of 'mysql2.db:3306' with a physical snapshot from an existing cluster member. To use this method by default, set the 'recoveryMethod' option to 'clone'.

The incremental state recovery may be safely used if you are sure all updates ever executed in the cluster were done with GTIDs enabled, there are no purged transactions and the new instance contains the same GTID set as the cluster or a subset of it. To use this method by default, set the 'recoveryMethod' option to 'incremental'.


Please select a recovery method [C]lone/[I]ncremental recovery/[A]bort (default Clone): C
Validating instance configuration at mysql2.db:3306...

This instance reports its own address as mysql2.db:3306

Instance configuration is suitable.
NOTE: Group Replication will communicate with other members using 'mysql2.db:3306'. Use the localAddress option to override.

A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.

Adding instance to the cluster...

Monitoring recovery process of the new cluster member. Press ^C to stop monitoring and let it continue in background.
Clone based state recovery is now in progress.

NOTE: A server restart is expected to happen as part of the clone process. If the
server does not support the RESTART command or does not come back after a
while, you may need to manually start it back.

* Waiting for clone to finish...
NOTE: mysql2.db:3306 is being cloned from mysql1.db:3306
** Stage DROP DATA: Completed
** Clone Transfer
    FILE COPY  ############################################################  100%  Completed
    PAGE COPY  ############################################################  100%  Completed
    REDO COPY  ############################################################  100%  Completed

NOTE: mysql2.db:3306 is shutting down...

* Waiting for server restart... ready
* mysql2.db:3306 has restarted, waiting for clone to finish...
** Stage RESTART: Completed
* Clone process has finished: 73.65 MB transferred in 2 sec (36.83 MB/s)

State recovery already finished for 'mysql2.db:3306'

The instance 'mysql2.db:3306' was successfully added to the cluster.

MySQL mysql1.db:3306 ssl JS > cluster.addInstance('icadmin@mysql2.db:3306')

NOTE: The target instance 'mysql2.db:3306' has not been pre-provisioned (GTID set is empty). The Shell is unable to decide whether incremental state recovery can correctly provision it.

The safest and most convenient way to provision a new instance is through automatic clone provisioning, which will completely overwrite the state of 'mysql2.db:3306' with a physical snapshot from an existing cluster member. To use this method by default, set the 'recoveryMethod' option to 'clone'.

The incremental state recovery may be safely used if you are sure all updates ever executed in the cluster were done with GTIDs enabled, there are no purged transactions and the new instance contains the same GTID set as the cluster or a subset of it. To use this method by default, set the 'recoveryMethod' option to 'incremental'.

Please select a recovery method [C]lone/[I]ncremental recovery/[A]bort (default Clone): C

Validating instance configuration at mysql2.db:3306...

This instance reports its own address as mysql2.db:3306

Instance configuration is suitable.

NOTE: Group Replication will communicate with other members using 'mysql2.db:3306'. Use the localAddress option to override.

A new instance will be added to the InnoDB cluster. Depending on the amount of

data on the cluster this might take from a few seconds to several hours.

Adding instance to the cluster...

Monitoring recovery process of the new cluster member. Press ^C to stop monitoring and let it continue in background.

Clone based state recovery is now in progress.

NOTE: A server restart is expected to happen as part of the clone process. If the

server does not support the RESTART command or does not come back after a

while, you may need to manually start it back.

* Waiting for clone to finish...

NOTE: mysql2.db:3306 is being cloned from mysql1.db:3306

** Stage DROP DATA: Completed

** Clone Transfer

FILE COPY ############################################################ 100% Completed

PAGE COPY ############################################################ 100% Completed

REDO COPY ############################################################ 100% Completed

NOTE: mysql2.db:3306 is shutting down...

* Waiting for server restart... ready

* mysql2.db:3306 has restarted, waiting for clone to finish...

** Stage RESTART: Completed

* Clone process has finished: 73.65 MB transferred in 2 sec (36.83 MB/s)

State recovery already finished for 'mysql2.db:3306'

The instance 'mysql2.db:3306' was successfully added to the cluster.

使用cluster.status()查看当前集群状态


 MySQL  mysql1.db:3306 ssl  JS > cluster.status()
{
    "clusterName": "testCluster",
    "defaultReplicaSet": {
        "name": "default",
        "primary": "mysql1.db:3306",
        "ssl": "REQUIRED",
        "status": "OK_NO_TOLERANCE",
        "statusText": "Cluster is NOT tolerant to any failures.",
        "topology": {
            "mysql1.db:3306": {
                "address": "mysql1.db:3306",
                "memberRole": "PRIMARY",
                "mode": "R/W",
                "readReplicas": {},
                "replicationLag": "applier_queue_applied",
                "role": "HA",
                "status": "ONLINE",
                "version": "8.0.31"
            },
            "mysql2.db:3306": {
                "address": "mysql2.db:3306",
                "memberRole": "SECONDARY",
                "mode": "R/O",
                "readReplicas": {},
                "replicationLag": "applier_queue_applied",
                "role": "HA",
                "status": "ONLINE",
                "version": "8.0.31"
            }
        },
        "topologyMode": "Single-Primary"
    },
    "groupInformationSourceMember": "mysql1.db:3306"
}

MySQL mysql1.db:3306 ssl JS > cluster.status()

{

"clusterName": "testCluster",

"defaultReplicaSet": {

"name": "default",

"primary": "mysql1.db:3306",

"ssl": "REQUIRED",

"status": "OK_NO_TOLERANCE",

"statusText": "Cluster is NOT tolerant to any failures.",

"topology": {

"mysql1.db:3306": {

"address": "mysql1.db:3306",

"memberRole": "PRIMARY",

"mode": "R/W",

"readReplicas": {},

"replicationLag": "applier_queue_applied",

"role": "HA",

"status": "ONLINE",

"version": "8.0.31"

"mysql2.db:3306": {

"address": "mysql2.db:3306",

"memberRole": "SECONDARY",

"mode": "R/O",

"readReplicas": {},

"replicationLag": "applier_queue_applied",

"role": "HA",

"status": "ONLINE",

"version": "8.0.31"

}

"topologyMode": "Single-Primary"

"groupInformationSourceMember": "mysql1.db:3306"

}

重复这个过程，添加mysql3


 MySQL  mysql1.db:3306 ssl  JS > cluster.addInstance('icadmin@mysql3.db:3306')

NOTE: The target instance 'mysql3.db:3306' has not been pre-provisioned (GTID set is empty). The Shell is unable to decide whether incremental state recovery can correctly provision it.
The safest and most convenient way to provision a new instance is through automatic clone provisioning, which will completely overwrite the state of 'mysql3.db:3306' with a physical snapshot from an existing cluster member. To use this method by default, set the 'recoveryMethod' option to 'clone'.

The incremental state recovery may be safely used if you are sure all updates ever executed in the cluster were done with GTIDs enabled, there are no purged transactions and the new instance contains the same GTID set as the cluster or a subset of it. To use this method by default, set the 'recoveryMethod' option to 'incremental'.


Please select a recovery method [C]lone/[I]ncremental recovery/[A]bort (default Clone):
Validating instance configuration at mysql3.db:3306...

This instance reports its own address as mysql3.db:3306

Instance configuration is suitable.
NOTE: Group Replication will communicate with other members using 'mysql3.db:3306'. Use the localAddress option to override.

A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.

Adding instance to the cluster...

Monitoring recovery process of the new cluster member. Press ^C to stop monitoring and let it continue in background.
Clone based state recovery is now in progress.

NOTE: A server restart is expected to happen as part of the clone process. If the
server does not support the RESTART command or does not come back after a
while, you may need to manually start it back.

* Waiting for clone to finish...
NOTE: mysql3.db:3306 is being cloned from mysql1.db:3306
** Stage DROP DATA: Completed
** Clone Transfer
    FILE COPY  ############################################################  100%  Completed
    PAGE COPY  ############################################################  100%  Completed
    REDO COPY  ############################################################  100%  Completed

NOTE: mysql3.db:3306 is shutting down...

* Waiting for server restart... ready
* mysql3.db:3306 has restarted, waiting for clone to finish...
** Stage RESTART: Completed
* Clone process has finished: 73.65 MB transferred in 2 sec (36.83 MB/s)

State recovery already finished for 'mysql3.db:3306'

The instance 'mysql3.db:3306' was successfully added to the cluster.

MySQL mysql1.db:3306 ssl JS > cluster.addInstance('icadmin@mysql3.db:3306')

NOTE: The target instance 'mysql3.db:3306' has not been pre-provisioned (GTID set is empty). The Shell is unable to decide whether incremental state recovery can correctly provision it.

The safest and most convenient way to provision a new instance is through automatic clone provisioning, which will completely overwrite the state of 'mysql3.db:3306' with a physical snapshot from an existing cluster member. To use this method by default, set the 'recoveryMethod' option to 'clone'.

Please select a recovery method [C]lone/[I]ncremental recovery/[A]bort (default Clone):

Validating instance configuration at mysql3.db:3306...

This instance reports its own address as mysql3.db:3306

Instance configuration is suitable.

NOTE: Group Replication will communicate with other members using 'mysql3.db:3306'. Use the localAddress option to override.

A new instance will be added to the InnoDB cluster. Depending on the amount of

data on the cluster this might take from a few seconds to several hours.

Adding instance to the cluster...

Monitoring recovery process of the new cluster member. Press ^C to stop monitoring and let it continue in background.

Clone based state recovery is now in progress.

NOTE: A server restart is expected to happen as part of the clone process. If the

server does not support the RESTART command or does not come back after a

while, you may need to manually start it back.

* Waiting for clone to finish...

NOTE: mysql3.db:3306 is being cloned from mysql1.db:3306

** Stage DROP DATA: Completed

** Clone Transfer

FILE COPY ############################################################ 100% Completed

PAGE COPY ############################################################ 100% Completed

REDO COPY ############################################################ 100% Completed

NOTE: mysql3.db:3306 is shutting down...

* Waiting for server restart... ready

* mysql3.db:3306 has restarted, waiting for clone to finish...

** Stage RESTART: Completed

* Clone process has finished: 73.65 MB transferred in 2 sec (36.83 MB/s)

State recovery already finished for 'mysql3.db:3306'

The instance 'mysql3.db:3306' was successfully added to the cluster.

再次查看集群状态，可以看到mysql1是主，mysql2和3是从


 MySQL  mysql1.db:3306 ssl  JS > cluster.status()
{
    "clusterName": "testCluster",
    "defaultReplicaSet": {
        "name": "default",
        "primary": "mysql1.db:3306",
        "ssl": "REQUIRED",
        "status": "OK",
        "statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
        "topology": {
            "mysql1.db:3306": {
                "address": "mysql1.db:3306",
                "memberRole": "PRIMARY",
                "mode": "R/W",
                "readReplicas": {},
                "replicationLag": "applier_queue_applied",
                "role": "HA",
                "status": "ONLINE",
                "version": "8.0.31"
            },
            "mysql2.db:3306": {
                "address": "mysql2.db:3306",
                "memberRole": "SECONDARY",
                "mode": "R/O",
                "readReplicas": {},
                "replicationLag": "applier_queue_applied",
                "role": "HA",
                "status": "ONLINE",
                "version": "8.0.31"
            },
            "mysql3.db:3306": {
                "address": "mysql3.db:3306",
                "memberRole": "SECONDARY",
                "mode": "R/O",
                "readReplicas": {},
                "replicationLag": "applier_queue_applied",
                "role": "HA",
                "status": "ONLINE",
                "version": "8.0.31"
            }
        },
        "topologyMode": "Single-Primary"
    },
    "groupInformationSourceMember": "mysql1.db:3306"
}

MySQL mysql1.db:3306 ssl JS > cluster.status()

{

"clusterName": "testCluster",

"defaultReplicaSet": {

"name": "default",

"primary": "mysql1.db:3306",

"ssl": "REQUIRED",

"status": "OK",

"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",

"topology": {

"mysql1.db:3306": {

"address": "mysql1.db:3306",

"memberRole": "PRIMARY",

"mode": "R/W",

"readReplicas": {},

"replicationLag": "applier_queue_applied",

"role": "HA",

"status": "ONLINE",

"version": "8.0.31"

"mysql2.db:3306": {

"address": "mysql2.db:3306",

"memberRole": "SECONDARY",

"mode": "R/O",

"readReplicas": {},

"replicationLag": "applier_queue_applied",

"role": "HA",

"status": "ONLINE",

"version": "8.0.31"

"mysql3.db:3306": {

"address": "mysql3.db:3306",

"memberRole": "SECONDARY",

"mode": "R/O",

"readReplicas": {},

"replicationLag": "applier_queue_applied",

"role": "HA",

"status": "ONLINE",

"version": "8.0.31"

}

"topologyMode": "Single-Primary"

"groupInformationSourceMember": "mysql1.db:3306"

}

使用 cluster.checkInstanceState() 检查集群内节点状态


 MySQL  mysql1.db:3306 ssl  JS > cluster.checkInstanceState('icadmin@mysql1.db:3306')
Cluster.checkInstanceState: The instance 'mysql1.db:3306' already belongs to the cluster: 'testCluster'. (RuntimeError)
 MySQL  mysql1.db:3306 ssl  JS > cluster.checkInstanceState('icadmin@mysql2.db:3306')
Cluster.checkInstanceState: The instance 'mysql2.db:3306' already belongs to the cluster: 'testCluster'. (RuntimeError)
 MySQL  mysql1.db:3306 ssl  JS > cluster.checkInstanceState('icadmin@mysql3.db:3306')
Cluster.checkInstanceState: The instance 'mysql3.db:3306' already belongs to the cluster: 'testCluster'. (RuntimeError)

MySQL mysql1.db:3306 ssl JS > cluster.checkInstanceState('icadmin@mysql1.db:3306')

Cluster.checkInstanceState: The instance 'mysql1.db:3306' already belongs to the cluster: 'testCluster'. (RuntimeError)

MySQL mysql1.db:3306 ssl JS > cluster.checkInstanceState('icadmin@mysql2.db:3306')

Cluster.checkInstanceState: The instance 'mysql2.db:3306' already belongs to the cluster: 'testCluster'. (RuntimeError)

MySQL mysql1.db:3306 ssl JS > cluster.checkInstanceState('icadmin@mysql3.db:3306')

Cluster.checkInstanceState: The instance 'mysql3.db:3306' already belongs to the cluster: 'testCluster'. (RuntimeError)

5.集群测试

5.1.写入测试

连上mysql1，往mysql1里写下数据测试下，在mysql2和3上查看


mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | 11036483-680c-11ed-932f-000c2956aa71 | mysql1.db   |        3306 | ONLINE       | PRIMARY     | 8.0.31         | MySQL                      |
| group_replication_applier | 12c67576-680c-11ed-b216-000c2924d927 | mysql2.db   |        3306 | ONLINE       | SECONDARY   | 8.0.31         | MySQL                      |
| group_replication_applier | 12e569b4-680c-11ed-b942-000c29409b22 | mysql3.db   |        3306 | ONLINE       | SECONDARY   | 8.0.31         | MySQL                      |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

3 rows in set (0.00 sec)


mysql> CREATE DATABASE test;
E test;
CREATE TABLE t1 (c1Query OK, 1 row affected (0.02 sec)

mysql> USE test;
 INT PRIMARY KEY, c2 TEXT NOTDatabase changed
mysql> CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);
NSERT INTO t1 VALUES (1, 'Luis');Query OK, 0 rows affected (0.04 sec)

mysql> INSERT INTO t1 VALUES (1, 'Luis');
Query OK, 1 row affected (0.01 sec)

mysql> CREATE DATABASE test;

E test;

CREATE TABLE t1 (c1Query OK, 1 row affected (0.02 sec)

mysql> USE test;

INT PRIMARY KEY, c2 TEXT NOTDatabase changed

mysql> CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);

NSERT INTO t1 VALUES (1, 'Luis');Query OK, 0 rows affected (0.04 sec)

mysql> INSERT INTO t1 VALUES (1, 'Luis');

Query OK, 1 row affected (0.01 sec)


mysql> select * from test.t1;
+----+------+
| c1 | c2   |
+----+------+
|  1 | Luis |
+----+------+
1 row in set (0.00 sec)

mysql> select * from test.t1;

+----+------+

| c1 | c2 |

+----+------+

| 1 | Luis |

+----+------+

1 row in set (0.00 sec)

5.2 灾备测试

我们尝试直接停掉mysql1的服务来模拟主库挂掉，在两外两个机器上查看集群状态，可以看到mysql2自动成为了PRIMARY


mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | 12c67576-680c-11ed-b216-000c2924d927 | mysql2.db   |        3306 | ONLINE       | PRIMARY     | 8.0.31         | MySQL                      |
| group_replication_applier | 12e569b4-680c-11ed-b942-000c29409b22 | mysql3.db   |        3306 | ONLINE       | SECONDARY   | 8.0.31         | MySQL                      |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
2 rows in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

2 rows in set (0.00 sec)

向mysql2写入数据测试


mysql> INSERT INTO test.t1 VALUES (2, 'HylTest');
Query OK, 1 row affected (0.00 sec)

mysql>

mysql> INSERT INTO test.t1 VALUES (2, 'HylTest');

Query OK, 1 row affected (0.00 sec)

mysql>

到mysql3查看，可以看到数据正常


mysql> select * from test.t1;
+----+---------+
| c1 | c2      |
+----+---------+
|  1 | Luis    |
|  2 | HylTest |
+----+---------+
2 rows in set (0.00 sec)

mysql> select * from test.t1;

+----+---------+

| c1 | c2 |

+----+---------+

| 1 | Luis |

| 2 | HylTest |

+----+---------+

2 rows in set (0.00 sec)

重新把mysql1启动，在mysql2也就是当前的主节点查看集群，可以看到mysql1自动成为了SECONDARY


mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | 11036483-680c-11ed-932f-000c2956aa71 | mysql1.db   |        3306 | ONLINE       | SECONDARY   | 8.0.31         | MySQL                      |
| group_replication_applier | 12c67576-680c-11ed-b216-000c2924d927 | mysql2.db   |        3306 | ONLINE       | PRIMARY     | 8.0.31         | MySQL                      |
| group_replication_applier | 12e569b4-680c-11ed-b942-000c29409b22 | mysql3.db   |        3306 | ONLINE       | SECONDARY   | 8.0.31         | MySQL                      |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

3 rows in set (0.00 sec)

到mysql1检查下c1=2也就是宕机期间的数据还在不在，可以发现mysql1重新加入集群后会自动恢复宕机期间的数据


mysql> select * from test.t1;
+----+---------+
| c1 | c2      |
+----+---------+
|  1 | Luis    |
|  2 | HylTest |
+----+---------+
2 rows in set (0.00 sec)

mysql> select * from test.t1;

+----+---------+

| c1 | c2 |

+----+---------+

| 1 | Luis |

| 2 | HylTest |

+----+---------+

2 rows in set (0.00 sec)

附录：用到的命令


mysql1
192.168.2.192

mysql2
192.168.2.171

mysql3
192.168.2.219

scp .\mysql-apt-config_0.8.24-1_all.deb mysql1@192.168.2.192:/home/mysql1
scp .\mysql-apt-config_0.8.24-1_all.deb mysql2@192.168.2.171:/home/mysql2
scp .\mysql-apt-config_0.8.24-1_all.deb mysql3@192.168.2.219:/home/mysql3
sudo dpkg -i mysql-apt-config_0.8.24-1_all.deb
sudo apt install -y net-tools mysql-apt-config
#https://mirrors.tuna.tsinghua.edu.cn
sudo sed -i "s@http://.*archive.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.list
sudo sed -i "s@http://.*security.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.list
sudo apt update
sudo apt install -y mysql-server mysql-shell

root
bqRnG3iP
mysql -u root -p 

MySQL 8 uses a new authentication based on improved SHA256-based password methods. It is recommended 
that all new MySQL Server installations use this method going forward. This new authentication       
plugin requires new versions of connectors and clients, with support for this new authentication     
method (caching_sha2_password). Currently MySQL 8 Connectors and community drivers built with        
libmysqlclient21 support this new method. Clients built with older versions of libmysqlclient may    
not be able to connect to the new server.                                                            
                                                                                                     
To retain compatibility with older client software, the default authentication plugin can be set to  
the legacy value (mysql_native_password) This should only be done if required third-party software   
has not been updated to work with the new authentication method. The change will be written to the   
file /etc/mysql/mysql.conf.d/default-auth-override.cnf                                               
                                                                                                     
After installation, the default can be changed by setting the default_authentication_plugin server   
setting.                                                                                             

#for ubuntu default mysql source
sudo cat /etc/mysql/debian.cnf
debian-sys-maint

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf
report_host=mysql1.db
report_host=mysql2.db
report_host=mysql3.db

sudo service mysql restart

#/etc/hosts
192.168.2.192 mysql1.db
192.168.2.171 mysql2.db
192.168.2.219 mysql3.db

icadmin
bqRnG3iP

dba.configureInstance('root@localhost:3306',{clusterAdmin:"'icadmin'@'localhost'"})
dba.configureInstance('root@localhost:3306',{clusterAdmin:"'icadmin'@'192.168.2.%'"})

dba.checkInstanceConfiguration('icadmin@localhost:3306') 
dba.checkInstanceConfiguration('icadmin@mysql1.db:3306')
dba.checkInstanceConfiguration('icadmin@mysql2.db:3306')
dba.checkInstanceConfiguration('icadmin@mysql3.db:3306')

\connect icadmin@mysql1.db:3306
dba.createCluster('testCluster', {replicationAllowedHost:'192.168.2.0/24'})

var cluster = dba.getCluster()
cluster.setupAdminAccount("icadmin")

cluster.addInstance('icadmin@mysql2.db:3306')
cluster.addInstance('icadmin@mysql3.db:3306')

SELECT * FROM performance_schema.replication_group_members;
 
CREATE DATABASE test;
USE test;
CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);
INSERT INTO t1 VALUES (1, 'Luis');
INSERT INTO t1 VALUES (2, 'HylTest');
INSERT INTO t1 VALUES (3, 'HylTest3');
INSERT INTO t1 VALUES (4, 'HylTest4');
 
select * from test.t1;

mysql1

192.168.2.192

mysql2

192.168.2.171

mysql3

192.168.2.219

scp .\mysql-apt-config_0.8.24-1_all.deb mysql1@192.168.2.192:/home/mysql1

scp .\mysql-apt-config_0.8.24-1_all.deb mysql2@192.168.2.171:/home/mysql2

scp .\mysql-apt-config_0.8.24-1_all.deb mysql3@192.168.2.219:/home/mysql3

sudo dpkg -i mysql-apt-config_0.8.24-1_all.deb

sudo apt install -y net-tools mysql-apt-config

#https://mirrors.tuna.tsinghua.edu.cn

sudo sed -i "s@http://.*archive.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.list

sudo sed -i "s@http://.*security.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.list

sudo apt update

sudo apt install -y mysql-server mysql-shell

root

bqRnG3iP

mysql -u root -p

MySQL 8 uses a new authentication based on improved SHA256-based password methods. It is recommended

that all new MySQL Server installations use this method going forward. This new authentication

plugin requires new versions of connectors and clients, with support for this new authentication

method (caching_sha2_password). Currently MySQL 8 Connectors and community drivers built with

libmysqlclient21 support this new method. Clients built with older versions of libmysqlclient may

not be able to connect to the new server.

To retain compatibility with older client software, the default authentication plugin can be set to

the legacy value (mysql_native_password) This should only be done if required third-party software

has not been updated to work with the new authentication method. The change will be written to the

file /etc/mysql/mysql.conf.d/default-auth-override.cnf

After installation, the default can be changed by setting the default_authentication_plugin server

setting.

#for ubuntu default mysql source

sudo cat /etc/mysql/debian.cnf

debian-sys-maint

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

report_host=mysql1.db

report_host=mysql2.db

report_host=mysql3.db

sudo service mysql restart

#/etc/hosts

192.168.2.192 mysql1.db

192.168.2.171 mysql2.db

192.168.2.219 mysql3.db

icadmin

bqRnG3iP

dba.configureInstance('root@localhost:3306',{clusterAdmin:"'icadmin'@'localhost'"})

dba.configureInstance('root@localhost:3306',{clusterAdmin:"'icadmin'@'192.168.2.%'"})

dba.checkInstanceConfiguration('icadmin@localhost:3306')

dba.checkInstanceConfiguration('icadmin@mysql1.db:3306')

dba.checkInstanceConfiguration('icadmin@mysql2.db:3306')

dba.checkInstanceConfiguration('icadmin@mysql3.db:3306')

\connect icadmin@mysql1.db:3306

dba.createCluster('testCluster', {replicationAllowedHost:'192.168.2.0/24'})

var cluster = dba.getCluster()

cluster.setupAdminAccount("icadmin")

cluster.addInstance('icadmin@mysql2.db:3306')

cluster.addInstance('icadmin@mysql3.db:3306')

SELECT * FROM performance_schema.replication_group_members;

CREATE DATABASE test;

USE test;

CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);

INSERT INTO t1 VALUES (1, 'Luis');

INSERT INTO t1 VALUES (2, 'HylTest');

INSERT INTO t1 VALUES (3, 'HylTest3');

INSERT INTO t1 VALUES (4, 'HylTest4');

select * from test.t1;

ubuntu安装速度测试

安系统时候在均使用清华源情况下居然发现有错,顺便测下怎么给资源能装的快点

#https://mirrors.tuna.tsinghua.edu.cn

都是40G预分配硬盘

是否联网|核数内存|单个CPU核数*CPU个数|测试结果虚拟磁盘名

联网|8c1g| 1c*8 不开启VT-x| 反复安装失败从未见过，可能是CPU内存给少了

联网|8c2g| 1c*8 不开启VT-x| 将近25min，怀疑是CPU给少了 mysql1.vmdk

联网|8c2g| 4c*2 不开启VT-x| 18min，大概10min装完，但是自动升级花了8min mysql1-0.vmdk

断网|8c2g| 4c*2 不开启VT-x| 6min，果然还是断网大法最好，后面都断网装系统好了。这样首次启动联网即可有DHCP下发的IP不影响测试 mysql1-1.vmdk

继续尝试增加核心数看看能否更快

断网|8c2g| 8c*1 不开启VT-x| 5min45s，基本持平，所以CPU数量并不影响这个速度，尝试削减核数 mysql1-2.vmdk

断网|4c2g| 4c*1 不开启VT-x|5min51s，基本持平，所以CPU核数并不影响这个速度，怀疑是单线程，因此试试1c*1 mysql1-3.vmdk

断网|1c2g| 1c*1 不开启VT-x|7min23s，明显下滑，说明不是单线程，至少是2个线程，不然不会这么长时间 mysql1-4.vmdk

断网|2c2g| 2c*1 不开启VT-x|6min22s，回归6min上下了，说明安装的过程确实是2个线程 mysql1-5.vmdk

断网|2c2g| 2c*1 开启VT-x|TODO 有闲心再测估计不会再快了

参考文档

https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-install-linux-quick.html

https://dev.mysql.com/doc/mysql-shell/8.0/en/configuring-router-user.html

https://dev.mysql.com/doc/mysql-shell/8.0/en/innodb-cluster-user-accounts.html#admin-api-configuring-users

https://dev.mysql.com/doc/mysql-shell/8.0/en/check-instance-configuration.html

https://dev.mysql.com/doc/mysql-shell/8.0/en/creating-user-accounts-for-admin-api.html

20221110nas扩容

综述

按之前所说，保留群晖系统，但文件不存群晖而放到HP服务器的硬件raid磁盘阵列

常见方案无非smb nfs sftp webdav等，但我之前想直接搞出个iSCSI挂到群晖只是理论上行，从esxi没能找到这种做法

思考了下，原理是esxi是第一层虚拟化，之后所有的东西已经不再直接和硬件打交道，所以就算能搞出iSCSI协议也有性能损失，还不如继续用更上层协议

现有的openwrt已经是一个linux内核了，理论上装插件就能支持这些。然而不幸的是openwrt原来是未路由器开发的嵌入式系统，但我并没有嵌入式系统的开发和debug经验，在尝试过安装插件各种蜜汁bug后自己编译，依然不行。更坑爹的是编译需要科学上网，编译环境的科学上网依赖openwrt内部插件分流，死循环

详细尝试记录在后面，持续更新中

openwrt尝试

暂时创建thin模式的虚拟磁盘4T，挂到opwnwrt的/data下

和普通服务器操作一样，cfdisk可以方便的创建分区，选GPT即可

mkfs.ext4 /dev/sdb可以格式化磁盘

给nas新建一个用户vim /etc/passwd

echo “nas:*:1000:1000:nas:/tmp:/bin/ash” >>/etc/passwd

passwd nas设置密码

/etc/group 新增组

echo “nas:x:1000:” >> /etc/group

smbpasswd -a nas添加smbpasswd用户

之后进入openwrt的挂载点下新建挂载点后选择自定义挂载位置到/data即可

然后开启openwrt的文件共享，这里使用samba，选择允许用户nas，保存应用即可

之后网页保存

注意，这里如果要扩容的是overlay分区，官方的x86-64镜像无法扩容overlay是已知bug，见https://github.com/openwrt/openwrt/issues/7352
如镜像版本 openwrt-22.03.2-x86-64-generic-squashfs-combined-efi
直接在/etc/rc.local 插入”mount /dev/sda3 /overlay” 就能绕过去，不需要重新编译

lean的版本修过了所以没问题不要浪费时间找乱七八糟的教程了，浪费时间
https://github.com/coolsnowwolf/lede/blob/master/package/system/fstools/patches/0001-fstools-support-extroot-for-non-MTD-rootfs_data.patch

这样操作完发现没权限写入，给加下权限 chmod o+w /data && chmod g+w /data

测速发现只有300Mbps的速度，感觉不对劲看了下openwrt的监控发现是CPU100%,尝试增加CPU核数到2继续测试

2core

尝试升级samba版本到4后发现luci-app-samba4装不上，暂时手动配置server /etc/samba/smb.config.template

[openwrtstorage]
path = /data
valid users = nas
writable = yes
read only = no
create mask = 0666
director mask = 0755
inherit owner = yes

这样虽然速度不太稳定，但勉强能跑满千兆网了，后面再研究为啥

nfs尝试

安装服务 sudo apt install nfs-kernel-server

添加共享 echo “/data 192.168.2.1(rw,sync,no_hide)”>>/etc/exports

修改端口 http://bryanw.tk/2012/specify-nfs-ports-ubuntu-linux/ 注意kernal-server的配置要用–port

rpcinfo -p查看端口

跨防火墙失败，因为ubuntu下的start-stop-daemon加了参数后就无法传递给mountd了，导致端口无法修改

https://wiki.debian.org/SecuringNFS

http://bryanw.tk/2012/specify-nfs-ports-ubuntu-linux/

直接放和nas一个网段，速度依旧跑不满千兆。。。。迷

openmediavault尝试

https://forum.openmediavault.org/

下载iso后新建虚机安装，这次数据盘直接用100G预分配的测速IO

在test下安装启动后没啥问题，修改到管理网的交换机下和nas放一起继续测速

整个界面简介明了，常见协议都支持

写入速度排查

这种情况下测试，客户端本地磁盘原地复制读写速度均大于1Gbps峰值，下载到同网段windows可以跑满1G带宽，可以排除客户端磁盘和网络瓶颈以及服务器网络瓶颈

scp linux->linux同文件的速度是30MBps，约240Mbps，因此怀疑服务器磁盘连续速度有瓶颈

dd测试 dd if=/dev/zero of=./test.data bs=1G count=1 oflag=direct发现结果如下

记录了1+0 的读入
记录了1+0 的写出
1073741824字节（1.1 GB，1.0 GiB）已复制，36.2007 s，29.7 MB/s

果然啊。。。折腾半天居然是这个问题

接下来继续排查是esxi的虚拟化导致，还是硬件故障

在esxi命令行下执行 time dd if=/dev/zero of=test.data bs=100M count=50
50+0 records in
50+0 records out
real 0m 24.03s
user 0m 16.95s
sys 0m 0.00s

平均速度是208MB/s，说明也不是HPraid卡和硬件的瓶颈，硬件本身至少有这个写入速度的能力

那就只能继续怀疑esxi虚拟化带来的问题了。。。。TMD上次win10启动缓慢查了半天就是esxi的锅，这次磁盘写入缓慢没准又是esxi的问题

不过也可能是我哪里没选对？

仔细观察发现esxi提醒的虚拟机操作系统的类型不对，我默认选的都是linux64，它建议我选Debian 10 64。选择后重启系统再试

dd if=/dev/zero of=./test.data bs=1G count=1 oflag=direct
记录了1+0 的读入
记录了1+0 的写出
1073741824字节（1.1 GB，1.0 GiB）已复制，17.7218 s，60.6 MB/s

确实有提升，真的是因为这个选项导致的吗？

swapoff -a 禁掉swap后再试，etc/fstab也注释掉swap分区

dd if=/dev/zero of=./test.data bs=1G count=1 oflag=direct
dd: 1073741824 字节的输入缓冲导致内存耗尽（1.0 GiB）

果然dd的运行依赖内存，更改成10次100M继续测速

dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，25.1924 s，41.6 MB/s

这个速度和windows上网卡320Mbps左右带宽监控的速度能对上

奇怪的是相同的虚拟机磁盘，win10下能有130MBps左右的写入速度，还是在同时读的情况下

所以综合排除应该是linux系统或者是ext4文件系统的问题，在旧NAS上也是单盘ext4相同命令测速如下

dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
10+0 records in
10+0 records out
1048576000 bytes (1.0 GB) copied, 7.46678 s, 140 MB/s

uname -a输出如下，尝试从这上面寻找点线索？

Linux openmediavault 5.16.0-0.bpo.4-amd64 #1 SMP PREEMPT Debian 5.16.12-1~bpo11+1 (2022-03-08) x86_64 GNU/Linux

尝试将磁盘改成独立，不受esxi快照影响继续测试如下

root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，4.37703 s，240 MB/s
root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，4.24505 s，247 MB/s
root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，4.25252 s，247 MB/s

root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，3.99055 s，263 MB/s

再次测速如下图，终于能跑满了，但奇怪的是，为啥非独立的NTFS磁盘没这个问题呢？debian linux下挂载的ext4才会这样

thin+独立模式测试

预分配100G测试ok后，继续尝试thin模式，因为我还想省点地方拿来测集群搭建用

root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，21.3584 s，49.1 MB/s
root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，21.3513 s，49.1 MB/s
root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，20.5089 s，51.1 MB/s
root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，17.6997 s，59.2 MB/s
root@openmediavault:~/data# dd if=/dev/zero of=./test.data bs=100M count=10 oflag=direct
记录了10+0 的读入
记录了10+0 的写出
1048576000字节（1.0 GB，1000 MiB）已复制，22.037 s，47.6 MB/s

这样差不多480Mbps的带宽，按这个效果来看的话，后面需要整理下文件做分类，然后弄两块虚拟磁盘都选独立。

thin模式用作长期存储，所以写入慢点也无所谓，同时是按实际大小消耗磁盘阵列，把需要备份的文件存到这里，有硬件raid的保护可以满足安全性提升的目的

预分配的磁盘用作下载热数据+日常使用，或者干脆接着用原来的nas和本地盘也行，反正都是下的电影和番剧，极端情况丢了就丢了，把种子放到HP服务器存一份就行

另外后面mysql集群使用的虚拟硬盘，必须勾选这个独立+预分配，否则会对后续分布式的性能测试带来难以排查的性能问题

参考

https://blog.csdn.net/kk380446/article/details/114213014

https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html

20221029解决esxi启动win10缓慢的bug

无情开评的小伙伴给装的esxi第二次启动win10时间巨长，翻阅论坛后发现是esxi的bug，升级就行，至少是vSphere 6.7 Update 3

触发条件也是完美符合我的场景。。。vmfs6+win10新版+有快照

讨论贴

https://communities.vmware.com/t5/ESXi-Discussions/windows-10-1809-slow/m-p/1424925

官方说明

https://kb.vmware.com/s/article/67426

ESXi670-202210001.zip

国内好人的镜像https://www.dinghui.org/vmware-iso-download.html

vmware -vl
VMware ESXi 6.7.0 build-8169922
VMware ESXi 6.7.0 GA

进ssh执行后升级重启服务器即可

esxcli software vib install -d=/vmfs/volumes/datastore1/patchs/ESXi670-202210001.zip

升级后检查版本

vmware -vl

VMware ESXi 6.7.0 build-20497097
VMware ESXi 6.7.0 Update 3

升级输出如下


Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: VMW_bootbank_bnxtnet_20.6.101.7-24vmw.670.3.73.14320388, VMW_bootbank_bnxtroce_20.6.101.0-20vmw.670.1.28.10302608, VMW_bootbank_brcmfcoe_11.4.1078.26-14vmw.670.3.159.18828794, VMW_bootbank_elxnet_11.4.1097.0-5vmw.670.3.73.14320388, VMW_bootbank_i40en_1.8.1.9-2vmw.670.3.73.14320388, VMW_bootbank_igbn_0.1.1.0-5vmw.670.3.73.14320388, VMW_bootbank_ipmi-ipmi-devintf_39.1-5vmw.670.1.28.10302608, VMW_bootbank_ipmi-ipmi-msghandler_39.1-5vmw.670.1.28.10302608, VMW_bootbank_ipmi-ipmi-si-drv_39.1-5vmw.670.1.28.10302608, VMW_bootbank_iser_1.0.0.0-1vmw.670.1.28.10302608, VMW_bootbank_ixgben_1.7.1.16-2vmw.670.3.104.16075168, VMW_bootbank_lpfc_11.4.33.26-14vmw.670.3.104.16075168, VMW_bootbank_lsi-mr3_7.708.07.00-3vmw.670.3.73.14320388, VMW_bootbank_lsi-msgpt2_20.00.06.00-2vmw.670.3.73.14320388, VMW_bootbank_lsi-msgpt35_09.00.00.00-5vmw.670.3.73.14320388, VMW_bootbank_lsi-msgpt3_17.00.02.00-1vmw.670.3.73.14320388, VMW_bootbank_misc-drivers_6.7.0-2.48.13006603, VMW_bootbank_mtip32xx-native_3.9.8-1vmw.670.1.28.10302608, VMW_bootbank_ne1000_0.8.4-2vmw.670.2.48.13006603, VMW_bootbank_nenic_1.0.29.0-1vmw.670.3.73.14320388, VMW_bootbank_net-e1000_8.0.3.1-5vmw.670.3.112.16701467, VMW_bootbank_net-vmxnet3_1.1.3.0-3vmw.670.3.104.16075168, VMW_bootbank_nfnic_4.0.0.44-0vmw.670.3.104.16075168, VMW_bootbank_nhpsa_2.0.22-3vmw.670.1.28.10302608, VMW_bootbank_nmlx4-core_3.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_nmlx4-en_3.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_nmlx4-rdma_3.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_nmlx5-core_4.17.13.1-1vmw.670.3.73.14320388, VMW_bootbank_nmlx5-rdma_4.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_ntg3_4.1.8.0-4vmw.670.3.189.20497097, VMW_bootbank_nvme_1.2.2.28-5vmw.670.3.159.18828794, VMW_bootbank_nvmxnet3_2.0.0.29-1vmw.670.1.28.10302608, VMW_bootbank_qedentv_2.0.6.4-10vmw.670.1.28.10302608, VMW_bootbank_qfle3f_1.0.25.0.2-15vmw.670.3.104.16075168, VMW_bootbank_sfvmk_1.0.0.1003-7vmw.670.3.104.16075168, VMW_bootbank_smartpqi_1.0.1.553-28vmw.670.3.73.14320388, VMW_bootbank_vmkfcoe_1.0.0.1-1vmw.670.1.28.10302608, VMW_bootbank_vmkusb_0.1-4vmw.670.3.159.18828794, VMW_bootbank_vmw-ahci_2.0.7-2vmw.670.3.143.17700523, VMW_bootbank_xhci-xhci_1.0-3vmw.670.3.185.20491463, VMware_bootbank_cpu-microcode_6.7.0-3.170.19898894, VMware_bootbank_elx-esx-libelxima.so_11.4.1184.2-3.89.15160138, VMware_bootbank_esx-base_6.7.0-3.189.20497097, VMware_bootbank_esx-ui_1.43.10-20199807, VMware_bootbank_esx-update_6.7.0-3.189.20497097, VMware_bootbank_esx-xserver_6.7.0-3.189.20497097, VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-16vmw.670.1.28.10302608, VMware_bootbank_lsu-intel-vmd-plugin_1.0.0-2vmw.670.1.28.10302608, VMware_bootbank_lsu-lsi-drivers-plugin_1.0.0-1vmw.670.2.48.13006603, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-13vmw.670.1.28.10302608, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-9vmw.670.2.48.13006603, VMware_bootbank_lsu-smartpqi-plugin_1.0.0-4vmw.670.3.159.18828794, VMware_bootbank_native-misc-drivers_6.7.0-3.89.15160138, VMware_bootbank_qlnativefc_3.1.8.0-5vmw.670.3.73.14320388, VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.36-2vmw.670.3.116.16713306, VMware_bootbank_vsan_6.7.0-3.189.20235859, VMware_bootbank_vsanhealth_6.7.0-3.189.20235860, VMware_locker_tools-light_12.0.6.20104755-20491463
   VIBs Removed: VMW_bootbank_bnxtnet_20.6.101.7-11vmw.670.0.0.8169922, VMW_bootbank_brcmfcoe_11.4.1078.0-8vmw.670.0.0.8169922, VMW_bootbank_elxnet_11.4.1094.0-5vmw.670.0.0.8169922, VMW_bootbank_i40en_1.3.1-18vmw.670.0.0.8169922, VMW_bootbank_igbn_0.1.0.0-15vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-devintf_39.1-4vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.670.0.0.8169922, VMW_bootbank_iser_1.0.0.0-1vmw.670.0.0.8169922, VMW_bootbank_ixgben_1.4.1-11vmw.670.0.0.8169922, VMW_bootbank_lpfc_11.4.33.1-6vmw.670.0.0.8169922, VMW_bootbank_lsi-mr3_7.702.13.00-4vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt2_20.00.04.00-4vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt35_03.00.01.00-10vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt3_16.00.01.00-1vmw.670.0.0.8169922, VMW_bootbank_misc-drivers_6.7.0-0.0.8169922, VMW_bootbank_mtip32xx-native_3.9.6-1vmw.670.0.0.8169922, VMW_bootbank_ne1000_0.8.3-4vmw.670.0.0.8169922, VMW_bootbank_nenic_1.0.11.0-1vmw.670.0.0.8169922, VMW_bootbank_net-e1000_8.0.3.1-5vmw.670.0.0.8169922, VMW_bootbank_net-vmxnet3_1.1.3.0-3vmw.670.0.0.8169922, VMW_bootbank_nhpsa_2.0.22-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-core_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-en_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-rdma_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx5-core_4.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx5-rdma_4.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_ntg3_4.1.3.0-1vmw.670.0.0.8169922, VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922, VMW_bootbank_nvmxnet3_2.0.0.27-1vmw.670.0.0.8169922, VMW_bootbank_qedentv_2.0.6.4-8vmw.670.0.0.8169922, VMW_bootbank_qfle3f_1.0.25.0.2-14vmw.670.0.0.8169922, VMW_bootbank_smartpqi_1.0.1.553-10vmw.670.0.0.8169922, VMW_bootbank_vmkfcoe_1.0.0.0-1vmw.670.0.0.8169922, VMW_bootbank_vmkusb_0.1-1vmw.670.0.0.8169922, VMW_bootbank_vmw-ahci_1.2.0-6vmw.670.0.0.8169922, VMW_bootbank_xhci-xhci_1.0-3vmw.670.0.0.8169922, VMware_bootbank_cpu-microcode_6.7.0-0.0.8169922, VMware_bootbank_elx-esx-libelxima.so_11.4.1184.0-0.0.8169922, VMware_bootbank_esx-base_6.7.0-0.0.8169922, VMware_bootbank_esx-ui_1.25.0-7872652, VMware_bootbank_esx-xserver_6.7.0-0.0.8169922, VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-13vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-12vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-8vmw.670.0.0.8169922, VMware_bootbank_native-misc-drivers_6.7.0-0.0.8169922, VMware_bootbank_qlnativefc_3.0.1.0-5vmw.670.0.0.8169922, VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922, VMware_bootbank_vsan_6.7.0-0.0.8169922, VMware_bootbank_vsanhealth_6.7.0-0.0.8169922, VMware_locker_tools-light_10.2.0.7253323-8169922
   VIBs Skipped: VMW_bootbank_ata-libata-92_3.00.9.2-16vmw.670.0.0.8169922, VMW_bootbank_ata-pata-amd_0.3.10-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-atiixp_0.4.6-4vmw.670.0.0.8169922, VMW_bootbank_ata-pata-cmd64x_0.2.5-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-pdc2027x_1.0-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-serverworks_0.4.3-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-sil680_0.4.8-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-via_0.3.3-2vmw.670.0.0.8169922, VMW_bootbank_block-cciss_3.6.14-10vmw.670.0.0.8169922, VMW_bootbank_char-random_1.0-3vmw.670.0.0.8169922, VMW_bootbank_ehci-ehci-hcd_1.0-4vmw.670.0.0.8169922, VMW_bootbank_elxiscsi_11.4.1174.0-2vmw.670.0.0.8169922, VMW_bootbank_hid-hid_1.0-3vmw.670.0.0.8169922, VMW_bootbank_iavmd_1.2.0.1011-2vmw.670.0.0.8169922, VMW_bootbank_ima-qla4xxx_2.02.18-1vmw.670.0.0.8169922, VMW_bootbank_lpnic_11.4.59.0-1vmw.670.0.0.8169922, VMW_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.670.0.0.8169922, VMW_bootbank_net-bnx2_2.2.4f.v60.10-2vmw.670.0.0.8169922, VMW_bootbank_net-bnx2x_1.78.80.v60.12-2vmw.670.0.0.8169922, VMW_bootbank_net-cdc-ether_1.0-3vmw.670.0.0.8169922, VMW_bootbank_net-cnic_1.78.76.v60.13-2vmw.670.0.0.8169922, VMW_bootbank_net-e1000e_3.2.2.1-2vmw.670.0.0.8169922, VMW_bootbank_net-enic_2.1.2.38-2vmw.670.0.0.8169922, VMW_bootbank_net-fcoe_1.0.29.9.3-7vmw.670.0.0.8169922, VMW_bootbank_net-forcedeth_0.61-2vmw.670.0.0.8169922, VMW_bootbank_net-igb_5.0.5.1.1-5vmw.670.0.0.8169922, VMW_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.670.0.0.8169922, VMW_bootbank_net-libfcoe-92_1.0.24.9.4-8vmw.670.0.0.8169922, VMW_bootbank_net-mlx4-core_1.9.7.0-1vmw.670.0.0.8169922, VMW_bootbank_net-mlx4-en_1.9.7.0-1vmw.670.0.0.8169922, VMW_bootbank_net-nx-nic_5.0.621-5vmw.670.0.0.8169922, VMW_bootbank_net-tg3_3.131d.v60.4-2vmw.670.0.0.8169922, VMW_bootbank_net-usbnet_1.0-3vmw.670.0.0.8169922, VMW_bootbank_nvmxnet3-ens_2.0.0.21-1vmw.670.0.0.8169922, VMW_bootbank_ohci-usb-ohci_1.0-3vmw.670.0.0.8169922, VMW_bootbank_pvscsi_0.1-2vmw.670.0.0.8169922, VMW_bootbank_qcnic_1.0.2.0.4-1vmw.670.0.0.8169922, VMW_bootbank_qfle3_1.0.50.11-9vmw.670.0.0.8169922, VMW_bootbank_qfle3i_1.0.2.3.9-3vmw.670.0.0.8169922, VMW_bootbank_qflge_1.1.0.11-1vmw.670.0.0.8169922, VMW_bootbank_sata-ahci_3.0-26vmw.670.0.0.8169922, VMW_bootbank_sata-ata-piix_2.12-10vmw.670.0.0.8169922, VMW_bootbank_sata-sata-nv_3.5-4vmw.670.0.0.8169922, VMW_bootbank_sata-sata-promise_2.12-3vmw.670.0.0.8169922, VMW_bootbank_sata-sata-sil24_1.1-1vmw.670.0.0.8169922, VMW_bootbank_sata-sata-sil_2.3-4vmw.670.0.0.8169922, VMW_bootbank_sata-sata-svw_2.3-3vmw.670.0.0.8169922, VMW_bootbank_scsi-aacraid_1.1.5.1-9vmw.670.0.0.8169922, VMW_bootbank_scsi-adp94xx_1.0.8.12-6vmw.670.0.0.8169922, VMW_bootbank_scsi-aic79xx_3.1-6vmw.670.0.0.8169922, VMW_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.670.0.0.8169922, VMW_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.670.0.0.8169922, VMW_bootbank_scsi-fnic_1.5.0.45-3vmw.670.0.0.8169922, VMW_bootbank_scsi-hpsa_6.0.0.84-3vmw.670.0.0.8169922, VMW_bootbank_scsi-ips_7.12.05-4vmw.670.0.0.8169922, VMW_bootbank_scsi-iscsi-linux-92_1.0.0.2-3vmw.670.0.0.8169922, VMW_bootbank_scsi-libfc-92_1.0.40.9.3-5vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid2_2.00.4-9vmw.670.0.0.8169922, VMW_bootbank_scsi-mpt2sas_19.00.00.00-2vmw.670.0.0.8169922, VMW_bootbank_scsi-mptsas_4.23.01.00-10vmw.670.0.0.8169922, VMW_bootbank_scsi-mptspi_4.23.01.00-10vmw.670.0.0.8169922, VMW_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.670.0.0.8169922, VMW_bootbank_shim-iscsi-linux-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-iscsi-linux-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libata-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libata-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfc-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfc-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfcoe-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfcoe-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-3-0_6.7.0-0.0.8169922, VMW_bootbank_uhci-usb-uhci_1.0-3vmw.670.0.0.8169922, VMW_bootbank_usb-storage-usb-storage_1.0-3vmw.670.0.0.8169922, VMW_bootbank_usbcore-usb_1.0-3vmw.670.0.0.8169922, VMW_bootbank_vmkata_0.1-1vmw.670.0.0.8169922, VMW_bootbank_vmkplexer-vmkplexer_6.7.0-0.0.8169922, VMware_bootbank_esx-dvfilter-generic-fastpath_6.7.0-0.0.8169922, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-9vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-mpt2sas-plugin_2.0.0-7vmw.670.0.0.8169922, VMware_bootbank_rste_2.0.2.0088-7vmw.670.0.0.8169922

Installation Result

Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

Reboot Required: true

VIBs Installed: VMW_bootbank_bnxtnet_20.6.101.7-24vmw.670.3.73.14320388, VMW_bootbank_bnxtroce_20.6.101.0-20vmw.670.1.28.10302608, VMW_bootbank_brcmfcoe_11.4.1078.26-14vmw.670.3.159.18828794, VMW_bootbank_elxnet_11.4.1097.0-5vmw.670.3.73.14320388, VMW_bootbank_i40en_1.8.1.9-2vmw.670.3.73.14320388, VMW_bootbank_igbn_0.1.1.0-5vmw.670.3.73.14320388, VMW_bootbank_ipmi-ipmi-devintf_39.1-5vmw.670.1.28.10302608, VMW_bootbank_ipmi-ipmi-msghandler_39.1-5vmw.670.1.28.10302608, VMW_bootbank_ipmi-ipmi-si-drv_39.1-5vmw.670.1.28.10302608, VMW_bootbank_iser_1.0.0.0-1vmw.670.1.28.10302608, VMW_bootbank_ixgben_1.7.1.16-2vmw.670.3.104.16075168, VMW_bootbank_lpfc_11.4.33.26-14vmw.670.3.104.16075168, VMW_bootbank_lsi-mr3_7.708.07.00-3vmw.670.3.73.14320388, VMW_bootbank_lsi-msgpt2_20.00.06.00-2vmw.670.3.73.14320388, VMW_bootbank_lsi-msgpt35_09.00.00.00-5vmw.670.3.73.14320388, VMW_bootbank_lsi-msgpt3_17.00.02.00-1vmw.670.3.73.14320388, VMW_bootbank_misc-drivers_6.7.0-2.48.13006603, VMW_bootbank_mtip32xx-native_3.9.8-1vmw.670.1.28.10302608, VMW_bootbank_ne1000_0.8.4-2vmw.670.2.48.13006603, VMW_bootbank_nenic_1.0.29.0-1vmw.670.3.73.14320388, VMW_bootbank_net-e1000_8.0.3.1-5vmw.670.3.112.16701467, VMW_bootbank_net-vmxnet3_1.1.3.0-3vmw.670.3.104.16075168, VMW_bootbank_nfnic_4.0.0.44-0vmw.670.3.104.16075168, VMW_bootbank_nhpsa_2.0.22-3vmw.670.1.28.10302608, VMW_bootbank_nmlx4-core_3.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_nmlx4-en_3.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_nmlx4-rdma_3.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_nmlx5-core_4.17.13.1-1vmw.670.3.73.14320388, VMW_bootbank_nmlx5-rdma_4.17.13.1-1vmw.670.2.48.13006603, VMW_bootbank_ntg3_4.1.8.0-4vmw.670.3.189.20497097, VMW_bootbank_nvme_1.2.2.28-5vmw.670.3.159.18828794, VMW_bootbank_nvmxnet3_2.0.0.29-1vmw.670.1.28.10302608, VMW_bootbank_qedentv_2.0.6.4-10vmw.670.1.28.10302608, VMW_bootbank_qfle3f_1.0.25.0.2-15vmw.670.3.104.16075168, VMW_bootbank_sfvmk_1.0.0.1003-7vmw.670.3.104.16075168, VMW_bootbank_smartpqi_1.0.1.553-28vmw.670.3.73.14320388, VMW_bootbank_vmkfcoe_1.0.0.1-1vmw.670.1.28.10302608, VMW_bootbank_vmkusb_0.1-4vmw.670.3.159.18828794, VMW_bootbank_vmw-ahci_2.0.7-2vmw.670.3.143.17700523, VMW_bootbank_xhci-xhci_1.0-3vmw.670.3.185.20491463, VMware_bootbank_cpu-microcode_6.7.0-3.170.19898894, VMware_bootbank_elx-esx-libelxima.so_11.4.1184.2-3.89.15160138, VMware_bootbank_esx-base_6.7.0-3.189.20497097, VMware_bootbank_esx-ui_1.43.10-20199807, VMware_bootbank_esx-update_6.7.0-3.189.20497097, VMware_bootbank_esx-xserver_6.7.0-3.189.20497097, VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-16vmw.670.1.28.10302608, VMware_bootbank_lsu-intel-vmd-plugin_1.0.0-2vmw.670.1.28.10302608, VMware_bootbank_lsu-lsi-drivers-plugin_1.0.0-1vmw.670.2.48.13006603, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-13vmw.670.1.28.10302608, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-9vmw.670.2.48.13006603, VMware_bootbank_lsu-smartpqi-plugin_1.0.0-4vmw.670.3.159.18828794, VMware_bootbank_native-misc-drivers_6.7.0-3.89.15160138, VMware_bootbank_qlnativefc_3.1.8.0-5vmw.670.3.73.14320388, VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.36-2vmw.670.3.116.16713306, VMware_bootbank_vsan_6.7.0-3.189.20235859, VMware_bootbank_vsanhealth_6.7.0-3.189.20235860, VMware_locker_tools-light_12.0.6.20104755-20491463

VIBs Removed: VMW_bootbank_bnxtnet_20.6.101.7-11vmw.670.0.0.8169922, VMW_bootbank_brcmfcoe_11.4.1078.0-8vmw.670.0.0.8169922, VMW_bootbank_elxnet_11.4.1094.0-5vmw.670.0.0.8169922, VMW_bootbank_i40en_1.3.1-18vmw.670.0.0.8169922, VMW_bootbank_igbn_0.1.0.0-15vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-devintf_39.1-4vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.670.0.0.8169922, VMW_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.670.0.0.8169922, VMW_bootbank_iser_1.0.0.0-1vmw.670.0.0.8169922, VMW_bootbank_ixgben_1.4.1-11vmw.670.0.0.8169922, VMW_bootbank_lpfc_11.4.33.1-6vmw.670.0.0.8169922, VMW_bootbank_lsi-mr3_7.702.13.00-4vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt2_20.00.04.00-4vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt35_03.00.01.00-10vmw.670.0.0.8169922, VMW_bootbank_lsi-msgpt3_16.00.01.00-1vmw.670.0.0.8169922, VMW_bootbank_misc-drivers_6.7.0-0.0.8169922, VMW_bootbank_mtip32xx-native_3.9.6-1vmw.670.0.0.8169922, VMW_bootbank_ne1000_0.8.3-4vmw.670.0.0.8169922, VMW_bootbank_nenic_1.0.11.0-1vmw.670.0.0.8169922, VMW_bootbank_net-e1000_8.0.3.1-5vmw.670.0.0.8169922, VMW_bootbank_net-vmxnet3_1.1.3.0-3vmw.670.0.0.8169922, VMW_bootbank_nhpsa_2.0.22-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-core_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-en_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx4-rdma_3.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx5-core_4.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_nmlx5-rdma_4.17.9.12-1vmw.670.0.0.8169922, VMW_bootbank_ntg3_4.1.3.0-1vmw.670.0.0.8169922, VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922, VMW_bootbank_nvmxnet3_2.0.0.27-1vmw.670.0.0.8169922, VMW_bootbank_qedentv_2.0.6.4-8vmw.670.0.0.8169922, VMW_bootbank_qfle3f_1.0.25.0.2-14vmw.670.0.0.8169922, VMW_bootbank_smartpqi_1.0.1.553-10vmw.670.0.0.8169922, VMW_bootbank_vmkfcoe_1.0.0.0-1vmw.670.0.0.8169922, VMW_bootbank_vmkusb_0.1-1vmw.670.0.0.8169922, VMW_bootbank_vmw-ahci_1.2.0-6vmw.670.0.0.8169922, VMW_bootbank_xhci-xhci_1.0-3vmw.670.0.0.8169922, VMware_bootbank_cpu-microcode_6.7.0-0.0.8169922, VMware_bootbank_elx-esx-libelxima.so_11.4.1184.0-0.0.8169922, VMware_bootbank_esx-base_6.7.0-0.0.8169922, VMware_bootbank_esx-ui_1.25.0-7872652, VMware_bootbank_esx-xserver_6.7.0-0.0.8169922, VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-13vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-12vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-8vmw.670.0.0.8169922, VMware_bootbank_native-misc-drivers_6.7.0-0.0.8169922, VMware_bootbank_qlnativefc_3.0.1.0-5vmw.670.0.0.8169922, VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922, VMware_bootbank_vsan_6.7.0-0.0.8169922, VMware_bootbank_vsanhealth_6.7.0-0.0.8169922, VMware_locker_tools-light_10.2.0.7253323-8169922

VIBs Skipped: VMW_bootbank_ata-libata-92_3.00.9.2-16vmw.670.0.0.8169922, VMW_bootbank_ata-pata-amd_0.3.10-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-atiixp_0.4.6-4vmw.670.0.0.8169922, VMW_bootbank_ata-pata-cmd64x_0.2.5-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-pdc2027x_1.0-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-serverworks_0.4.3-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-sil680_0.4.8-3vmw.670.0.0.8169922, VMW_bootbank_ata-pata-via_0.3.3-2vmw.670.0.0.8169922, VMW_bootbank_block-cciss_3.6.14-10vmw.670.0.0.8169922, VMW_bootbank_char-random_1.0-3vmw.670.0.0.8169922, VMW_bootbank_ehci-ehci-hcd_1.0-4vmw.670.0.0.8169922, VMW_bootbank_elxiscsi_11.4.1174.0-2vmw.670.0.0.8169922, VMW_bootbank_hid-hid_1.0-3vmw.670.0.0.8169922, VMW_bootbank_iavmd_1.2.0.1011-2vmw.670.0.0.8169922, VMW_bootbank_ima-qla4xxx_2.02.18-1vmw.670.0.0.8169922, VMW_bootbank_lpnic_11.4.59.0-1vmw.670.0.0.8169922, VMW_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.670.0.0.8169922, VMW_bootbank_net-bnx2_2.2.4f.v60.10-2vmw.670.0.0.8169922, VMW_bootbank_net-bnx2x_1.78.80.v60.12-2vmw.670.0.0.8169922, VMW_bootbank_net-cdc-ether_1.0-3vmw.670.0.0.8169922, VMW_bootbank_net-cnic_1.78.76.v60.13-2vmw.670.0.0.8169922, VMW_bootbank_net-e1000e_3.2.2.1-2vmw.670.0.0.8169922, VMW_bootbank_net-enic_2.1.2.38-2vmw.670.0.0.8169922, VMW_bootbank_net-fcoe_1.0.29.9.3-7vmw.670.0.0.8169922, VMW_bootbank_net-forcedeth_0.61-2vmw.670.0.0.8169922, VMW_bootbank_net-igb_5.0.5.1.1-5vmw.670.0.0.8169922, VMW_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.670.0.0.8169922, VMW_bootbank_net-libfcoe-92_1.0.24.9.4-8vmw.670.0.0.8169922, VMW_bootbank_net-mlx4-core_1.9.7.0-1vmw.670.0.0.8169922, VMW_bootbank_net-mlx4-en_1.9.7.0-1vmw.670.0.0.8169922, VMW_bootbank_net-nx-nic_5.0.621-5vmw.670.0.0.8169922, VMW_bootbank_net-tg3_3.131d.v60.4-2vmw.670.0.0.8169922, VMW_bootbank_net-usbnet_1.0-3vmw.670.0.0.8169922, VMW_bootbank_nvmxnet3-ens_2.0.0.21-1vmw.670.0.0.8169922, VMW_bootbank_ohci-usb-ohci_1.0-3vmw.670.0.0.8169922, VMW_bootbank_pvscsi_0.1-2vmw.670.0.0.8169922, VMW_bootbank_qcnic_1.0.2.0.4-1vmw.670.0.0.8169922, VMW_bootbank_qfle3_1.0.50.11-9vmw.670.0.0.8169922, VMW_bootbank_qfle3i_1.0.2.3.9-3vmw.670.0.0.8169922, VMW_bootbank_qflge_1.1.0.11-1vmw.670.0.0.8169922, VMW_bootbank_sata-ahci_3.0-26vmw.670.0.0.8169922, VMW_bootbank_sata-ata-piix_2.12-10vmw.670.0.0.8169922, VMW_bootbank_sata-sata-nv_3.5-4vmw.670.0.0.8169922, VMW_bootbank_sata-sata-promise_2.12-3vmw.670.0.0.8169922, VMW_bootbank_sata-sata-sil24_1.1-1vmw.670.0.0.8169922, VMW_bootbank_sata-sata-sil_2.3-4vmw.670.0.0.8169922, VMW_bootbank_sata-sata-svw_2.3-3vmw.670.0.0.8169922, VMW_bootbank_scsi-aacraid_1.1.5.1-9vmw.670.0.0.8169922, VMW_bootbank_scsi-adp94xx_1.0.8.12-6vmw.670.0.0.8169922, VMW_bootbank_scsi-aic79xx_3.1-6vmw.670.0.0.8169922, VMW_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.670.0.0.8169922, VMW_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.670.0.0.8169922, VMW_bootbank_scsi-fnic_1.5.0.45-3vmw.670.0.0.8169922, VMW_bootbank_scsi-hpsa_6.0.0.84-3vmw.670.0.0.8169922, VMW_bootbank_scsi-ips_7.12.05-4vmw.670.0.0.8169922, VMW_bootbank_scsi-iscsi-linux-92_1.0.0.2-3vmw.670.0.0.8169922, VMW_bootbank_scsi-libfc-92_1.0.40.9.3-5vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.670.0.0.8169922, VMW_bootbank_scsi-megaraid2_2.00.4-9vmw.670.0.0.8169922, VMW_bootbank_scsi-mpt2sas_19.00.00.00-2vmw.670.0.0.8169922, VMW_bootbank_scsi-mptsas_4.23.01.00-10vmw.670.0.0.8169922, VMW_bootbank_scsi-mptspi_4.23.01.00-10vmw.670.0.0.8169922, VMW_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.670.0.0.8169922, VMW_bootbank_shim-iscsi-linux-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-iscsi-linux-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libata-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libata-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfc-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfc-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfcoe-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-libfcoe-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-1-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-2-0_6.7.0-0.0.8169922, VMW_bootbank_shim-vmklinux-9-2-3-0_6.7.0-0.0.8169922, VMW_bootbank_uhci-usb-uhci_1.0-3vmw.670.0.0.8169922, VMW_bootbank_usb-storage-usb-storage_1.0-3vmw.670.0.0.8169922, VMW_bootbank_usbcore-usb_1.0-3vmw.670.0.0.8169922, VMW_bootbank_vmkata_0.1-1vmw.670.0.0.8169922, VMW_bootbank_vmkplexer-vmkplexer_6.7.0-0.0.8169922, VMware_bootbank_esx-dvfilter-generic-fastpath_6.7.0-0.0.8169922, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-9vmw.670.0.0.8169922, VMware_bootbank_lsu-lsi-mpt2sas-plugin_2.0.0-7vmw.670.0.0.8169922, VMware_bootbank_rste_2.0.2.0088-7vmw.670.0.0.8169922


尝试解决风扇转速问题

esxcli software vib install -d /vmfs/volumes/datastore1/patchs/ams-esxi6.7-bundle-11.2.0-6.zip
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: HPE_bootbank_amsd_670.11.2.0-12.7535516, HPE_bootbank_cru_670.6.7.10.14-1OEM.670.0.0.7535516, HPE_bootbank_fc-enablement_670.3.20.12-7535516, HPE_bootbank_ilo_670.10.1.0.16-1OEM.670.0.0.7535516, HPE_bootbank_smx-limited_670.03.13.00.3-7535516
   VIBs Removed:
   VIBs Skipped:

尝试解决风扇转速问题

esxcli software vib install -d /vmfs/volumes/datastore1/patchs/ams-esxi6.7-bundle-11.2.0-6.zip

Installation Result

Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

Reboot Required: true

VIBs Installed: HPE_bootbank_amsd_670.11.2.0-12.7535516, HPE_bootbank_cru_670.6.7.10.14-1OEM.670.0.0.7535516, HPE_bootbank_fc-enablement_670.3.20.12-7535516, HPE_bootbank_ilo_670.10.1.0.16-1OEM.670.0.0.7535516, HPE_bootbank_smx-limited_670.03.13.00.3-7535516

VIBs Removed:

VIBs Skipped:

安装后从ilo能看到

Name	Description	Version
amsd	Agentless Management Service for ESXi 6.7	670.11.2.0-12.7535516
cru	CRU Kernel Module for vSphere ESXi 6.7	670.6.7.10.14-1OEM.670.0.0.7535516
fc-enablement	Fiber Channel Enablement Package for ESXi 6.7.0	670.3.20.12-7535516
ilo	Channel Interface Kernel Module for vSphere ESXi 6.7	670.10.1.0.16-1OEM.670.0.0.7535516
lsu-smartpqi-plugin	SMARTPQI LSU Management Plugin	1.0.0-4vmw.670.3.159.18828794
smartpqi	VMware native driver module for Microsemi SmartPqi controllers	1.0.1.553-28vmw.670.3.73.14320388
smx-limited	WBEM Providers Limited for ESXi 6.7	670.03.13.00.3-7535516

20221024mysql集群搭建

背景

有了基础的虚拟化环境，打算从头搭建一个分布式应用环境，学习学习各种集群的搭建过程。在公司这些都是别的部门弄好了直接用的，包括高可用、灾备等特性都有值得我研究学习的地方。

集群方式选择

mysql的集群方法

传统方法是基于源节点二进制日志重放事件，并且要求日志文件和位置在源和副本之间是同步的

更新一点的方法是基于 global transaction identifiers (GTIDs)，GTIDs是事务性的，不需要依赖日志文件和位置就能工作，这极大的简化了许多常见的副本任务.使用GTIDs的方法保证了源和副本之间的一致性，所有的事务只要在源上提交了就一定会被应用到副本上。

基于日志的方法

源节点个每个副本都应该被设置唯一的ID (using the server_id system variable)，顺带一提，每个副本必须设置源节点的host name, log file文件名，文件内的偏移量

这些细节可以通过mysql的session语句控制。MySQL 8.0.23开始是 CHANGE REPLICATION SOURCE TO ，之前是 CHANGE MASTER TO . 这些细节被存储在副本连接的元数据存储中 (see Section 17.2.4, “Relay Log and Replication Metadata Repositories”).

这里有多种不同的方法可以设置副本, 使用某个具体的方法取决于你如何设置以及你是否在源节点已经有想要复制的数据

想要安装多实例的 MySQL, 你可以使用 InnoDB Cluster ，它可以让你使用 MySQL Shell 轻易的管理mysql服务器集群. InnoDB Cluster 用程序化的方式提供了环境包装了MySQL Group Replication，它可以让你轻易的部署一个高可用的Mysql集群。另外 InnoDB Cluster的接口可以无缝集成 MySQL Router, 它可以让你的程序无需编写降级逻辑的情况下连接集群. 对于不需要高可用的相似场景可以使用 InnoDB ReplicaSet. 这里是安装过程here.

这里是一些通用任务来改概括所有步骤:

1.在源节点上开启binlog并配置unique server ID Section 17.1.2.1, “Setting the Replication Source Configuration”.

可以考虑使用 –init-file 执行SET GLOBAL server_id = 2;


SERVER_ID=2
echo "SET GLOBAL server_id = $SERVER_ID;" > /etc/mysql/init.sql
echo "init_file=/etc/mysql/init.sql" >> /etc/mysql/mysql.conf.d/mysqld.cnf
service mysql restart

SERVER_ID=2

echo "SET GLOBAL server_id = $SERVER_ID;" > /etc/mysql/init.sql

echo "init_file=/etc/mysql/init.sql" >> /etc/mysql/mysql.conf.d/mysqld.cnf

service mysql restart

查看这个值


show global variables like 'server_id';

0 1	show global variables like 'server_id';

2.在每个副本上配置a unique server ID. Section 17.1.2.2, “Setting the Replica Configuration”.

重复1

3.可选，为每个副本创建不同的用户用于鉴权 Section 17.1.2.3, “Creating a User for Replication”.


CREATE USER 'repl'@'192.168.2.%' IDENTIFIED BY 'password';
GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.2.%'

CREATE USER 'repl'@'192.168.2.%' IDENTIFIED BY 'password';

GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.2.%'

4.在创建快照或重启副本进程之前，在记录源节点上的binlog当前位置. 当配置你副本节点时你需要这个信息 Section 17.1.2.4, “Obtaining the Replication Source Binary Log Coordinates

在源节点上执行


FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

FLUSH TABLES WITH READ LOCK;

SHOW MASTER STATUS;

记录File | Position

5.如果你在源节点上已经有数据了并且想要同步到副本上，你需要创建数据的快照来复制数据到副本. 你所使用的存储引擎会决定你如何创建快照

当你使用 MyISAM时,你必须在源上停止处理语句来获得read-lock, 在允许源节点继续执行语句之前获取当前binlog的坐标并导出数据。 Section 17.1.2.4, “Obtaining the Replication Source Binary Log Coordinates”.

当你使用 InnoDB, 你不需要获取read-lock，一个够长的事务足够可以传输数据的快照（这么翻译感觉怪怪的，原文： you do not need a read-lock and a transaction that is long enough to transfer the data snapshot is sufficient.） Section 15.19, “InnoDB and MySQL Replication”.

6.在副本上设置连接的源节点，例如host name,、登录凭证、binlog文件名和位置 Section 17.1.2.7, “Setting the Source Configuration on the Replica”.

在副本节点上执行


CHANGE REPLICATION SOURCE TO
SOURCE_HOST='source_host_name',
SOURCE_USER='replication_user_name',
SOURCE_PASSWORD='replication_password',
SOURCE_LOG_FILE='recorded_log_file_name',
SOURCE_LOG_POS=recorded_log_position;

CHANGE REPLICATION SOURCE TO

SOURCE_HOST='source_host_name',

SOURCE_USER='replication_user_name',

SOURCE_PASSWORD='replication_password',

SOURCE_LOG_FILE='recorded_log_file_name',

SOURCE_LOG_POS=recorded_log_position;

7.对源和副本实施特定的安全措施 Section 17.3, “Replication Security”.

基础配置完成后，选择你的场景：

添加一个新的副本并同步空的源节点 Section 17.1.2.6.1, “Setting Up Replication with New Source and Replicas”.
向一个已有的mysql服务器设置新的源Section 17.1.2.6.2, “Setting Up Replication with Existing Data”.
向已有的分布式集群添加副本Section 17.1.2.8, “Adding Replicas to a Replication Environment”.

在管理MySQL集群之前，阅读完整的章节并尝试这里所有的语句，同时熟悉副本的启动选项

Section 13.4.1, “SQL Statements for Controlling Source Servers,

Section 13.4.2, “SQL Statements for Controlling Replica Servers.

Section 17.1.6, “Replication and Binary Logging Options and Variables.

基于GTIDs的方法

TODO

MRG搭建


ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4ZxaNgw6+2u6uQrNik11NZ30FMd7eMB1xGAx2T5VB2651fM8/WVF22iwc1A4BwvWVIPv5lkDIGMlYZ41qsnYZY0pioYkZfq2PnE2WrL/R5XHG7Cnuvt61aqy7Zrdwt/p2WEE/dQHN7U4nYCwGTHLaFz6c0YGtUNHzTuRJqjgQ1yMzqQJj+0oWxyNvuhH3n5ejeJuyWc5cP+O2cYDfCStYpA0KlmDjyaAHaJ2M/n+GYOX9q+4s1mIDsjUoCTWJ5OpBflKiKTPUNCEFH3waFsew2N/WF53gyycYCIc11Bdib7XFiSoydLOGh4JP9NZVNeEl5YMSCoHFbZcWEMSKTpwoFFISnkJPnUnYg0V1NeuA8fhx3b58+/TbdPPMCZIO5TCruCBhJK0UD2D64z167wLm4ZEOh4PLwQIYL6n719XpHdlZBKVv6UOmt+Z3nFH4W5jUNAde/+5r6LC3xg6MJeXTmBEO/omO37HCDCYq+4Wq5xru5EsR1t4rXb0lnZ6zX0= hyl-jumpbox\hyl-jumpbox@HYL-JUMPBOX


sudo apt install net-tools mysql-server -y

ssh mysql1@192.168.2.190
ssh mysql2@192.168.2.169
ssh mysql3@192.168.2.216

sudo cat /etc/mysql/debian.cnf

mysql1
192.168.2.190
user     = debian-sys-maint
password = UYbTJYUnTKzEnerQ

mysql2
192.168.2.169
user     = debian-sys-maint
password = NQTc7vw0eAzqrABq

mysql3
192.168.2.216
user     = debian-sys-maint
password = 0y2FH9aksKPTNqbu

192.168.2.190 mysql1
192.168.2.169 mysql2
192.168.2.216 mysql3

mysql -u debian-sys-maint -p
CREATE USER 'mysql'@'localhost' IDENTIFIED BY 'password';
GRANT ALL ON *.* TO 'mysql'@'localhost';

CREATE USER 'mysql'@'192.168.2.%' IDENTIFIED BY 'password';
GRANT ALL ON *.* TO 'mysql'@'192.168.2.%';
flush privileges;

mysql -u mysql -p
password

mysql -u mysql -h 192.168.2.190 -p
mysql -u mysql -h 192.168.2.169 -p
mysql -u mysql -h 192.168.2.216 -p
password

lisenport从127.0.0.1改为0.0.0.0监听所有网卡，或指定IP远程才可以连接3306


#setenforce=0
#打开/etc/selinux/config文件,修改SELINUX=DISABLED（需重启服务器才能生效）


disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"
server_id=1
gtid_mode=ON
enforce_gtid_consistency=ON
plugin_load_add='group_replication.so'
plugin_load_add='mysql_clone.so'
group_replication_group_name="52d4b7bf-56db-11ed-b0fa-000c29409b22"
group_replication_start_on_boot=off
group_replication_local_address="192.168.2.190:33061"
group_replication_group_seeds="192.168.2.190:33061,192.168.2.169:33061,192.168.2.216:33061"
group_replication_bootstrap_group=off

disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"
server_id=2
gtid_mode=ON
enforce_gtid_consistency=ON
plugin_load_add='group_replication.so'
plugin_load_add='mysql_clone.so'
group_replication_group_name="52d4b7bf-56db-11ed-b0fa-000c29409b22"
group_replication_start_on_boot=off
group_replication_local_address="192.168.2.169:33061"
group_replication_group_seeds="192.168.2.190:33061,192.168.2.169:33061,192.168.2.216:33061"
group_replication_bootstrap_group=off

disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"
server_id=3
gtid_mode=ON
enforce_gtid_consistency=ON
plugin_load_add='group_replication.so'
plugin_load_add='mysql_clone.so'
group_replication_group_name="52d4b7bf-56db-11ed-b0fa-000c29409b22"
group_replication_start_on_boot=off
group_replication_local_address="192.168.2.216:33061"
group_replication_group_seeds="192.168.2.190:33061,192.168.2.169:33061,192.168.2.216:33061"
group_replication_bootstrap_group=off

CREATE USER rpl_user@'%' IDENTIFIED WITH mysql_native_password BY 'password';
SET SQL_LOG_BIN=0;
ALTER USER rpl_user@'%' IDENTIFIED WITH mysql_native_password BY 'password';
GRANT REPLICATION SLAVE ON *.* TO rpl_user@'%';
GRANT CONNECTION_ADMIN ON *.* TO rpl_user@'%';
GRANT BACKUP_ADMIN ON *.* TO rpl_user@'%';
GRANT GROUP_REPLICATION_STREAM ON *.* TO rpl_user@'%';
FLUSH PRIVILEGES;
SET SQL_LOG_BIN=1;

SHOW PLUGINS;

master上启动同步后会开始监听33061端口，注意如果有防火墙需要打开，网上的文章不靠谱，没检查监听就去关防火墙是高危操作！
SET GLOBAL group_replication_bootstrap_group=ON;
START GROUP_REPLICATION USER='rpl_user', PASSWORD='password';
SET GLOBAL group_replication_bootstrap_group=OFF;

SELECT * FROM performance_schema.replication_group_members;

CREATE DATABASE test;
USE test;
CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);
INSERT INTO t1 VALUES (1, 'Luis');
INSERT INTO t1 VALUES (2, 'HylTest');
INSERT INTO t1 VALUES (3, 'HylTest3');
INSERT INTO t1 VALUES (4, 'HylTest4');

select * from test.t1;
SHOW BINLOG EVENTS;
truncate table test.t1;

slave上加入group
CHANGE REPLICATION SOURCE TO SOURCE_USER='rpl_user', SOURCE_PASSWORD='password' FOR CHANNEL 'group_replication_recovery';
START GROUP_REPLICATION USER='rpl_user', PASSWORD='password';

#停止同步
STOP GROUP_REPLICATION;

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4ZxaNgw6+2u6uQrNik11NZ30FMd7eMB1xGAx2T5VB2651fM8/WVF22iwc1A4BwvWVIPv5lkDIGMlYZ41qsnYZY0pioYkZfq2PnE2WrL/R5XHG7Cnuvt61aqy7Zrdwt/p2WEE/dQHN7U4nYCwGTHLaFz6c0YGtUNHzTuRJqjgQ1yMzqQJj+0oWxyNvuhH3n5ejeJuyWc5cP+O2cYDfCStYpA0KlmDjyaAHaJ2M/n+GYOX9q+4s1mIDsjUoCTWJ5OpBflKiKTPUNCEFH3waFsew2N/WF53gyycYCIc11Bdib7XFiSoydLOGh4JP9NZVNeEl5YMSCoHFbZcWEMSKTpwoFFISnkJPnUnYg0V1NeuA8fhx3b58+/TbdPPMCZIO5TCruCBhJK0UD2D64z167wLm4ZEOh4PLwQIYL6n719XpHdlZBKVv6UOmt+Z3nFH4W5jUNAde/+5r6LC3xg6MJeXTmBEO/omO37HCDCYq+4Wq5xru5EsR1t4rXb0lnZ6zX0= hyl-jumpbox\hyl-jumpbox@HYL-JUMPBOX

sudo apt install net-tools mysql-server -y

ssh mysql1@192.168.2.190

ssh mysql2@192.168.2.169

ssh mysql3@192.168.2.216

sudo cat /etc/mysql/debian.cnf

mysql1

192.168.2.190

user = debian-sys-maint

password = UYbTJYUnTKzEnerQ

mysql2

192.168.2.169

user = debian-sys-maint

password = NQTc7vw0eAzqrABq

mysql3

192.168.2.216

user = debian-sys-maint

password = 0y2FH9aksKPTNqbu

192.168.2.190 mysql1

192.168.2.169 mysql2

192.168.2.216 mysql3

mysql -u debian-sys-maint -p

CREATE USER 'mysql'@'localhost' IDENTIFIED BY 'password';

GRANT ALL ON *.* TO 'mysql'@'localhost';

CREATE USER 'mysql'@'192.168.2.%' IDENTIFIED BY 'password';

GRANT ALL ON *.* TO 'mysql'@'192.168.2.%';

flush privileges;

mysql -u mysql -p

password

mysql -u mysql -h 192.168.2.190 -p

mysql -u mysql -h 192.168.2.169 -p

mysql -u mysql -h 192.168.2.216 -p

password

lisenport从127.0.0.1改为0.0.0.0监听所有网卡，或指定IP远程才可以连接3306

#setenforce=0

#打开/etc/selinux/config文件,修改SELINUX=DISABLED（需重启服务器才能生效）

disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"

server_id=1

gtid_mode=ON

enforce_gtid_consistency=ON

plugin_load_add='group_replication.so'

plugin_load_add='mysql_clone.so'

group_replication_group_name="52d4b7bf-56db-11ed-b0fa-000c29409b22"

group_replication_start_on_boot=off

group_replication_local_address="192.168.2.190:33061"

group_replication_group_seeds="192.168.2.190:33061,192.168.2.169:33061,192.168.2.216:33061"

group_replication_bootstrap_group=off

disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"

server_id=2

gtid_mode=ON

enforce_gtid_consistency=ON

plugin_load_add='group_replication.so'

plugin_load_add='mysql_clone.so'

group_replication_group_name="52d4b7bf-56db-11ed-b0fa-000c29409b22"

group_replication_start_on_boot=off

group_replication_local_address="192.168.2.169:33061"

group_replication_group_seeds="192.168.2.190:33061,192.168.2.169:33061,192.168.2.216:33061"

group_replication_bootstrap_group=off

disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY"

server_id=3

gtid_mode=ON

enforce_gtid_consistency=ON

plugin_load_add='group_replication.so'

plugin_load_add='mysql_clone.so'

group_replication_group_name="52d4b7bf-56db-11ed-b0fa-000c29409b22"

group_replication_start_on_boot=off

group_replication_local_address="192.168.2.216:33061"

group_replication_group_seeds="192.168.2.190:33061,192.168.2.169:33061,192.168.2.216:33061"

group_replication_bootstrap_group=off

CREATE USER rpl_user@'%' IDENTIFIED WITH mysql_native_password BY 'password';

SET SQL_LOG_BIN=0;

ALTER USER rpl_user@'%' IDENTIFIED WITH mysql_native_password BY 'password';

GRANT REPLICATION SLAVE ON *.* TO rpl_user@'%';

GRANT CONNECTION_ADMIN ON *.* TO rpl_user@'%';

GRANT BACKUP_ADMIN ON *.* TO rpl_user@'%';

GRANT GROUP_REPLICATION_STREAM ON *.* TO rpl_user@'%';

FLUSH PRIVILEGES;

SET SQL_LOG_BIN=1;

SHOW PLUGINS;

master上启动同步后会开始监听33061端口，注意如果有防火墙需要打开，网上的文章不靠谱，没检查监听就去关防火墙是高危操作！

SET GLOBAL group_replication_bootstrap_group=ON;

START GROUP_REPLICATION USER='rpl_user', PASSWORD='password';

SET GLOBAL group_replication_bootstrap_group=OFF;

SELECT * FROM performance_schema.replication_group_members;

CREATE DATABASE test;

USE test;

CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 TEXT NOT NULL);

INSERT INTO t1 VALUES (1, 'Luis');

INSERT INTO t1 VALUES (2, 'HylTest');

INSERT INTO t1 VALUES (3, 'HylTest3');

INSERT INTO t1 VALUES (4, 'HylTest4');

select * from test.t1;

SHOW BINLOG EVENTS;

truncate table test.t1;

slave上加入group

CHANGE REPLICATION SOURCE TO SOURCE_USER='rpl_user', SOURCE_PASSWORD='password' FOR CHANNEL 'group_replication_recovery';

START GROUP_REPLICATION USER='rpl_user', PASSWORD='password';

#停止同步

STOP GROUP_REPLICATION;

注意，任意节点脱离group后需要手动执行start group_replication重新加入，执行前的数据保留在离开group时的状态

例如mysql2重新加入group变化如下


mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | 5b1e6080-578f-11ed-bd92-000c2956aa71 | mysql1      |        3306 | ONLINE       | PRIMARY     | 8.0.31         | XCom                       |
| group_replication_applier | 5c6b9aaa-578f-11ed-946f-000c2924d927 | mysql2      |        3306 | RECOVERING   | SECONDARY   | 8.0.31         | XCom                       |
| group_replication_applier | 8153354e-578f-11ed-87f7-000c29409b22 | mysql3      |        3306 | ONLINE       | SECONDARY   | 8.0.31         | XCom                       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | 5b1e6080-578f-11ed-bd92-000c2956aa71 | mysql1      |        3306 | ONLINE       | PRIMARY     | 8.0.31         | XCom                       |
| group_replication_applier | 5c6b9aaa-578f-11ed-946f-000c2924d927 | mysql2      |        3306 | ONLINE       | SECONDARY   | 8.0.31         | XCom                       |
| group_replication_applier | 8153354e-578f-11ed-87f7-000c29409b22 | mysql3      |        3306 | ONLINE       | SECONDARY   | 8.0.31         | XCom                       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.01 sec)

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

3 rows in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+

3 rows in set (0.01 sec)

尝试mysql shell自动配置

https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-innodb-cluster.html

参考文档

https://dev.mysql.com/doc/refman/8.0/en/replication.html

https://dev.mysql.com/doc/refman/8.0/en/group-replication-deploying-in-single-primary-mode.html

https://ethendev.github.io/2018/12/17/JPA-MySQL-read-write-separation/

https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-innodb-cluster.html

20221021服务器初始化ilo+esxi+openwert软路由配置

背景

见 https://blog.hylstudio.cn/archives/1022

服务器到货后直播初始化，有同学想看我翻车hhh

网络拓扑设计

连线

略两根电源+3根网线，按设计网络拓扑连接即可

ilo远程接管

网络设置

把电脑的网卡IP设定到和ilo网口相同的子网后可通过443端口进入ilo网页，密码在服务器上印着

进入iLO Dedicated Network Port – IPv4 Settings

勾选 Enable DHCPv4

Use DHCPv4 Supplied Gateway
Use DHCPv4 Supplied Domain Name
Use DHCPv4 Supplied DNS Servers
Use DHCPv4 Supplied WINS Servers

点击Submit后并不会生效，修改时间后一同保存重启ilo

修改BIOS时间

进入iLO Dedicated Network Port – SNTP Settings

修改时间服务器地址为ntp.tencent.com，这里直接用腾讯云的

时区选择GMT+8

点击Submit后并不会生效，按提示点击reset重启ilo后生效

重启过程时间不一定，可以开长ping访问ilo端口的IP，能ping通了网页上也就能登录了

按下图创建自己常用的账户

raid

服务器raid的配置是在服务器启动时候配置，无情开评的小伙伴帮忙配好了raid5，为了防止服务器大晚上起飞还是先不看了

看了下从esxi里已经是一块硬盘了，建立的VMFS6，因此从esxi暴露iSCSI给nas的方案pass。目前的这个配置下会更复杂，如果想省心继续使用群晖系统，还是寻找新的方式从这组raid5上抠出来一块挂到nas上比较方便省事。虽然我继续可以选择iSCSI，但nfs也不是不行，到时候测下速度看看哪个好搞就搞哪个方案，nfs兜底肯定能搞定

远程接管esxi

进入iLO Overview

点击Integrated Remote Console 后面的Java Web Start 进入虚拟控制台

无情开评的小伙伴已经装好了esxi，进入这个页面可看到esxi用的网口和静态IP

按F2进入系统设置，第一次先输入密码，然而看着密码输入框的我一脸懵逼。。大哥并没告诉我esxi密码啊啊啊啊，问了下拿到后才登录。这里差评，到货都3天了都没想起来告诉我一下这个的密码，还要我主动问！！

说好的什么都没配置呢（

第二次会进入这样的页面，按回车修改IP地址后重启管理网即可通过443端口进入esxi管理页面

进入网页后即可正常使用，按下图进入用户添加，输入自己常用的用户名添加身份

进入主机-权限-给自己添加管理员后即可使用自己账户登录

虚拟机

剩下的就是新建虚拟机了，这部分没啥好说的，基本上就是大号的Vmware，会用vmware就会用这个

为了测试虚拟交换机我配置对了没，搞一个虚拟机win10接到不同的虚拟交换机上看看IP就知道了，拓扑看上面的图就行

顺带一提网页版的esxi控制台做的还是很NB的，不知道是不是内网造成的错觉，但感觉远程的体验完爆各种vnc

硬盘速度很给力，装win10进度条从0到100只花了大概2分钟

要是啥硬件装系统都是这个速度，我就再也不怕重装系统折腾了

虚拟交换机

混杂模式是否允许交换机上的虚机看到其他vlan流量

MAC 地址更改是否允许交换机上的虚机修改自己的MAC地址

伪传输懒得看了，总之也是和MAC地址有关系，留给网工的研究去吧，总之除了存储iSCSI之外都选否就对了

按之前的设计有4个子网，但iSCSI的没找到明显入口就先放弃了。默认自带一个虚拟交换机了当作管理网，再新建两个交换机当作公网和测试内网

管理网，IP为0.x

模拟公网，IP为1.x

测试网，IP为2.x

新建跳板机，添加三个网卡分别接到admin public test中，通过控制面板的状态即可看到是否拿到了正确的地址

到这里开始对我来说难度就逐渐下降了，比较软件才是我的专业=-=

断开所有网卡连接后依次连接admin public test，分别测试网络连通性

效果：连接test的网卡一定没网，连接admin和public的网卡和直接把线插到光猫和路由器上一样，只不过是虚拟的

有了跳板机，接下来就可以方便的配置软路由了，因为软路由无实体且方便的配置方法都在网页上，命令行操作不便，一定会借助这个跳板机

目前图里跳板机连接了公网入口，当软路由配置完成后直接断开public的那张网卡连接即可满足我的安全需求

openwrt

固件下载

官网 https://downloads.openwrt.org/releases/22.03.2/targets/x86/64/

lede https://github.com/coolsnowwolf/lede/releases

注意，generic-ext4-combined-efi.img.gz 下载后的img需要用磁盘格式转换工具，如StarWind V2V Conterver

转成vmdk上传到raw下

esxi开启ssh后后执行vmkfstools -i raw/xxx.vmdk xxx.vmdk

raw/xxx.vmdk raw/xxx-flat.vmdk会复制一份到当前目录 xxx.vmdk和xxx-flat.vmdk

用转换后的vmdk新建虚拟机后即可调整磁盘容量

我先后尝试了官网版和lede版本，发现lede版本的功能更多些，虽然不知道用不用的上，但还是用了lede的

按网口顺序增加/etc/config/network的配置，我这里是两个wan 一个lan，观察windows连接到lan网卡是否正常即可完成软路由初始化

中间如果出现问题，配合ping和tracert排除故障即可（这部分才是最恶心的，软工出身的我配置路由器和交换机真的头疼，硬件命令行还好，纯软件模拟的根本搞不清楚对应关系hhhh）

注意下双wan口通过/etc/config/network配置之后有几率无法上网，需要先通过网页上把新加的第二个wan划到防火墙的wan zone里

配置参考如下


root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd82:5306:6840::/48'
        option packet_steering '1'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth2'
        option proto 'static'
        option ipaddr '192.168.2.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option ifname 'eth0'
        option proto 'dhcp'

config interface 'wan6'
        option ifname 'eth0'
        option proto 'dhcpv6'

config interface 'wan2'
        option ifname 'eth1'
        option proto 'dhcp'

config interface 'wan26'
        option ifname 'eth1'
        option proto 'dhcpv6'

config interface 'VPN'
        option ifname 'ipsec0'
        option proto 'static'
        option ipaddr '10.10.10.1'
        option netmask '255.255.255.0'

config interface 'vpn0'
        option ifname 'tun0'
        option proto 'none'

root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'

option ifname 'lo'

option proto 'static'

option ipaddr '127.0.0.1'

option netmask '255.0.0.0'

config globals 'globals'

option ula_prefix 'fd82:5306:6840::/48'

option packet_steering '1'

config interface 'lan'

option type 'bridge'

option ifname 'eth2'

option proto 'static'

option ipaddr '192.168.2.1'

option netmask '255.255.255.0'

option ip6assign '60'

config interface 'wan'

option ifname 'eth0'

option proto 'dhcp'

config interface 'wan6'

option ifname 'eth0'

option proto 'dhcpv6'

config interface 'wan2'

option ifname 'eth1'

option proto 'dhcp'

config interface 'wan26'

option ifname 'eth1'

option proto 'dhcpv6'

config interface 'VPN'

option ifname 'ipsec0'

option proto 'static'

option ipaddr '10.10.10.1'

option netmask '255.255.255.0'

config interface 'vpn0'

option ifname 'tun0'

option proto 'none'

能耗

进入ilo的电源管理，选择Static Low Power Mode

进入esxi的电源管理，选择低能耗

通过ilo可观察服务器功率

nas扩容

居然已经快三点了，还是直播原神日常吧hhhh这次是真日常

见 https://blog.hylstudio.cn/archives/1078

虚机测试环境

有了隔离的子网后，就可以搭建任意的服务器折腾学习了

见后续文章

参考文档

http://hzhcontrols.com/new-206249.html

https://wenku.baidu.com/view/629900cd142ded630b1c59eef8c75fbfc77d94d4.html

20221015新nas&私有云实验环境规划

背景

因原群晖已经试用了3年并且是单盘，考虑到数据风险决定对它做升级。本来在B站无情开评看到的FusionCube 500完美满足我的需求，但询问售价后果断放弃hhhhh根本不是个人用户能玩得起的设备。但也是因此机缘巧合发现无情开评居然承接个人业务需求，他们负责硬件组装和虚拟化平台配置，服务周到省心省力，欢迎有兴趣但懒得自己折腾的同学联系他们帮忙，比自己折腾要方便的多。就这样我又多了一台服务器，在到货之前提前设计下网络结构和软件结构

目标

1.保证家里其他人的IPTV、无线设备的正常使用，即使服务器挂掉也不应该影响原有网络结构

2.加强nas上数据的安全性，重要数据利用nas自带功能备份一份到新的服务器上

3.创建新子网3和4，子网3与光猫相连赋予公网访问权，子网4和公网隔离但能通过子网2访问，虚拟化一个软路由方便控制子网3和4下虚机

4.剩余资源搭建一个分布式的私有实验环境放到子网4，满足我对开源软件和分布式软件开发和测试的需求，解放我的台式机。后面也许就有机会做各种实验了

5.基础网络安全防护，内网随意折腾但从公网无法轻易穿透到内网

网络结构设计

现网络结构

相关设备如下，为了网络安全不暴露具体设备型号信息

1.光猫

2.无线路由器

3.群晖nas

4.iptv

5.手机、电脑、平板等无线客户端

目前nas上和一个台式机上运行着tailscale，配合ipv6可以让我随时从公网穿透回子网2做操作，虽然麻烦了点但足够安全。

其余需要对外暴露的端口均通过光猫和无线路由器人工配置端口和协议范围，可以满足公网上手机和电脑对nas的访问需求

新网络结构

看了下服务器文档，有4个千兆口/可选两个万兆口，~~如果实际到货有2个以上接口的话，可以考虑把nas改为接到服务器下面，剩下的接光猫使用，~~（为了nas稳定使用还是算了，等服务器运行稳定了再考虑把nas挂到服务器下面用软路由精细控制流量）

nas本身的加强因预留了足够的磁盘空间，并且无情开评会帮忙配置好磁盘阵列，因此我只需要划一部分空间给nas就行了。这部分既可以开个linux，也可以到时候优先看看能不能直接在ESXI里通过iSCSI搞定。这样还可以继续使用群晖正版系统，虽然理论上可以把我的白群转到虚拟化里做”黑”群，但我懒得折腾还是算了，等nas的磁盘坏了再考虑

个人电脑都在子网2下操作子网4的路径待设计，想避免出现网络故障还要满足子网4一定的隔离和访问需求，不是网工专业的我设计这玩意真的头疼= = 实在搞不定到时候再问问同学吧hhh

服务器的管理口和esxi的管理口都放到子网2下

服务器软件集群设计

TODO

群晖扩容

远程磁盘创建

TODO in ESXI

远程磁盘挂载

群晖本身的DSM只看到了FTP SFTP WebDav CIFS NFS这几种协议来远程挂载，没看到能把DSM当作iscsi的客户端的，网上能找到的资料都是把DSM当作iscsi服务器来使用。之所以选择iscsi是因为不想再创建一个虚拟机来提供FTP SFTP WebDav CIFS NFS这些协议转换，尽可能使用ESXI本身的虚拟化能力，减少后期维护成本

但尝试登陆群晖命令行后执行iscsiadm 是可以的，因此将尝试直接用命令行挂载

TODO