记录一下跨版本升级Kubernetes的过程

好像Kubernetes只支持逐主版本升级

从1.13.3到1.13.12

Master

apt install kubelet=1.13.12-00 kubeadm=1.13.12-00 kubectl=1.13.12-00
kubeadm upgrade apply v1.13.12

Node

apt install kubelet=1.13.12-00 kubeadm=1.13.12-00 kubectl=1.13.12-00
kubeadm upgrade node config --kubelet-version v1.13.12

从1.13.12到1.14.0

Master

apt install kubelet=1.14.0-00 kubeadm=1.14.0-00 kubectl=1.14.0-00 kubernetes-cni=0.7.5-00
kubeadm upgrade apply v1.14.0

Node

apt install kubelet=1.14.0-00 kubeadm=1.14.0-00 kubectl=1.14.0-00 kubernetes-cni=0.7.5-00
kubeadm upgrade node config --kubelet-version v1.14.0

从1.14.0到1.15.0

Master

apt install kubelet=1.15.0-00 kubeadm=1.15.0-00 kubectl=1.15.0-00
kubeadm upgrade apply v1.15.0

Node

apt install kubelet=1.15.0-00 kubeadm=1.15.0-00 kubectl=1.15.0-00
kubeadm upgrade node config --kubelet-version v1.15.0

好像到这里为止的升级手段都太过于直接粗暴了，应该在升级node之前先把状态置为维护中的

从1.15.0到1.16.0

Master

apt-mark unhold kubeadm kubelet
apt-get install -y kubeadm=1.16.0-00
kubeadm upgrade plan

这里出现了报错

[ERROR CoreDNSUnsupportedPlugins]: there are unsupported plugins in the CoreDNS

也就是coredns中出现了不再被支持的plugin，但是根据官方的issue，这个问题可以忽略

kubeadm upgrade apply v1.16.0 --ignore-preflight-errors=all
apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.16.0-00 kubectl=1.16.0-00 && \\
apt-mark hold kubelet kubectl 
sudo systemctl restart kubelet

Node

首先升级kubeadm

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.15.x-00 && \\
apt-mark hold kubeadm

然后在Master节点上执行cordon命令让节点进入维护状态

kubectl cordon $NODE

之后升级组件，重启服务

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.16.0-00 kubectl=1.16.0-00 && \\
apt-mark hold kubelet kubectl 

kubeadm upgrade node config --kubelet-version v1.16.0

sudo systemctl restart kubelet

最后结束维护状态

kubectl uncordon $NODE

等待节点Ready，发现节点无法ready，查看kubelet的状态发现

 Unable to update cni config: no valid networks found in /etc/cni/net.d

查阅资料得到的结论是flannel的版本也太低了，需要把之前部署的flannel删掉重新部署。在执行delete -f 的时候，发现DaemonSet和PodSecurityPolicy这两类资源已经从extensions/v1beta1移到正式版本里了，所以还需要手动清除这些资源

然后重新部署flannel，之后集群已ready

从1.16到1.17

Master

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.17.0-00 && \\
apt-mark hold kubeadm

kubeadm upgrade apply v1.17.0

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.17.0-00 kubectl=1.17.0-00 && \\
apt-mark hold kubelet kubectl 

sudo systemctl restart kubelet

kubectl cordon $NODE

Node

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.17.0-00 && \\
apt-mark hold kubeadm

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.17.0-00 kubectl=1.17.0-00 && \\
apt-mark hold kubelet kubectl 

kubeadm upgrade node config --kubelet-version v1.17.0

systemctl restart kubelet

Master

kubectl uncordon $NODE

从1.17到1.18

Master

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.18.0-00 && \\
apt-mark hold kubeadm

kubeadm upgrade apply v1.18.0

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.18.0-00 kubectl=1.18.0-00 && \\
apt-mark hold kubelet kubectl 

sudo systemctl restart kubelet

kubectl cordon $NODE

Node

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.18.0-00 && \\
apt-mark hold kubeadm

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.18.0-00 kubectl=1.18.0-00 && \\
apt-mark hold kubelet kubectl 

拉取配置

kubeadm upgrade node config --kubelet-version v1.18.0

发现这个命令已经被取消了，不需要了

systemctl restart kubelet

Master

kubectl uncordon $NODE

从1.18到1.19

Master

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.19.0-00 && \\
apt-mark hold kubeadm

kubeadm upgrade apply v1.19.0

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.19.0-00 kubectl=1.19.0-00 && \\
apt-mark hold kubelet kubectl 

sudo systemctl restart kubelet

kubectl cordon $NODE

Node

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.19.0-00 && \\
apt-mark hold kubeadm

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.19.0-00 kubectl=1.19.0-00 && \\
apt-mark hold kubelet kubectl 

systemctl restart kubelet

Master

kubectl uncordon $NODE

从1.19.0到1.19.6

Master

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.19.6-00 && \\
apt-mark hold kubeadm

kubeadm upgrade apply v1.19.6

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.19.6-00 kubectl=1.19.6-00 && \\
apt-mark hold kubelet kubectl 

sudo systemctl restart kubelet

kubectl cordon $NODE

Node

apt-mark unhold kubeadm && \\
apt-get install -y kubeadm=1.19.6-00 && \\
apt-mark hold kubeadm

apt-mark unhold kubelet kubectl && \\
apt-get install -y kubelet=1.19.6-00 kubectl=1.19.6-00 && \\
apt-mark hold kubelet kubectl 

systemctl restart kubelet

Master

kubectl uncordon $NODE