kubeadm 重新生成证书

故障现象:

kubernetes 运行了一段时间后,集群出现异常通过,执行命令journalctl -xe -f 发现以下日志内容

Unable to connect to the server: x509: certificate has expired or is not yet valid

Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]


failed to check the health of member 6c70a880257288f on https://10.19.10.12:2379: Get https://10.19.10.12:2379/health: remote error: tls: bad certificate

故障排查

根据日志内容排查判断为证书过期,之后去查看证书有效期

$:/etc/kubernetes.bak/pki# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep "Not "
Not Before: Apr 15 02:36:33 2019 GMT
Not After : Apr 14 10:49:08 2020 GMT

发现集群异常的时间,恰好是证书过期的时间.后通过github查询得到结果,kubeadm默认证书签名时间为一年。

故障解决

检查配置文件

生成kubeadm.conf

kubeadm config print init-defaults > kubeadm.conf

检查 kubeadm.conf

apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.13.0  # kubernetes 版本
apiServer:
  certSANs:
  - 192.168.0.118 # 所有节点IP地址
  extraArgs:
    service-node-port-range: 80-32767
    advertise-address: 0.0.0.0
controlPlaneEndpoint: "10.19.10.15:16443"  # APIserver 地址
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers #这里使用国内的镜像仓库,否则在重新签发的时候会报错:could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt"

生成证书与配置文件

备份现有证书

mv /etc/kubernetes/pki /etc/kubernetes/pki.old

生成新的证书

kubeadm alpha phase certs all --config  ~/kubeadm.yaml

生成新的配置文件

备份现有conf 配置

find /etc/kubernetes/ -name "*.conf" -exec mv {} {}.old \;

生成新的配置文件

kubeadm init phase  kubeconfig all  --config /root/kubeadm-config.yaml

将新生成的admin配置文件覆盖掉原本的admin文件

mv $HOME/.kube/config $HOME/.kube/config.old
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
sudo chmod 777 $HOME/.kube/config

重启 kube-apiserver,kube-controller,kube-scheduler 组件

更新 ETCD 证书

编辑 etcd 配置etcd-config.yaml

apiVersion: "kubeadm.k8s.io/v1beta1"
kind: ClusterConfiguration
etcd:
  local:
    serverCertSANs:
      - "10.19.10.12" # etcd 所有节点列表

    peerCertSANs:
      - "10.19.10.12" # etcd 所有节点列表

生成 server 和 peer 证书

kubeadm init phase certs etcd-ca --config tcd-config.yaml
kubeadm init phase certs etcd-server --config tcd-config.yaml
kubeadm init phase certs etcd-peer --config tcd-config.yaml
kubeadm init phase certs etcd-healthcheck-client --config tcd-config.yaml
kubeadm init phase certs apiserver-etcd-client --config tcd-config.yaml

将证书分发至所有服务器。至此解决 kubeadm 集群证书过期问题

未经允许不得转载:99ya » kubeadm 重新生成证书