CoreDNS 其实就是一个 DNS 服务,而 DNS 作为一种常见的服务发现手段,所以很多开源项目以及工程师都会使用 CoreDNS 为集群提供服务发现的功能,Kubernetes 就在集群中使用 CoreDNS 解决服务发现的问题。
如果想要在分布式系统实现服务发现的功能,CoreDNS 其实是一个非常好的选择,CoreDNS作为一个已经进入CNCF并且在Kubernetes中作为DNS服务使用的应用,其本身的稳定性和可用性已经得到了证明,同时它基于插件实现的方式非常轻量并且易于使用,插件链的使用也使得第三方插件的定义变得非常的方便。
Coredns 架构
整个 CoreDNS 服务都建立在一个使用 Go 编写的 HTTP/2 Web 服务器 Caddy 。
集群信息介绍
Kubernetes
v1.13
k8s-master
172.20.101.157
172.20.101.165
172.20.101.164
CLUSTER_CIDR
10.254.0.0/16
CLUSTER_DNS
10.254.0.10
Coredns 项目下载
下载地址1:
wget https://github.com/coredns/deployment/archive/master.zip
unzip master.zip
下载地址2:
git clone https://github.com/coredns/deployment.git
安装部署
确认是否存在已运行dns服务
kubectl get pods -o wide -n=kube-system
#删除命令
kubectl delete --namespace=kube-system deployment ****-dns
安装主目录
cd /workspace/
下载
git clone https://github.com/coredns/deployment.git
安装目录
cd /workspace/deployment/kubernetes
查看项目文件
[root@node01 kubernetes]# ll
CoreDNS-k8s_version.md
coredns.yaml.sed
deploy.sh
README.md
rollback.sh
Scaling_CoreDNS.md
Upgrading_CoreDNS.md
重要文件介绍
deploy.sh 是一个用于在已经运行kube-dns的集群中生成运行CoreDNS部署文件(manifest)的工具脚本。它使用 coredns.yaml.sed文件作为模板,创建一个ConfigMap和CoreDNS的deployment,然后更新集群中已有的kube-dns 服务的selector使用CoreDNS的deployment。重用已有的服务并不会在服务的请求中发生冲突。
deploy 脚本使用方法
usage: ./deploy.sh [ -r REVERSE-CIDR ] [ -i DNS-IP ] [ -d CLUSTER-DOMAIN ] [ -t YAML-TEMPLATE ]
-r : Define a reverse zone for the given CIDR. You may specifcy this option more
than once to add multiple reverse zones. If no reverse CIDRs are defined,
then the default is to handle all reverse zones (i.e. in-addr.arpa and ip6.arpa)
-i : Specify the cluster DNS IP address. If not specificed, the IP address of
the existing "kube-dns" service is used, if present.
-s : Skips the translation of kube-dns configmap to the corresponding CoreDNS Corefile configuration.
Coredns 与 kubernetes 版本匹配:
参考地址:
https://github.com/coredns/deployment/blob/master/kubernetes/CoreDNS-k8s_version.md
Kubernetes v1.14 ==> CoreDNS v1.3.1
Kubernetes v1.13 ==> CoreDNS v1.2.6 <<===本环境使用版本
生成安装配置文件
./deploy.sh -r 10.254.0.0/16 -i 10.254.0.10 -d cluster.local -t coredns.yaml.sed -s >coredns.yaml
验证配置文件核心配置
[root@node01 kubernetes]# more coredns.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: coredns
namespace: kube-system
。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。
---
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health
kubernetes cluster.local 10.254.0.0/16 { 《+====监听域名和CIDR
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
---
。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。
containers:
- name: coredns
image: coredns/coredns:1.3.1 《===修改镜像版本:image: coredns/coredns:1.2.6
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 170Mi
requests:
cpu: 100m
memory: 70Mi
args: [ "-conf", "/etc/coredns/Corefile" ]
volumeMounts:
。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。
执行安装:
kubectl apply -f coredns.yaml
serviceaccount/coredns unchanged
clusterrole.rbac.authorization.k8s.io/system:coredns unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:coredns unchanged
configmap/coredns configured
deployment.apps/coredns configured
service/kube-dns created
直接安装方法
首先要确定使用镜像是对的,执行方法如下:
./deploy.sh -r 10.254.0.0/16 -i 10.254.0.10 -t coredns.yaml -d | kubectl apply -f -
验证服务
kubectl get svc -o wide -n=kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kube-dns ClusterIP 10.254.0.10 <none> 53/UDP,53/TCP,9153/TCP 84s k8s-app=kube-dns
查看 coredns 详细信息
kubectl get pods -o wide -n=kube-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-b97f7df6d-gwjzj 1/1 Running 0 15h 10.254.100.3 172.20.101.166 <none> <none>
coredns-b97f7df6d-jq7q6 1/1 Running 0 15h 10.254.87.3 172.20.101.160 <none>
登陆node节点验证coredns 版本
[root@node04 ~]# docker logs 2076a98b7461
.:53
2019-03-14T10:58:53.9Z [INFO] CoreDNS-1.2.6
2019-03-14T10:58:53.9Z [INFO] linux/amd64, go1.11.2, 756749c
CoreDNS-1.2.6
linux/amd64, go1.11.2, 756749c
[INFO] plugin/reload: Running configuration MD5 = 2f886b3d3ac0d768123559b4705a7dbb
测试DNS解析
修改master节点和所有node节点的/etc/systemd/system/kube-kubelet.service,
修改内容如红色所注,与Corefile中的值对应。
CLUSTER_CIDR
10.254.0.0/16
CLUSTER_DNS
10.254.0.10
修改 kubelet 启动配置文件
vi /k8s/kubnode/cfg/kubelet
KUBELET_OPTS="--logtostderr=true \
--v=4 \
--hostname-override=172.20.101.166 \
--kubeconfig=/k8s/kubnode/cfg/kubelet.kubeconfig \
--bootstrap-kubeconfig=/k8s/kubnode/cfg/bootstrap.kubeconfig \
--config=/k8s/kubnode/cfg/kubelet.config \
--cert-dir=/k8s/kubnode/ssl \
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"
#添加内容如下
--cluster-dns=10.254.0.10 \
--cluster-domain=cluster.local.
#或者添加配置到 /etc/systemd/system/kube-kubelet.service
### 重启 kubelet 服务
systemctl daemon-reload
systemctl enable kubelet
systemctl restart kubelet
systemctl status kubelet -l
启动 nginx 测试服务
kubectl run nginx --replicas=2 --image=nginx:alpine --port=80
kubectl expose deployment nginx --type=NodePort --name=example-service-nodeport
kubectl expose deployment nginx --name=example-service
启动一个工具镜像
kubectl run curl --image=radial/busyboxplus:curl
验证服务启动成功
kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
curl-66959f6557-879c6 1/1 Running 0 4m52s 10.254.87.4 172.20.101.160 <none> <none>
nginx-665764c8c9-dgjgv 1/1 Running 0 7m19s 10.254.100.2 172.20.101.166 <none> <none>
nginx-665764c8c9-z9rrm 1/1 Running 0 7m19s 10.254.87.2 172.20.101.160 <none> <none>
[root@node01 kubernetes]#
kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
example-service ClusterIP 10.254.210.43 <none> 80/TCP 9s run=nginx
example-service-nodeport NodePort 10.254.204.43 <none> 80:36136/TCP 14s run=nginx
kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 105m <none>
登陆curl镜像 (node节点执行)
[root@node04 ~]# docker exec -it f18900873efe sh
验证外网解析
ping qq.com
PING qq.com (111.161.64.48): 56 data bytes
64 bytes from 111.161.64.48: seq=0 ttl=47 time=6.331 ms
64 bytes from 111.161.64.48: seq=1 ttl=47 time=6.293 ms
验证内部解析:
[ root@curl-66959f6557-879c6:/ ]$ nslookup kubernetes
Server: 10.254.0.10
Address 1: 10.254.0.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes
Address 1: 10.254.0.1 kubernetes.default.svc.cluster.local
使用 curl测试域名解析
[ root@curl-66959f6557-879c6:/ ]$
curl example-service
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h2>Welcome to nginx!</h2>
...........................
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
清理服务
删除测试服务
kubectl get svc
kubectl delete svc example-service example-service-nodeport
删除部署空间
kubectl get deployment
kubectl delete deploy nginx curl
如果想删除 coredns执行如下操作:
kubectl delete svc kube-dns -n=kube-system
curl 报错 排查解决
[root@node01 kubernetes]#
curl example-service
curl: (6) Could not resolve host: example-service; Unknown error
[root@node01 kubernetes]# curl example-service
解决办法:
修改 kubelet 启动配置文件 (node节点)
vi /k8s/kubnode/cfg/kubelet
KUBELET_OPTS="--logtostderr=true \
--v=4 \
--hostname-override=172.20.101.166 \
--kubeconfig=/k8s/kubnode/cfg/kubelet.kubeconfig \
--bootstrap-kubeconfig=/k8s/kubnode/cfg/bootstrap.kubeconfig \
--config=/k8s/kubnode/cfg/kubelet.config \
--cert-dir=/k8s/kubnode/ssl \
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"
#添加内容如下
--cluster-dns=10.3.0.10 \
--cluster-domain=cluster.local.
#或者添加配置到 /etc/systemd/system/kube-kubelet.service
重启 kubelet 服务
systemctl daemon-reload
systemctl enable kubelet
systemctl restart kubelet
systemctl status kubelet -l
报错2:
[root@node01 coredns]# ./deploy.sh 10.254.0.0/16 cluster.local | kubectl apply -f -
Error from server (NotFound): services "kube-dns" not found
error: no objects passed to apply
报错:
./deploy.sh: line 39: jq: command not found
解决办法:
因为项目使用了jq命令,需要安装jq程序
yum -y install jq conntrack-tools
参考资料:
https://juejin.im/post/5b45cea9f265da0f652370ce#heading-33
https://www.cnblogs.com/boshen-hzb/p/7511432.html
http://lizhe.name/node/326
https://github.com/minminmsn/k8s1.13/blob/master/kubernetes/kubernetes1.13.1%2Betcd3.3.10%2Bflanneld0.10%E9%9B%86%E7%BE%A4%E9%83%A8%E7%BD%B2.md