life

the site subtitle

k8s 学习

2017.11.15

概念

kube-dns

  • 解决的问题
不是kubernetes中必须安装的，但是推荐安装

虽然kubernetes中有service的概念，访问服务可以访问端口，
但是如果是一个app，一个db，
链接db时需要通过增加app查询ip的能力，需要重复的开发，修改应用

刚开始的做法采用了docker使用过的方法--环境变量。
每个pod启动时，会通过环境变量设置所有服务的ip与port，
这样pod中的应用可以启动时，读取环境变量来获取依赖服务的地址信息
，但是，一个问题是，必须依赖服务先启动才可以正常使用

最理想的方案就是直接使用服务的名字，不需要关心它实际的ip，即dns系统。
kubernetes 也提供了这种功能
  • 存在的形态

DNS服务不是独立的系统，是一种 addon 作为插件来安装，可以把它看做运行在集群上的应用。

DNS 有两种配置方式: - 在1.3之前使用 etcd + kube2sky + skydns 的方式

会有三个容器运行:

etcd: 保存所有DNS数据

kube2sky: 通过kubernetes API 监听 Service 的变化， 然后同步到 etcd

skyDNS: 根据etcd中的数据，对外提供DNS查询服务

  • 在1.3之后可以使用 kubedns + dnsmasq 的方式

kubedns: 提供原来kube2sky + etcd + skyDNS 的功能，可以单独对外提供DNS查询服务

dnsmasq: 轻量级的DNS服务软件， 可以提供DNS缓存功能。 kubeDNS模式下，dnsmasq正在内存这种预留一块大小1G（默认）的地方，保存当前最常用的DNS查询记录， 如果缓存中没有要查找的记录，就会到kubeDNS中查询，并把结果缓存起来.

离线安装

环境

  • vagrant 虚拟三台机器
  • centos7.2

步骤

  1. 指定gcr.io的host，如果不指定需要手动下载google-containers
 61.91.161.217 google.com
 61.91.161.217 gcr.io
 61.91.161.217 www.gcr.io
 61.91.161.217 console.cloud.google.com
 61.91.161.217 storage.googleapis.com
  1. 关闭防火墙
$ vi /etc/selinux/config
SELINUX=permissive

$ setenforce 0
$ getenforce
Ppermission
$ systemctl stop firewalld
$ systemctl disable firewalld
  1. Docker会随着kubernetes安装，删除本地docker
$ yum list installed | docker
$ yum remove -y docekr-engine.x86_64 docekr-engine-selinux.noarch
  1. 开始安装kubernetes,离线源采用cbs.centos.org/repos/$tag/x86_64/os/Packages/ tag=virt7-docker-common-candidate
# 删除本机repo
$ rm -rf /etc/yum.repos.d/$tag.repo
$ yum install -y createrepo
$ mkdir -p /data/softs/localyum 将下载的软件全部放入该文件夹内
$ createrepo -v /data/softs/localyum

# enable=1 表示启用这个更新库
# gpgcheck=1 表示使用gpg文件检查软件包签名
# gpgkey= 表示gpg文件所在位置，此处也可以有http方式的位置
$ cat << EOF >> /etc/yum.repos.d/local.repo
[local]
name=local
baseurl=file:///data/softs/localyum
enable=1
gpgcheck=0 
EOF
$ yum clean all
$ yum makecache
$ yum install -y kubernetes flannel etcd
  1. 配置docker
# 修改docker配置 sed -i:改变输入文件 -r:使用正则表达式
$ rm -rf /etc/systemd/system/docker.service.d/docker.conf
$ cat /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target docker-containerd.service
Wants=docker-storage-setup.service
Requires=docker-containerd.service rhel-push-plugin.socket

[Service]
Type=notify
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
ExecStart=/usr/bin/dockerd-current \
          --add-runtime oci=/usr/libexec/docker/docker-runc-current \
          --default-runtime=oci \
          --authorization-plugin=rhel-push-plugin \
          --containerd /run/containerd.sock \
          --exec-opt native.cgroupdriver=systemd \
          --userland-proxy-path=/usr/libexec/docker/docker-proxy-current \
          $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $ADD_REGISTRY \
          $BLOCK_REGISTRY \
          $INSECURE_REGISTRY
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

$ sed -ir 's/^OPTION.*/OPTIONS=''"'"'--selinux-enabled --log-driver=journald --graph=\/data\/env\/docker --insecure-registry master:5000'"'"'/' /etc/sysconfig/docker
$ mkdir -p /data/env/docker
$ systemctl enable docker && systemctl restart docker 
# 查询docker状态
$ systemctl status docker
  1. 配置kubernetes
# 备份kubernetes配置
$ mv /etc/kubernetes/config /etc/kubernetes/config.bak
$ vi /etc/kubernetes/config
KUBE_ETCD_SERVERS="--etcd-servers=http://kubemaster:2379"
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://kubemaster:8080"
  1. master 配置etcd, kubernetes, flanneld, kube-apiserver， kube-controller-manager, kube-scheduler
$ mv /etc/etcd/etcd.conf /etc/etcd/etcd.conf.bak
$ vi /etc/etcd/etcd.conf
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"

#[cluster]
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"

$ mv /etc/kubernetes/apiserver /etc/kubernetes/apiserver.bak
$ vi /etc/kubernetes/apiserver
# The address on the local server to listen to.
KUBE_API_ADDRESS="--address=0.0.0.0"

# The port on the local server to listen on.
KUBE_API_PORT="--port=8080"

# Port kubelets listen on
KUBELET_PORT="--kubelet-port=10250"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"

# Add your own!
KUBE_API_ARGS=""

$ systemctl start etcd
$ etcdctl mkdir /kube-centos/network
$ etcdctl mk /kube-centos/network/config "{ \"Network\": \"172.30.0.0/16\", \"SubnetLen\": 24, \"Backend\": { \"Type\": \"vxlan\" } }" 
{ "Network": "172.30.0.0/16", "SubnetLen": 24, "Backend": { "Type": "vxlan" } }

$ mv /etc/sysconfig/flanneld/ /etc/sysconfig/flanneld.bak
$ vi /etc/sysconfig/flanneld
# Flanneld configuration options

# etcd url location.  Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://kubemaster:2379"

# etcd config key.  This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/kube-centos/network"

# Any additional options that you want to pass
#FLANNEL_OPTIONS=""

$ for SERVICES in etcd kube-apiserver kube-controller-manager kube-cheduler flanneld; do
  systemctl restart $SERVICES
  systemctl enable $SERVICES
  systemctl status $SERVICES
done
  1. node节点上配置kubernetes(every node)
$ mv /etc/kubernetes/kubelet /etc/kubernetes/kubelet.bak
$ vi /etc/kubernetes/kubelet 
# The address for the info server to serve on
KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
# Check the node number!
KUBELET_HOSTNAME="--hostname-override=kubenode0"

# Location of the api-server
KUBELET_API_SERVER="--api-servers=http://kubemaster:8080"

# Add your own!
KUBELET_ARGS=""
  1. node节点上配置flanneld
$ mv /etc/sysconfig/flanneld /etc/sysconfig/flanneld.bak
$ vi /etc/sysconfig/flanneld
# Flanneld configuration options

# etcd url location.  Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://kubemaster:2379"

# etcd config key.  This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/kube-centos/network"

# Any additional options that you want to pass
#FLANNEL_OPTIONS=""
  1. node节点上启动服务
$ for SERVICES in kube-proxy kubelet flanneld docker; do
  systemctl restart $SERVICES
  systemctl enable $SERVICES
  systemctl status $SERVICES
done

$ ps -ef | grep kube
root       2976      1  3 15:10 ?        00:00:00 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://kubemaster:8080
root       3063      1  0 15:10 ?        00:00:00 /usr/bin/flanneld -etcd-endpoints=http://kubemaster:2379 -etcd-prefix=/kube-centos/network
root       3214      1  5 15:10 ?        00:00:00 /usr/bin/kubelet --logtostderr=true --v=0 --api-servers=http://kubemaster:8080 --address=0.0.0.0 --port=10250 --hostname-override=kubenode0 --allow-privileged=false
root       3357   2569  0 15:10 pts/0    00:00:00 grep --color=auto kube

# Configure kubectl 
$ kubectl config set-cluster default-cluster --server=http://kubemaster:8080
$ kubectl config set-context default-context --cluster=default-cluster  --user=default-admin
$ kubectl config use-context default-context
  1. 查看集群状态
$ kubectl cluster-info
Kubernetes master is running at http://localhost:8080

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

$ kubectl get deployment --namespace=kube-system
No resources found.

# exec every machine useful
$ kubectl get nodes
NAME        STATUS    AGE
kubenode0   Ready     4m
kubenode1   Ready     2m

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-19T19:39:41Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-19T19:39:41Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}


现在你可以用windows访问：http://192.168.1.158:8080/api/:
{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "192.168.1.158:6443"
    }
  ]
}

访问https://192.168.1.158:6443/api:

{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "192.168.1.158:6443"
    }
  ]
}

参考资料

kubernetes 简介：kube-dns 和服务发现

#tomorrow continue

遇到问题

  • ​
发表评论
△