k8s 安装ingress nginx controller 并部署.net core ingress服务

192.168.28.132 k8smaster
192.168.28.133 k8snode1
192.168.28.134 k8snode2
192.168.28.135 k8snode3
192.168.28.131 www.img.com

www.img.com 是harbor 镜像仓库

k8s (kubectl version)版本是 1.23

准备安装ingress-nginx/controller-v1.1.0

一、修改apiserver 启动参数

vi /etc/kubernetes/manifests/kube-apiserver.yaml

在“enable-admission-plugins=NodeRestriction”后面添加 “,MutatingAdmissionWebhook,ValidatingAdmissionWebhook” 参数

--enable-admission-plugins=NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook

二、下载ingress-nginx controller：

https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.0/deploy/static/provider/cloud/deploy.yaml

(下载不了，可能需要科学上网或绑定hosts)

复制一份，命名为“deploy-v1.1.0.yaml”， 并修改“deploy-v1.1.0.yaml”， 搜索Deployment （dnsPolicy）并修改，带“#”号的是修改点，

1. dnsPolicy换成ClusterFirstWithHostNet
2. 新加 hostNetwork: true
3. 新加 nodeName: k8snode1 ，指定部署到k8snode1
4. 新加

tolerations: 
      - key: node-role.kubernetes.io/master
        operator: Exists

“deploy-v1.1.0.yaml” - Deployment 局部修改示例：

  template:            
    metadata:          
      labels:          
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:              
      dnsPolicy: ClusterFirstWithHostNet  #既能使用宿主机DNS，又能使用集群DNS
      hostNetwork: true                   #与宿主机共享网络
      nodeName: k8snode1              #设置只能在k8snode1节点运行
      tolerations:  					  #设置能容忍master污点
      - key: node-role.kubernetes.io/master
        operator: Exists
      containers:   
        - name: controller
          image: k8s.gcr.io/ingress-nginx/controller:v1.1.0@sha256:f766669fdcf3dc26347ed273a55e754b427eb4411ee075a53f30718b4499076a
          imagePullPolicy: IfNotPresent

三、安装ingress-nginx-controller：

kubectl apply -f deploy-v1.1.0.yaml

(镜像拉不下来时，可能需要科学上网)

查看状态

kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx -o wide

NAME                                        READY   STATUS      RESTARTS   AGE   IP               NODE       NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-rnfvr        0/1     Completed   0          98m   10.244.3.67      k8snode3   <none>           <none>
ingress-nginx-admission-patch-mzsv2         0/1     Completed   0          98m   10.244.2.6       k8snode2   <none>           <none>
ingress-nginx-controller-648c6ccb64-lnstv   1/1     Running     0          98m   192.168.28.133   k8snode1   <none>           <none>

可以看到nginx-controller部署到了node1,内网IP是：192.168.28.133

如果一直是非“Running”，可以用下面的命令查看pod状态，然后百度解决

kubectl describe pod ingress-nginx-controller-648c6ccb64-lnstv --namespace=ingress-nginx

到node-1上看下本地端口： netstat -lntup | grep nginx

tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      7385/nginx: worker  
tcp        0      0 127.0.0.1:10245         0.0.0.0:*               LISTEN      55267/nginx-ingress 
tcp        0      0 127.0.0.1:10246         0.0.0.0:*               LISTEN      7382/nginx: worker  
tcp        0      0 127.0.0.1:10247         0.0.0.0:*               LISTEN      7382/nginx: worker  
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      7385/nginx: worker  
tcp        0      0 0.0.0.0:8181            0.0.0.0:*               LISTEN      7385/nginx: worker  
tcp6       0      0 :::8443                 :::*                    LISTEN      55267/nginx-ingress 
tcp6       0      0 :::443                  :::*                    LISTEN      7382/nginx: worker  
tcp6       0      0 :::10254                :::*                    LISTEN      55267/nginx-ingress 
tcp6       0      0 :::80                   :::*                    LISTEN      7382/nginx: worker  
tcp6       0      0 :::8181                 :::*                    LISTEN      7382/nginx: worker  

四、创建 ingress 服务：

用到的.net core mvc Dockerfile等

打包.net core mvc 镜像

Dockerfile：

#添加.net6基础镜像
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app

# 一般情况下必须开放
EXPOSE 80
# 如果使用https，记得打开443端口，但是一般不用
#EXPOSE 443

#修改为上海时区
RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime 
COPY publish/ /app
ENTRYPOINT ["dotnet", "net6mvc.dll"]

k8s 上部署 .net core mvc 服务

web-pod.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: net6mvc
  labels:
    k8s-app: net6mvc
spec:
  replicas: 3
  selector:
    matchLabels:
      k8s-app: net6mvc
  template:
    metadata:
      labels:
        k8s-app: net6mvc
    spec:
      containers:
      - name: net6mvc
        image: www.img.com/library/net6-mvc:1.0
        imagePullPolicy: Always
        ports:
        - containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: net6mvc
  name: net6mvc
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
    nodePort: 32143
  selector:
    k8s-app: net6mvc

k8s 上部署 ingress 服务

ingress-net6mvc.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-net6mvc
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
  - host: www.net6mvc.com
    http:
      paths:
      - path: "/"
        pathType: Prefix
        backend:
          service:
            name: net6mvc
            port:
              number: 80

www.net6mvc.com 是随便起的域名。

docker build -t www.img.com/library/net6-mvc:1.0 .
docker push www.img.com/library/net6-mvc:1.0
docker rmi www.img.com/library/net6-mvc:1.0

kubectl apply -f web-pod.yaml

kubectl apply -f ingress-net6mvc.yaml

查看ingress-nginx 的内网工作端口 kubectl get svc -n ingress-nginx

[root@k8smaster home]# kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.97.204.228    <pending>     80:32048/TCP,443:32094/TCP   115m
ingress-nginx-controller-admission   ClusterIP      10.108.100.124   <none>        443/TCP                      115m

可以看到nginx-controller http端口是：32048

C:\windows\system32\drivers\etc\，绑定hosts后，

192.168.28.133 www.net6mvc.com

浏览器访问：

http://www.net6mvc.com:32048/