[Ingress] minikube addons enable ingress 失败

[916980name]

$ minikube addons enable ingress
💡  ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
    ▪ Using image registry.k8s.io/ingress-nginx/controller:v1.7.0
    ▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
    ▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
🔎  Verifying ingress addon...

❌  Exiting due to MK_ADDON_ENABLE: enable failed: run callbacks: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: timed out waiting for the condition]

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│    Please also attach the following file to the GitHub issue:                             │
│    - /tmp/minikube_addons_4e79cb121815fc130446243219e97fe20a361239_0.log                  │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

我看有人说还是gcr的问题，但是有个一句话解决方案，不知道有没有同样问题的，希望能节省你的时间：
kubernetes/minikube#10544 (comment)

此时可以通过minikube ip的IP来访问，如http://192.168.49.2/
但是

想要给本地网络提供服务

尝试

$ kubectl port-forward ingress-nginx-controller-648cf4cd7d-dqb4j -n ingress-nginx --address 0.0.0.0 80:80
Unable to listen on port 80: Listeners failed to create with the following errors: [unable to create listener: Error listen tcp4 0.0.0.0:80: bind: permission denied]
error: unable to listen on any of the requested ports: [{80 80}]

加sudo

$ sudo kubectl port-forward ingress-nginx-controller-648cf4cd7d-dqb4j -n ingress-nginx --address 0.0.0.0 80:80
[sudo] password for xxxx: 
E0719 20:31:56.578262  850892 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0719 20:31:56.578542  850892 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0719 20:31:56.579715  850892 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0719 20:31:56.580944  850892 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
The connection to the server localhost:8080 was refused - did you specify the right host or port?

行80不给用是吧，换个高位端口成功：

kubectl port-forward ingress-nginx-controller-648cf4cd7d-dqb4j -n ingress-nginx --address 0.0.0.0 18080:80

此时可以通过本地网卡的IP来访问，有线无线回环皆可，如http://192.168.0.200:18080/hello
但是，我今天非要用上80

非要用上80端口

前面加个nginx吧

server {
    listen       80;
    listen  [::]:80;
    server_name  localhost;

    location / {
              proxy_redirect off;
              proxy_pass http://172.17.0.1:18080;
              proxy_set_header Host $host;
              proxy_set_header X-Real-IP $remote_addr;
              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_read_timeout 36000s;

    }
}

curl一下

$ curl http://192.168.0.200/hello
[v3] Hello, Kubernetes!, From host: hellok8s-deployment-75f68fb95d-xkb6m

舒服了

[916980name]

如果通过ingress访问前后端分离项目

前端react+nginx docker，后端java+jdk docker
那么可能会产生如下ingress配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: qa-ingress
  namespace: qa
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - http:
        paths:
          - path: /api(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: backend-clusterip
                port:
                  number: 8080
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend-clusterip
                port:
                  number: 80

如果是想要通过path rewrite进行分离的项目，写成这样，这个rewrite也会影响frontend。
导致前端请求被重写成/，表现为页面请求全200， 但是返回的都是同样的结果(index.html)，白屏。
在一度怀疑了前端项目的各种配置不限于react-router，package.json hompage，webpack之后
中间改成这样

...
...
    # nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/configuration-snippet: |-
      rewrite /api/(.+)$ /$1 break;
spec:
  ingressClassName: nginx
  rules:
    - http:
        paths:
          - path: /api
...
...

[Eyjafjalla6]

如果使用ingress-nginx，需要替换yaml中的镜像，否则报unauthenticated

#删除之前创建的deployment,service
kubectl delete deployment,service --all

1. 重新下载官方文件，然后替换其中的镜像

rm -f deploy.yaml
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/baremetal/deploy.yaml

一键替换所有官方镜像源为 daocloud 代理源

sed -i 's|registry.k8s.io|k8s.m.daocloud.io|g' deploy.yaml

替换主控制器镜像 (v1.11.2 是目前 K8s 1.30+ 最稳定的版本)

sed -i 's|docker.io/liubog2008/ingress-nginx-controller:v1.14.3|dyrnq/ingress-nginx-controller:v1.11.2|g' deploy.yaml

替换证书生成器镜像 (admission-create 和 admission-patch 使用同一个)

sed -i 's|docker.io/liubog2008/kube-webhook-certgen:v1.6.7|dyrnq/kube-webhook-certgen:v1.4.1|g' deploy.yaml

彻底删除所有的 @sha256 校验码 (这是镜像拉取成功的关键)

sed -i 's|@sha256:[a-z0-9]*||g' deploy.yaml

kubectl apply -f deploy.yaml
kubectl get pods -n ingress-nginx
#NAME READY STATUS RESTARTS AGE
#ingress-nginx-controller-86c8d7b99f-5m8hq 1/1 Running 0 82s

#如果pod启动失败，需要彻底删除
kubectl delete -f deploy.yaml
kubectl get ns ingress-nginx

如果还有东西，说明删除不彻底，要继续删

开启一个临时代理，以便我们可以直接访问 Kubernetes API

执行后这个命令会挂起，请保持这个终端不要关闭，或者在后面加个 & 符号让他后台运行

kubectl proxy &

1. 导出 JSON

kubectl get namespace ingress-nginx -o json > tmp.json

2. 修改 JSON (利用 sed 删除 spec 里的 finalizers 这一行)

注意：我们要把 "finalizers": ["kubernetes"] 变成 "finalizers": []

sed -i 's/"kubernetes"//g' tmp.json
curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/ingress-nginx/finalize
kubectl get ns ingress-nginx #应该删掉了