### What happened?


---

I have set up a cluster and deployed our production application. However, there is an issue with connectivity between microservices. Upon investigation, I found that our services cannot access the Eureka registry service using the internal cluster domain name.

The Eureka service is deployed on the nb-17-8 server. Pods deployed on the same server, nb-17-8, can access Eureka using DNS. For example, the three replicas of Eureka need to communicate with each other for synchronization (which works fine):

```
ndsp          eureka-0                                       1/1     Running                 1          71m     10.244.86.42    nb-17-8     <none>           <none>
ndsp          eureka-1                                       1/1     Running                 1          71m     10.244.86.44    nb-17-8     <none>           <none>
ndsp          eureka-2                                       1/1     Running                 1          71m     10.244.86.45    nb-17-8     <none>           <none>
```

Within the Eureka pods on nb-17-8, they cannot access 10.96.0.1 or 10.96.0.10:

```
kubectl exec -it -n ndsp eureka-0 -- sh
/ # nc -v 10.96.0.1 443
10.96.0.1 (10.96.0.1:443) open
^Cpunt!

/ # nc -v  10.96.0.10 53
10.96.0.10 (10.96.0.10:53) open
^Cpunt!
```

On other hosts, the Service IPs are accessible:

```
[root@nb-17-57 ~]#  nc -v 10.96.0.1 443
Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Connected to 10.96.0.1:443.
^C
[root@nb-17-57 ~]#  nc -v 10.96.0.10 53
Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Connected to 10.96.0.10:53.
```

On the hosts, accessing the Eureka pod via domain name is not possible, but using the Pod IP works (from all hosts and pods):

```
[root@nb-17-57 ~]# curl -I 10.244.86.42:8761
HTTP/1.1 200 
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Transfer-Encoding: chunked
Date: Wed, 29 May 2024 11:14:14 GMT
```

In the CoreDNS pod, there are the following error messages:

```
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:44838->8.8.8.8:53: i/o timeout
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:51686->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:54068->8.8.4.4:53: i/o timeout
```

All hosts have the following `resolv.conf` file content (since it's a closed internal network, domain name resolution uses the hosts file and there is no internal DNS server):

```
cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
```

Does anyone have any good solutions?

---

### What did you expect to happen?

Good DNS resolution



### How can we reproduce it (as minimally and precisely as possible)?

I don’t know, the environment is a big problem

### Anything else we need to know?

_No response_

### Kubernetes version

Kubernetes version: 1.20.0

### Cloud provider

<details>

</details>


### OS version

Host OS: BigCloud Enterprise Linux release 8.2.2107 (Core) （Like Centos）


### Install tools

Kubeadm

### Container runtime (CRI) and version (if applicable)

Docker  19.03.5

### Related plugins (CNI, CSI, ...) and versions (if applicable)

CNI and version: flannel v0.11.0
DNS resolution fails within the cluster, and it can only resolve the Pods deployed on the same host
This is a GitHub Issue
repo:kubernetes
owner:kubernetes
Title : DNS resolution fails within the cluster, and it can only resolve the Pods deployed on the same host
Issue date:

--- start body ---
### What happened?


---

I have set up a cluster and deployed our production application. However, there is an issue with connectivity between microservices. Upon investigation, I found that our services cannot access the Eureka registry service using the internal cluster domain name.

The Eureka service is deployed on the nb-17-8 server. Pods deployed on the same server, nb-17-8, can access Eureka using DNS. For example, the three replicas of Eureka need to communicate with each other for synchronization (which works fine):

```
ndsp          eureka-0                                       1/1     Running                 1          71m     10.244.86.42    nb-17-8     <none>           <none>
ndsp          eureka-1                                       1/1     Running                 1          71m     10.244.86.44    nb-17-8     <none>           <none>
ndsp          eureka-2                                       1/1     Running                 1          71m     10.244.86.45    nb-17-8     <none>           <none>
```

Within the Eureka pods on nb-17-8, they cannot access 10.96.0.1 or 10.96.0.10:

```
kubectl exec -it -n ndsp eureka-0 -- sh
/ # nc -v 10.96.0.1 443
10.96.0.1 (10.96.0.1:443) open
^Cpunt!

/ # nc -v  10.96.0.10 53
10.96.0.10 (10.96.0.10:53) open
^Cpunt!
```

On other hosts, the Service IPs are accessible:

```
[root@nb-17-57 ~]#  nc -v 10.96.0.1 443
Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Connected to 10.96.0.1:443.
^C
[root@nb-17-57 ~]#  nc -v 10.96.0.10 53
Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Connected to 10.96.0.10:53.
```

On the hosts, accessing the Eureka pod via domain name is not possible, but using the Pod IP works (from all hosts and pods):

```
[root@nb-17-57 ~]# curl -I 10.244.86.42:8761
HTTP/1.1 200 
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Transfer-Encoding: chunked
Date: Wed, 29 May 2024 11:14:14 GMT
```

In the CoreDNS pod, there are the following error messages:

```
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:44838->8.8.8.8:53: i/o timeout
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:51686->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:54068->8.8.4.4:53: i/o timeout
```

All hosts have the following `resolv.conf` file content (since it's a closed internal network, domain name resolution uses the hosts file and there is no internal DNS server):

```
cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
```

Does anyone have any good solutions?

---

### What did you expect to happen?

Good DNS resolution



### How can we reproduce it (as minimally and precisely as possible)?

I don’t know, the environment is a big problem

### Anything else we need to know?

_No response_

### Kubernetes version

Kubernetes version: 1.20.0

### Cloud provider

<details>

</details>


### OS version

Host OS: BigCloud Enterprise Linux release 8.2.2107 (Core) （Like Centos）


### Install tools

Kubeadm

### Container runtime (CRI) and version (if applicable)

Docker  19.03.5

### Related plugins (CNI, CSI, ...) and versions (if applicable)

CNI and version: flannel v0.11.0
--- end body ---

None