Envoy router verifies ate apiserver's serving cert

[Zoe Zhao (zoez7)]

Part of #170
Establishes mutual TLS between the atenet router and ate-apiserver, and updates the certificate plumbing it depends on.

Main changes:

1. The atenet router now verifies ate apiserver' serving certificate, and presents its client cert to ate apiserver. Previously the connection used InsecureSkipVerify. - This change is in commit c8ee6eb.
2. Repurpose the old servicedns signer into atesystemsigner. It only issues
   certs for pods in ate-system, every cert carries the pod's SPIFFE URI, and DNS
   SANs are opted into via the atesystem.podcert.ate.dev/expects-service
   userAnnotation (rather than inferred from Service membership). This change is in commit 88c6034.

Minor bug fixes:

1. Prevent servicednssigner from signing a cert with no DNS SANs.

• Tests pass
• Appropriate changes to documentation are included in the PR

[Taahir Ahmed (ahmedtd)]

Can you talk a little bit about the background of splitting out separate CAs / signers for the workerpools and ate-system components?

If at all possible, I would prefer to restrict us to two signers:

• One that issues certs with DNS SANs based on the K8s services that a pod is part of (though opt-in with an annotation or label is fine).
• One that issues SPIFFE certs that encode the k8s service account identity of the pod (and potentially, the pod ID, node ID etc in an additional extension).

The reason is that we can be reasonably confident that signers like these will land in upstream K8s in the near-ish future. The certificates here aren't special or unique to substrate, and they aren't part of the value proposition of the project. So I think we should plan to rebase on native K8s capabilities as they become available. That's easiest if we don't get very custom with the signers.

[Zoe Zhao (zoez7)]

I think the main motivations for splitting out separate CAs / signers was, if I mount the "podidentity" CA bundle as trusted client CAs in ate-system services, workerpool pods (therefore actors) would also be able to connect to ate-system servers like the Valkey cluster and atelet server.

But after typing it out loud - it seems this needs to be solved anyway (probably in a later milestone) by authorization inside servers such as valkey, instead of using separate CAs. Does that make sense to you?