IT 3300 : Virtualization

Kubernetes Networking

Services

Remember, each pod in a k8s cluster has a unique IP address (even pods on the same node), so there needs to be a way to automatically reconcile changes among pods so apps keep functioning.

A service is an abstraction wherein you define a set of pods and how they should be accessed. (defined in yaml or json)

Networking problems

  • How do containers communicate with one another?
    • Pods can inherently do this
  • Pod to Pod communications?
  • Pod-to-Service Communications
    • Use services
  • External-to-Service Communications
    • Use Services

Networking

Kubernetes is all about sharing machines between applications. Typically, sharing machines requires ensuring that two applications do not try to use the same ports. Coordinating ports across multiple developers is very difficult to do at scale and exposes users to cluster-level issues outside of their control. Dynamic port management is hard and tedious. K8s uses a different approach.

Networking

We already know:

  • A pod gets its' own IP address
  • pods on a node can communicate with all pods on all nodes without NAT
  • Containers within a pod share their network namespaces
    • meaning containers within a pod can access each other ports on 'localhost'
    • So containers in a pod must coordinate port usage

Service

An abstract way to expose an application running on a set of Pods as a network service. Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.

  • kubectl get services will show our current services.
  • kubectl describe services/kubernetes-bootcamp

Networking Service

Kubernetes Pods are created and destroyed to match the state of your cluster. Pods are nonpermanent resources. If you use a Deployment to run your app, it can create and destroy Pods dynamically.

Each Pod gets its own IP address, however in a Deployment, the set of Pods running in one moment in time could be different from the set of Pods running that application a moment later.

Networking Services

This leads to a problem: if some set of Pods (call them "backends") provides functionality to other Pods (call them "frontends") inside your cluster, how do the frontends find out and keep track of which IP address to connect to, so that the frontend can use the backend part of the workload?

Service

In Kubernetes, a Service is an abstraction which defines a logical set of Pods and a policy by which to access them (sometimes this pattern is called a micro-service). For example, frontend pods should not have to worry about the network changes that might be taking place with backend pods.

Example

Suppose you have a set of pods that each listen on 9376 and carry a label app=MyApp.

    apiVersion: v1
    kind: Service
    metadata:
      name: my-service
    spec:
      selector:
        app: MyApp
      ports:
        - protocol: TCP
          port: 80
          targetPort: 9376

Services

This specification creates a new Service object named "my-service", which targets TCP port 9376 on any Pod with the app=MyApp label.

Kubernetes assigns this Service an IP address (sometimes called the 'cluster IP')

The controller for the Service selector continuously scans for Pods that match its selector, and then POSTs any updates to an Endpoint object also named "my-service"

DNS

You can (and almost always should) set up a DNS service for your Kubernetes cluster using an add-on. In microk8s, you can microk8s enable dns

A cluster-aware DNS server, such as CoreDNS, watches the Kubernetes API for new Services and creates a set of DNS records for each one. If DNS has been enabled throughout your cluster then all Pods should automatically be able to resolve Services by their DNS name.

DNS

For example, if you have a Service called my-service in a Kubernetes namespace my-ns, the control plane and the DNS Service acting together create a DNS record for my-service.my-ns. Pods in the my-ns namespace should be able to find it by simply doing a name lookup for my-service (my-service.my-ns would also work).

Service Types

Kubernetes ServiceTypes allow you to specify what kind of Service you want. The default is ClusterIP. ServiceTypes allow you to expose a service onto an external IP address.

ServiceType - ClusterIP

Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.

ServiceType - NodePort

Exposes the Service on the same port of each selected Node in the cluster using NAT. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>. Superset of ClusterIP.

ServiceType - LoadBalancer

LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.

ServiceType - ExternalName

Maps the Service to the contents of the externalName field (for example, to the hostname api.foo.bar.example). The mapping configures your cluster's DNS server to return a CNAME record with that external hostname value. No proxying of any kind is set up.

Another external name example

    kind: Service
    apiVersion: v1
    metadata:
      name: my-service
    spec:
      type: ExternalName
      externalName: my.database.example.com

When you want to do curl -v http://my-service or curl -v http://my-service.default.svc.cluster.local according your namespace(in this example it is default), it will redirect you at the DNS level to http://my.database.example.com

Some examples

  • kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/service/networking/run-my-nginx.yaml
  • kubectl get pods -l run=my-nginx -o wide

From any node in the cluster we should be able to curl those ip's and get a response

Summary of previous example

Remember:

  • a pod gets its' own IP address
  • the containers in the pod were running nginx (listening on port 80)
  • When you curl the ip, since that is the only thing listening on port 80, we get a response
  • Also note that nothing is listening on port 80 on the NODE. (I.e. do a curl localhost) Nothing should be listening, unless you are using a previous node that you have installed a webserver on.

Some examples

Remember that these pods can be created and destroyed. The deployment will create new ones with new IP's. How can we make sure that things get routed correctly to our pods? Services

Run kubectl expose deployment/my-nginx, then kubectl get svc.

Some examples

The 'expose' command that we ran previously, does the same as if you ran kubectl apply -f on the following file:

    apiVersion: v1
    kind: Service
    metadata:
      name: my-nginx
      labels:
        run: my-nginx
    spec:
      ports:
      - port: 80
        protocol: TCP
      selector:
        run: my-nginx

Previous explanation

  • The spec will create a service that targets port 80 on ANY pod with the matching run: my-nginx label.
  • Look at kubctl describe svc my-nginx
  • Note the endpoints.
  • When a user now accesses the clusterip (the ip given in the service), port 8888, the service knows that it should forward traffic to pods of the given endpoint.

Some examples

  • Delete services with kubectl delete svc [name]

Some examples

To make applications accessible outside of the cluster use either:

  • NodePort
    • kubectl apply -f http://it3300.cs.utahtech.edu/notes/my-nginx.yaml
  • LoadBalancer (only avail on cloud platform) (will do later)

Nodeport examples

You should be able to do the following now:

  • kubectl describe svc my-nginx
  • If on a node in the cluster should be able to curl the endpoints directly at port 80.
  • Should also be able to curl <clusterip>:<port>
  • Should also be able to curl from anywhere using <hostip>:<nodeport> (assuming firewall allows)

Self Quiz

  • kubectl create deployment kubernetes-bootcamp --image=gcr.io/google-samples/kubernetes-bootcamp:v1
  • What ip addresses are the pods listening on? Can you curl (Hint: port 8080)
  • kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080
  • curl the localhost:<port> to see the page.
  • Scale it to 5.
  • curl the localhost:<port> multiple times.
  • What are the endpoints identified by the service?
  • Scale back to 2, did things change?
  • Delete the service, then recreate as a clusterip
  • kubectl expose deployment/kubernetes-bootcamp --port 8080