# Kubernetes Cluster Federation (KubeFed)

## Documentation

Below some useful links for the main tools of a Kubernetes federation:

- https://github.com/kubernetes-sigs/kubefed
- https://github.com/kubernetes-sigs/external-dns
- https://github.com/kubernetes-sigs/kubefed/blob/master/docs/ingressdns-with-externaldns.md


## Environment configuration

Before starting it is necessary create several Kubernetes clusters to make the federation. One cluster will be the Host (Host Cluster), in which will be install the KubeFed (Control Plane), and the remaining clusters will be the Members (Member Cluster).
To create the clusters on Openstack, it is possible to use the follow Juju bundle: https://jaas.ai/u/csd-garr/kubernetes-openstack-integrator/bundle.

Once all the clusters for federation have been created, it is useful to collect all cluster access configurations (.kube/config) in a single config file. In this way you can easily switch from one cluster to another.
Below is an example:

```bash
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-host
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-member1
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-member2
contexts:
- context:
    cluster: k8s-cluster-host
    user: admin-host
  name: k8s-context-host
- context:
    cluster: k8s-cluster-member1
    user: admin-member1
  name: k8s-context-member1
- context:
    cluster: k8s-cluster-member2
    user: admin-member2
  name: k8s-context-member2
current-context: k8s-context-host
kind: Config
preferences: {}
users:
- name: admin-host
  user:
    password: 8Rvl5qW...
    username: admin
- name: admin-member1
  user:
    password: 0JefxZq...
    username: admin
- name: admin-member2
  user:
    password: hJLiVLI...
    username: admin

```

In this way you have access to all the clusters by the context switch, below are the commands for the context switch:

list contexts:
```bash
$ kubectl config get-contexts
```

switch context:
```bash
$ kubectl config use-context <context-name>
```

## Installation

### KubeFed client (kubefedctl)

To install the KubeFed client run the following commands in your local machine:

```bash
$ VERSION=0.2.0-alpha.1
$ OS=linux
$ ARCH=amd64
$ curl -LO https://github.com/kubernetes-sigs/kubefed/releases/download/v${VERSION}/kubefedctl-${VERSION}-${OS}-${ARCH}.tgz
$ tar -zxvf kubefedctl-*.tgz
$ chmod u+x kubefedctl
$ sudo mv kubefedctl /usr/local/bin/ # make sure the location is in the PATH
```

**N.B.**: the versions we tested are: 0.1.0-rc6 and 0.2.0-alpha.1, you can try the latest version.

### KubeFed server (Control Plane)

To install the Controll Plane, switch to the host context and configuring the RBAC role for Tiller (in this case we use Helm v2):

```bash
$ cat << EOF | kubectl apply -f -
	apiVersion: v1
	kind: ServiceAccount
	metadata:
	  name: tiller
	  namespace: kube-system
	---
	apiVersion: rbac.authorization.k8s.io/v1
	kind: ClusterRoleBinding
	metadata:
	  name: tiller
	roleRef:
	  apiGroup: rbac.authorization.k8s.io
	  kind: ClusterRole
	  name: cluster-admin
	subjects:
	  - kind: ServiceAccount
	    name: tiller
	    namespace: kube-system
	EOF
```

helm init command:

```bash
$ helm init --service-account tiller
```

Now it is possible install the Control Plane on the Host Cluster:

```bash
$ helm repo add kubefed-charts https://raw.githubusercontent.com/kubernetes-sigs/kubefed/master/charts
$ helm install kubefed-charts/kubefed --name kubefed --version=0.2.0-alpha.1 --namespace kube-federation-system
```

**N.B.**: use the same version for client and server.

**N.B.**: in this way the Control Plane can find Federated resources in each namespace, to limit this action in a specific namespace add the parameter `--set global.scope=Namespaced`.


## Create the federation

To create a Kubernetes Federation it is necessary federate Member Clusters. Use the KubeFed `join` command:

```bash
$ kubefedctl join fed-cluster-member1 --cluster-context k8s-context-member1 --host-cluster-context k8s-context-host --v=2
$ kubefedctl join fed-cluster-member2 --cluster-context k8s-context-member2 --host-cluster-context k8s-context-host --v=2
```

Remove cluster to the federation, `unjoin` command:

```bash
$ kubefedctl unjoin fed-cluster-member1 --cluster-context k8s-context-member1 --host-cluster-context k8s-context-host --v=2
$ kubefedctl unjoin fed-cluster-member2 --cluster-context k8s-context-member2 --host-cluster-context k8s-context-host --v=2
```

Command to show federated clusters:

```bash
$ kubectl -n kube-federation-system get kubefedclusters
```


## Enable API

To enable a type of resource to be federated it is necessary to use the enable command of kubefedctl:

```bash
$ kubefedctl enable <api-resources-name.api-group> --kubefed-namespace kube-federation-system
```

To know the list of resourses:

```bash
$ kubectl api-resources -o wide
```

Some examples:

```bash
$ kubefedctl enable deployments.apps --kubefed-namespace kube-federation-system
$ kubefedctl enable services --kubefed-namespace kube-federation-system
$ kubefedctl enable ingresses.extensions --kubefed-namespace kube-federation-system
```

Command to disable federated API:

```bash
$ kubefedctl disable services --kubefed-namespace kube-federation-system
```

**N.B.**: Do not federate the “kind: pod” resource. In this case, federating a whole namespace will federate pods and deployments at the same time. Deployments will create replicasets which in turn create pods. It will result a duplication of the pods resources.


## Deploy application

Below the procedure to deploy and federate an application and enable ExternalDNS.

### Create namespace (Host Cluster)

The first step is create a namespace in the host cluster:

command line:

```bash
$ kubectl create namespace <namespace-name>
```

or create it by yaml file (resource/namespace.yaml):

```yaml
# namespace.yaml file
apiVersion: v1
kind: Namespace
metadata:
  name: fed-namespace
```

```bash
$ kubectl create -f resource/namespace.yaml
```

### Federate namespace

Once the namespace is created in the Host Cluster, it can be federate on the Member Cluster:

```yaml
# federated_namespace.yaml
apiVersion: types.kubefed.io/v1beta1
kind: FederatedNamespace
metadata:
  name: fed-namespace
  namespace: fed-namespace
spec:
  placement:
    clusters:
    - name: member-cluster-1
    - name: member-cluster-2
```

```bash
$ kubectl --context=<host-cluster-context> create -f resource/federated_namespace.yaml
```

**N.B.**: the option `--context` is not necessary but make sure that the right context is selected.