README.md 6.62 KB
Newer Older
1
2
3
4
5
6
7
8
9
# Kubernetes Cluster Federation (KubeFed)

## Documentation

Below some useful links for the main tools of a Kubernetes federation:

- https://github.com/kubernetes-sigs/kubefed
- https://github.com/kubernetes-sigs/external-dns
- https://github.com/kubernetes-sigs/kubefed/blob/master/docs/ingressdns-with-externaldns.md
10
11
12
13
14
15
16
17


## Environment configuration

Before starting it is necessary create several Kubernetes clusters to make the federation. One cluster will be the Host (Host Cluster), in which will be install the KubeFed (Control Plane), and the remaining clusters will be the Members (Member Cluster).
To create the clusters on Openstack, it is possible to use the follow Juju bundle: https://jaas.ai/u/csd-garr/kubernetes-openstack-integrator/bundle.

Once all the clusters for federation have been created, it is useful to collect all cluster access configurations (.kube/config) in a single config file. In this way you can easily switch from one cluster to another.
Marco Lorini's avatar
Marco Lorini committed
18
Below is an example:
19

20
```bash
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-host
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-member1
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-member2
contexts:
- context:
    cluster: k8s-cluster-host
    user: admin-host
  name: k8s-context-host
- context:
    cluster: k8s-cluster-member1
    user: admin-member1
  name: k8s-context-member1
- context:
    cluster: k8s-cluster-member2
    user: admin-member2
  name: k8s-context-member2
current-context: k8s-context-host
kind: Config
preferences: {}
users:
- name: admin-host
  user:
    password: 8Rvl5qW...
    username: admin
- name: admin-member1
  user:
    password: 0JefxZq...
    username: admin
- name: admin-member2
  user:
    password: hJLiVLI...
    username: admin

```

In this way you have access to all the clusters by the context switch, below are the commands for the context switch:

list contexts:
70
```bash
71
72
73
74
$ kubectl config get-contexts
```

switch context:
75
```bash
76
77
$ kubectl config use-context <context-name>
```
78
79
80
81
82

## Installation

### KubeFed client (kubefedctl)

Marco Lorini's avatar
Marco Lorini committed
83
To install the KubeFed client run the following commands in your local machine:
84

85
```bash
86
87
88
89
90
91
92
93
94
95
96
97
98
$ VERSION=0.2.0-alpha.1
$ OS=linux
$ ARCH=amd64
$ curl -LO https://github.com/kubernetes-sigs/kubefed/releases/download/v${VERSION}/kubefedctl-${VERSION}-${OS}-${ARCH}.tgz
$ tar -zxvf kubefedctl-*.tgz
$ chmod u+x kubefedctl
$ sudo mv kubefedctl /usr/local/bin/ # make sure the location is in the PATH
```

**N.B.**: the versions we tested are: 0.1.0-rc6 and 0.2.0-alpha.1, you can try the latest version.

### KubeFed server (Control Plane)

Marco Lorini's avatar
Marco Lorini committed
99
To install the Controll Plane, switch to the host context and configuring the RBAC role for Tiller (in this case we use Helm v2):
100

101
```bash
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
$ cat << EOF | kubectl apply -f -
	apiVersion: v1
	kind: ServiceAccount
	metadata:
	  name: tiller
	  namespace: kube-system
	---
	apiVersion: rbac.authorization.k8s.io/v1
	kind: ClusterRoleBinding
	metadata:
	  name: tiller
	roleRef:
	  apiGroup: rbac.authorization.k8s.io
	  kind: ClusterRole
	  name: cluster-admin
	subjects:
	  - kind: ServiceAccount
	    name: tiller
	    namespace: kube-system
	EOF
```

helm init command:
125
126

```bash
127
128
129
130
131
$ helm init --service-account tiller
```

Now it is possible install the Control Plane on the Host Cluster:

132
```bash
133
134
135
136
137
138
139
$ helm repo add kubefed-charts https://raw.githubusercontent.com/kubernetes-sigs/kubefed/master/charts
$ helm install kubefed-charts/kubefed --name kubefed --version=0.2.0-alpha.1 --namespace kube-federation-system
```

**N.B.**: use the same version for client and server.

**N.B.**: in this way the Control Plane can find Federated resources in each namespace, to limit this action in a specific namespace add the parameter `--set global.scope=Namespaced`.
140
141
142
143
144
145


## Create the federation

To create a Kubernetes Federation it is necessary federate Member Clusters. Use the KubeFed `join` command:

146
```bash
147
148
149
150
151
152
$ kubefedctl join fed-cluster-member1 --cluster-context k8s-context-member1 --host-cluster-context k8s-context-host --v=2
$ kubefedctl join fed-cluster-member2 --cluster-context k8s-context-member2 --host-cluster-context k8s-context-host --v=2
```

Remove cluster to the federation, `unjoin` command:

153
```bash
154
155
156
157
$ kubefedctl unjoin fed-cluster-member1 --cluster-context k8s-context-member1 --host-cluster-context k8s-context-host --v=2
$ kubefedctl unjoin fed-cluster-member2 --cluster-context k8s-context-member2 --host-cluster-context k8s-context-host --v=2
```

Marco Lorini's avatar
Marco Lorini committed
158
Command to show federated clusters:
159

160
```bash
161
162
$ kubectl -n kube-federation-system get kubefedclusters
```
163

164

Marco Lorini's avatar
Marco Lorini committed
165
## Enable API
166

Marco Lorini's avatar
Marco Lorini committed
167
To enable a type of resource to be federated it is necessary to use the enable command of kubefedctl:
168

169
```bash
170
171
172
173
174
$ kubefedctl enable <api-resources-name.api-group> --kubefed-namespace kube-federation-system
```

To know the list of resourses:

175
```bash
176
177
178
179
180
$ kubectl api-resources -o wide
```

Some examples:

181
```bash
182
183
184
185
186
187
188
$ kubefedctl enable deployments.apps --kubefed-namespace kube-federation-system
$ kubefedctl enable services --kubefed-namespace kube-federation-system
$ kubefedctl enable ingresses.extensions --kubefed-namespace kube-federation-system
```

Command to disable federated API:

189
```bash
190
191
192
193
$ kubefedctl disable services --kubefed-namespace kube-federation-system
```

**N.B.**: Do not federate the “kind: pod” resource. In this case, federating a whole namespace will federate pods and deployments at the same time. Deployments will create replicasets which in turn create pods. It will result a duplication of the pods resources.
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246


## Deploy application

Below the procedure to deploy and federate an application and enable ExternalDNS.

### Create namespace (Host Cluster)

The first step is create a namespace in the host cluster:

command line:

```bash
$ kubectl create namespace <namespace-name>
```

or create it by yaml file (resource/namespace.yaml):

```yaml
# namespace.yaml file
apiVersion: v1
kind: Namespace
metadata:
  name: fed-namespace
```

```bash
$ kubectl create -f resource/namespace.yaml
```

### Federate namespace

Once the namespace is created in the Host Cluster, it can be federate on the Member Cluster:

```yaml
# federated_namespace.yaml
apiVersion: types.kubefed.io/v1beta1
kind: FederatedNamespace
metadata:
  name: fed-namespace
  namespace: fed-namespace
spec:
  placement:
    clusters:
    - name: member-cluster-1
    - name: member-cluster-2
```

```bash
$ kubectl --context=<host-cluster-context> create -f resource/federated_namespace.yaml
```

**N.B.**: the option `--context` is not necessary but make sure that the right context is selected.