For most organizations, having multiple Kubernetes clusters for multiple teams, users and environments can be costly, challenging to manage, and expensive. You can mitigate cluster sprawl in your infrastructure by allowing the various users or team members to share a cluster.
One of the key features of Kubernetes is its ability to provide multitenancy, which allows multiple tenants to share the same cluster resources while maintaining their own isolated environments. This allows your organization to maximize resources and reduce costs while providing a secure and reliable environment for your applications.
These cluster tenants can be software teams, applications, customers, or projects. This model is useful when organizations are looking for a solution that will help them optimize infrastructure costs while accommodating different types of users and workloads in the same cluster.
In this article, we will discuss the concept of multitenancy in Kubernetes and how virtual clusters can help you implement multitenancy and be used to improve the efficiency of your organization.
Benefits of multitenancy in Kubernetes
The ability to share resources among different workloads and host different users while maintaining isolation comes with many benefits, some of which include the following;
- Resource isolation: To an extent, multitenancy guarantees resource isolation. You can allocate a certain amount of resources (e.g., CPU, memory, storage) per tenant, and those resources will be isolated from other tenants. With this model, you can ensure that one tenant does not consume all of the available resources, thus disrupting the performance of other tenants’ resources.
- Security isolation: With multitenancy, you can also provide tighter security for your tenants by separating the resources of different tenants and restricting access between them. This kind of security isolation can help prevent unauthorized access to the sensitive data of your tenants and reduce the risk of security breaches.
- Ease of management: Multitenancy can make it easier for you to manage many applications and users within a single cluster. So instead of managing multiple clusters, you can use multitenancy to manage all of the applications and users within a single cluster, reducing the administrative burden of your operations teams.
- Cost savings: By using multitenancy in your organization, you can reduce costs by allowing multiple tenants to share resources within a single cluster. This can be especially useful if you have many applications but do not need dedicated clusters for each one.
- Separation of concerns: Having multiple clusters in your kubernetes cluster allows you to have dedicated clusters per application. For example, you can have a separate virtual cluster for production, development, and testing within your kubernetes cluster, and you have dedicated virtual clusters for each application.
Drawbacks of multitenancy in Kubernetes
While there are a lot of benefits to be gained by implementing multitenancy in your organization, it is worth keeping in mind that it is not a magical perfect solution, and there can be some drawbacks. These include;
- Resource contention: This happens when multiple tenants compete for the same resources, leading to reduced performance. To manage this issue, you must allocate resources to tenants and monitor resource usage to ensure that no single tenant can monopolize resources.
- Complexity: Managing a multitenant cluster can be more complex than managing a single-tenant cluster. This is because you must ensure that the resources of each tenant are isolated and that access to those resources is always restricted as needed.
- Security risks: If you do not take proper security measures in your infrastructure, multitenancy can introduce additional risks. For example, if one tenant's resources are compromised, it could allow an attacker to access the resources of your other tenants as well.
- Compatibility issues: If different tenants are running applications with different dependencies or requirements, it can be challenging to ensure they are compatible. This can lead to compatibility issues and may require additional effort to resolve.
Implementing multitenancy in Kubernetes with virtual clusters
Virtual clusters are full-functioning Kubernetes clusters that run on top of your Kubernetes cluster. They utilize the worker nodes and networking of your cluster but appear to your tenants as a dedicated cluster. They are similar to virtual machines because they help partition your Kubernetes cluster into separate ones.
These virtual clusters help manage cluster-scoped resources. For example, operators requiring cluster-wide permissions can quickly deploy in your virtual cluster. Since virtual clusters reside in namespaces, you can easily manage and allocate the resources of your tenants by using resource quotas. This way, you can adequately partition your cluster in a fair and helpful way to your tenants. By doing this, there is also a resource demarcation so that one tenant cannot use up all the resources, leaving nothing for others.
You can also improve your developers’ experience by helping them self-service with the aid of virtual clusters. Virtual clusters can empower them to easily spin up near production-like environments to test their applications without waiting for permissions from the admin.
Virtual clusters are also a great way to implement multitenancy in your organization by providing an isolated environment for the various tenants of your cluster.
Creating virtual clusters with Vcluster
What is Vcluster
Vcluster is an open-source tool that lets you spin up virtual clusters in your namespace. They run as pods in your clusters, consisting of two containers; a control plane container that contains the API server, controller manager, and a connection (or mount) of the data store, and a Syncer container that copies the pods that are created within the vcluster to the underlying host cluster.
Vcluster also comes with a command line client, which can be used for managing your virtual cluster.
Prerequisites
Civo’s cloud-native infrastructure services are powered by Kubernetes. We will use a Civo Kubernetes cluster as the host cluster for this guide.
To get started, you will need the following:
- Civo Account
- Civo Command Line Client
- Kubectl for your operating system
- A Linux-like terminal environment
Instructions
After setting up the Civo command line with our API key using the instructions in the repository, you can create your cluster using the following command:
civo kubernetes create civo-cluster
Your Kubernetes cluster ‘civo-cluster’ is created, and is visible in your account dashboard.
Make sure you download the KUBECONFIG file of your cluster and point your kubectl utility to it to proceed. Next, install the Vcluster commandline client with the following code:
curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster
Check that the Vcluster CLI has been installed using the following:
vcluster --version
Now that you have Vcluster CLI installed you can now create a virtual cluster in your kubernetes cluster with the following command:
vcluster create my-vcluster
This creates a namespace and a virtual cluster in said namespace. It also automatically connects to the newly created virtual cluster through port-forwarding.
In a new terminal, you can interact with your newly created Virtual cluster by checking the namespaces using the kubectl CLI:
kubectl get ns
You can see that your virtual cluster has a default namespace, kube-system, and other namespaces just like your regular Kubernetes cluster.
Deploy a simple Ngnix web server to see just how well your virtual cluster works by first creating a new namespace with the kubectl CLI:
kubectl create ns nginx-demo
Next create a simple deployment in the nginx-demo namespace:
kubectl create deployment nginx-deployment -n demo-nginx --image=nginx
At once you can see that your newly created namespace and deployment are present on the virtual cluster. You can view these with the command kubectl get deployments -n demo-nginx.
Additionally, vcluster allows you to create a virtual cluster you can connect to using a load balancer instead of port-forwarding. To do this, add the expose flag to the cluster create command:
vcluster create my-exposed-vcluster --exposed
Disconnect from your virtual cluster using the following command:
vcluster disconnect
You can see the namespaces vcluster created to deploy your virtual cluster by using the kubectl command kubectl get ns on the cluster you created at the start:
Since your virtual cluster runs as pods in the namespace vcluster created, you can run management tasks on them, such as viewing their logs, debugging, etc. You can get the pods powering your virtual cluster by using the following command:
kubectl get po -n vcluster-my-vcluster
You can see the pods running your virtual cluster, as well as CoreDNS which facilitates Networking.
Conclusion
There are a lot of benefits to including multitenancy in your organization's Kubernetes infrastructure, it can help you save on cost, reduce complexity and help your developers self-service when needed. Virtual clusters are fully functioning Kubernetes clusters that run on top of an existing Kubernetes cluster:
- They utilize your cluster nodes and networking and appear as separate clusters to your tenant.
- They are a great way for implementing multitenancy in your Kubernetes cluster.
Vcluster is an open-source tool that allows you to create and manage virtual clusters in a namespace. It comes with a command line client, making it easy for you to use from your local machine or add to your CI/CD.
