Kubecost is a cost management platform built on open source that helps teams running Kubernetes maximize the efficiency of their spending.
Many teams using Kubernetes struggle to understand where their cloud spend is going, i.e. which team, project, or application is spending how much and on which infrastructure services. This is where Kubecost comes in: based on open source tools for collecting and aggregating the right set of metrics from the cluster and workloads, Kubecost can tell you the cost for the microservice or application that you are running in the cluster.
Kubecost helps developers with systems to track overruns and nudge developers and tools to show the savings of proper capacity management.
The key focus areas for Kubecost are:
Spend visibility: insights on the amounts teams are spending on their workloads.
Insights and optimization: after getting the visibility, infrastructure optimization and application performance improvements are the next step.
Governance: after finding the right balance between the cost spend and performance, running it over time with changing equations falls under this focus area.
Kubecost architecture
The main parts of the architecture are:
- Kubecost Pod: This will take the metrics from Kubernetes API and use the billing data from the configuration provided by the user or taken automatically from the public clouds. With this information, the pod generates a set of metrics that Prometheus can scrape for use.
- Kube-state metrics - Kubernetes API metrics, e.g. resource requests.
- node-exporter - node-level utilization metrics.
- cadvisor - for utilization metrics (CPU/Memory for containers).
- network metrics: Daemonset for collecting metrics.
- Prometheus - metrics data store.
Kubecost on Civo Kubernetes
Kubecost can be installed on any Kubernetes distribution. For this tutorial we will install Kubecost on a Civo Kubernetes cluster.
Cluster creation
You can create the cluster from the UI or from the Civo CLI. For this tutorial, let's create using the CLI.
$ civo k3s create kubecost-demo
The cluster kubecost-demo (9fe52857-f03e-451a-9c69-7125b06dcbbd) has been created
Above will create a 3 node cluster named kubecost-demo
We will need to get the Kubeconfig for the cluster and save to our desired location. If you do not specify a path, it will save it to ~/.kube/config.
$ civo k3s config kubecost-demo --save --local-path /Users/saiyampathak/civo/test/kubecost-demo.config
Access your cluster with:
KUBECONFIG=/Users/saiyampathak/civo/test/kubecost-demo.config kubectl get node
Let's make sure that kubectl knows to use our cluster's configuration file:
$ export KUBECONFIG=/Users/saiyampathak/civo/test/kubecost-demo.config
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3s-kubecost-demo-0edfef9e-node-pool-b3fe Ready <none> 39s v1.20.2+k3s1
k3s-kubecost-demo-0edfef9e-node-pool-e43a Ready <none> 38s v1.20.2+k3s1
k3s-kubecost-demo-0edfef9e-node-pool-9209 Ready <none> 38s v1.20.2+k3s1
Kubecost installation
Kubecost can be installed easily via helm or directly with a kubectl apply command
For this tutorial we will install via helm
helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm upgrade -i --create-namespace kubecost kubecost/cost-analyzer --namespace kubecost --set kubecostToken="aGVsbUBrdWJlY29zdC5jb20=xm343yadf98"
NAME: kubecost
LAST DEPLOYED: Mon Jul 19 23:01:32 2021
NAMESPACE: kubecost
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
--------------------------------------------------Kubecost has been successfully installed. When pods are Ready, you can enable port-forwarding with the following command:
kubectl port-forward --namespace kubecost deployment/kubecost-cost-analyzer 9090
Next, navigate to http://localhost:9090 in a web browser.
You will be able to see all the components deployed in the kubecost namespace
kubectl get pods -n kubecost
NAME READY STATUS RESTARTS AGE
kubecost-prometheus-node-exporter-7twvw 1/1 Running 0 2m40s
kubecost-prometheus-node-exporter-846qc 1/1 Running 0 2m40s
kubecost-prometheus-node-exporter-xcnt5 1/1 Running 0 2m40s
kubecost-kube-state-metrics-6c4df87fb6-bwbkd 1/1 Running 0 2m39s
kubecost-grafana-779599f58d-qlbll 3/3 Running 0 2m39s
kubecost-prometheus-server-6b5884f9d5-qjvzw 2/2 Running 0 2m39s
kubecost-cost-analyzer-69f485d5bd-d2rvn 3/3 Running 0 2m39s
Setting the right parameters
Once Kubecost is deployed onto the cluster, you can forward the port from the cluster and access the kubecost UI at http://localhost:9090 with teh following command:
kubectl port-forward --namespace kubecost deployment/kubecost-cost-analyzer 9090
You will be able to see the cluster on the page:
By default, Kubecost has some cost values set for the workloads in terms of CPU/Memory, etc. Since this is a Civo Kubernetes cluster, we will adjust the cost accordingly in the settings section.
In the settings section enable custom pricing:
We will be using the pricing for a Civo Kubernetes cluster of this size, as mentioned on the pricing page.
Edit in the costs based on charges from the pricing page. The specific charges for RAM and persistent volumes come out as:
Storage per GB - $0.10
Ram per GB - $4
After hitting save, let's start exploring Kubecost features
Kubecost Features
Now we have Kubecost deployed, we can start seeing the overall cost for the workloads, namespaces etc. Let's explore different Kubecost features.
Cost overview
This is the default overview page when you open the kubecost dashboard for a particular cluster. It calculates our costs based on custom pricing inputs we provided in the settings for civo Kubernetes. It provides a high-level view of cluster spend, efficiency, and key cost drivers.
Cost Allocation
Below is the cost allocation view per namespace where the first six columns show the actual cost within the Kubernetes cluster. Kubecost can present cost by any Kubernetes concept, so you can view cost by label, annotation, service, and more.
Next is the shared costs that can be defined in the settings. There can be various use cases for this, like having a cost shared in multiple namespaces with a specific label, or evenly distributing the cost for a particular product/service for all the teams.
You can also define external costs that the Kubernetes cluster is consuming. After drilling down into a particular namespace you will be able to see the pods running in that namespace and the costs associated with them.
Assets
Assets give you the total cost view and adjustments if you have defined any discounts in the settings.
Reports
All the allocations views can be saved as reports and shared with the teams and can be also useful for keeping a track on the spend per team or whatever criteria chosen from the filters provided.
Alerts
You can set up Alerts and notifications based on the thresholds defined.
Kubecost-kubectl integration
Kubecost has a krew plugin as well. As the repository says it:
provides command-line access to Kubernetes cost allocation metrics via the kubecost APIs. It allows developers, devops teams, and others to quickly determine the cost & efficiency for any Kubernetes workload.
You need to have krew installed on your system, after which you can install the cost plugin using:
kubectl krew install cost
Updated the local copy of plugin index.
Installing plugin: cost
Installed plugin: cost
\
| Use this plugin:
| kubectl cost
| Documentation:
| https://github.com/kubecost/kubectl-cost
| Caveats:
| \
| | Requires Kubecost (a cluster-side daemon) to be installed in your cluster.
| | See https://www.kubecost.com/install for installation instructions.
| /
/
WARNING: You installed plugin "cost" from the krew-index plugin repository.
These plugins are not audited for security by the Krew maintainers.
Run them at your own risk.
Now you can use this plugin to find the cost of a namespace or a pod directly using a kubectl command:
kubectl cost pod -n kubecost
+-----------+----------------------------------------------+--------------------+-----------------+
| NAMESPACE | POD | MONTHLY RATE (ALL) | COST EFFICIENCY |
+-----------+----------------------------------------------+--------------------+-----------------+
| kubecost | kubecost-cost-analyzer-69f485d5bd-d2rvn | 3.844531 | 0.000000 |
| | kubecost-prometheus-server-6b5884f9d5-qjvzw | 3.200000 | 0.000000 |
| | kubecost-grafana-779599f58d-qlbll | 0.000000 | 0.000000 |
| | kubecost-prometheus-node-exporter-846qc | 0.000000 | 0.000000 |
| | kubecost-prometheus-node-exporter-xcnt5 | 0.000000 | 0.000000 |
| | kubecost-kube-state-metrics-6c4df87fb6-bwbkd | 0.000000 | 0.000000 |
| | kubecost-prometheus-node-exporter-7twvw | 0.000000 | 0.000000 |
+-----------+----------------------------------------------+--------------------+-----------------+
| SUMMED | | USD 7.044531 | |
+-----------+----------------------------------------------+--------------------+-----------------+
This becomes very handy for the CLI lovers to get the cost of particular resources immediately with additional parameters.
Wrapping up
Kubecost looks to be a great open-source tool to measure costs for your workloads running on Kubernetes cluster. The best part is that you can configure the costs depending on where the cluster is running, and that it can be used to measure the cost of any Kubernetes cluster given its customisable pricing settings.
Let us know on Twitter @Civocloud and @SaiyamPathak if you try Kubecost out on Civo Kubernetes or even bare metal servers.
