GCP

Datalogz Deployment for GCP

Prerequisites

  • Create an IAM sub-account (optional but recommended). This will help to monitor the costs and be more secure, as it will not interfere with production or other workloads running in the existing GCP accounts. If you deploy Datalogz in another sub-account you need to take care of respective network settings to connect your data sources.

  • A Cloud DNS Zone with a Publicly resolvable DNS name. We use this domain to add an entry into it related to Datalogz URL.

  • Create a user with the following IAM permissions:

    • Compute Engine:*

    • IAM:*

    • Cloud Storage:*

    • Kubernetes Engine:*

    • Monitoring:*

    • Autoscaling:*

    • VPC Network:*

    • Cloud Volumes:*

    • ProjectNames:*

    • Cloud DNS:*

Note: You can either create a new IAM user or enable an existing user with the above permissions. We will use the GCP Credentials of this user to deploy Datalogz within your GCP Account using Terraform. After completion, you can remove these permissions from the user.

Deployment Details

Datalogz deploys into the Customer’s GCP Account within a newly created VPC Network on Kubernetes Engine using Helm (https://helm.sh/).

What cluster addons do we install in the GKE Cluster?

  • External DNS (https://github.com/kubernetes-sigs/external-dns)

  • GKE Cluster Autoscaler (https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler)

  • GKE Node Termination Handler (https://github.com/GoogleCloudPlatform/k8s-node-termination-handler)

If you already have a GKE Cluster running in their GCP Account, we can work with you to install Datalogz directly into it as well.

Datalogz uses Terraform (https://www.terraform.io/) to deploy and manage all the GCP resources along with using Ansible (https://www.ansible.com/) to install cluster-addons and Datalogz within the GKE Clusters.

Architecture Diagram

Pricing

The Datalogz product pricing can be broken down into the following GCP services:

Note: The mentioned pricing is based on the basic deployment of Datalogz. We use two On-Demand n2-standard-4 GCE machines in the basic deployment. Based on the usage of Datalogz, it can scale up the Frontend microservice thus resulting in an uptick of GCE costs. There can be additional Data Transfer costs based on usage which are hard to calculate. 80% of GCP cost incurred by Datalogz is due to GKE (Fixed cost per month) and GCE (Variable cost based on usage). Cost may vary slightly depending on the GCP region.

Last updated