Skip to main content

Otomi Architecture

Projects

Otomi consists out of the following projects:

Projectopen source
otomi-coreYes
otomi-tasksYes
otomi-clientsYes
otomi-consoleNo
otomi-apiNo

Otomi Core

Otomi Core is the heart of Otomi and contains a suite of the following integrated Kubernetes applications:

  • Istio: The service mesh framework with end-to-end transit encryption
  • Velero: Back up and restore your Kubernetes cluster resources and persistent volumes
  • Argo CD: Declarative continuous deployment
  • KubeClarity: Detect vulnerabilities of container images
  • Knative: Deploy and manage serverless workloads
  • Prometheus: Collecting container application metrics
  • Loki: Collecting container application logs
  • Harbor: Container image registry with role-based access control, image scanning, and image signing
  • HashiCorp Vault: Manage Secrets and Protect Sensitive Data
  • Kubeapps: Launching and managing applications on Kubernetes
  • Keycloak: Identity and access management for modern applications and services
  • OPA: Policy-based control for cloud-native environments
  • Let's Encrypt: A nonprofit Certificate Authority providing industry-recognized TLS certificates
  • Jaeger: End-to-end distributed tracing and monitor for complex distributed systems
  • Kiali: Observe Istio service mesh relations and connections
  • External DNS: Synchronize exposed ingresses with DNS providers
  • Drone: Continuous integration platform built on Docker
  • Gitea: Self-hosted Git service
  • Nginx Ingress Controller: Ingress controller for Kubernetes
  • Minio: High performance Object Storage compatible with Amazon S3 cloud storage service

Otomi Core is made available (per release) as a container image.

Otomi Core also contains the source code for Otomi CLI. Otomi CLI is a custom developed Command Line Interface for Otomi. Otomi CLI can be used for advanced initial configuration (bootstrapping), deployment, sync, push, template validation, and much more.

Otomi Tasks

Otomi Tasks consists of a set of Kubernetes jobs. These jobs ensure that the configuration of applications integrated in Otomi are always equal to the desired-state configuration (see Otomi Values). An example: If a team is created via Otomi Console (in combination with Otomi API), Otomi Tasks ensures that a project is created for the new team in Harbor, the access to the project in Harbor is configured, a robot account (that can be used to push images to the project registry) is created and that a pull secret is created in the namespace of the team.

Otomi Tasks is currently used to configure the following applications:

  • KeyCloak
  • Harbor
  • Gitea
  • Drone
  • Otomi (copy-certs and wait-for)

Otomi Clients

A factory to build and publish openapi clients used in the redkubes/otomi-tasks repo.

Otomi Clients is currently used to generate openapi clients for the following applications:

  • KeyCloak
  • Harbor
  • Gitea

Otomi API

Otomi API allows for a controlled change of all Otomi Values, based on a configuration scheme and is the brain of Otomi. Otomi API runs as a container on each cluster running.

Otomi Console

Otomi Console is the User Interface of Otomi. Otomi Console communicates with Otomi API for reading and changing Otomi Values configuration. Otomi Console also offers (via the Otomi Apps option) shortcuts to the UI of the various integrated applications.

Integrated Applications

Otomi contains four types of applications:

  • Core applications: applications that are activated by default
  • Shared applications: applications that are shared between teams. Shared applications are user-, and role-aware or not (user is anonymous)
  • Team applications: applications with a dedicated instance per team
  • Optional applications: applications that are optional and can be activated by the platform admin

The following table shows all integrated applications:

ApplicationCoreSharedTeamUser/role-awareOptional
IstioX
Ingress NGINX ControllerX
KnativeX
KeyCloakXX
Prometheus ServerXX
Prometheus kube state metricsXX
AlertmanagerXX
Grafana/LokiXXX
Gatekeeper OperatorX
Hashicorp VaultXXX
HarborXXX
KubeappsXX
DroneXX
GiteaXXX
HttpbinXX
JeagerX
KialiX
KubeClarityX
VeleroX
MinioX

Ingress architecture

Otomi installs and configures an advanced ingress architecture. Ingress for a service can be configured using Otomi Services. The following figure shows the ingress and SSO architecture.

img/ingress-overview

The ingress & SSO architecture overview explained:

  • (optional) an external gateway is used for termination of external traffic (e.g. an Azure Application Gateway or an AWS Application Load Balancer).
  • 2 Ingress NGINX controllers are deployed, one for public access and one for authenticated access.
  • Authenticated (SSO) access is handled by an oauth2 proxy and KeyCloak. The user logs in using the Otomi custom KeyCloak login page.
  • KeyCloak is configured with an external IDP (optional) or uses local accounts. After authentication, KeyCloak provides a normalized JWT token. The JWT token is used by integrated core applications (providing user and role information) and team services configured with SSO
  • 4 Istio (ingress) gateways are provisioned:
  1. a public gateway for routing public (non authenticated traffic to a service
  2. an authentication gateway to route authenticated traffic to a service
  3. a local gateway (for local cluster routing)
  4. a Knative gateway to route traffic to Knative services
  • For each service a Istio virtual service is configured.
  • One egress gateway is provisioned for all egress traffic (network policies allow all egress traffic).