Kubernetes YAML: How to Set Dynamic Values

• Introduction
• Step-by-Step Guide
• Code Example
• Additional Notes
• Summary
• Conclusion
• References

Managing dynamic values in Kubernetes YAML files is crucial for flexibility and automation. This article explores various methods to achieve this, each offering unique advantages depending on your requirements.

To manage dynamic values in Kubernetes YAML files, you have several options:

1. Using Environment Variables:

• Define environment variables in your shell.

export MY_VALUE="Hello"

• Reference them in your YAML using the $() syntax.

spec:
  containers:
  - name: my-container
    env:
    - name: GREETING
      value: $(MY_VALUE)

2. Command Substitution:

• Use command substitution to insert the output of a command into your YAML.

spec:
  containers:
  - name: my-container
    image: $(ls | grep my-image | head -1)

3. Templating Tools:

• Helm: Define charts with templates and values.

  # values.yaml
  image: my-image:latest
  replicaCount: 3
  
  # deployment.yaml
  spec:
    replicas: {{ .Values.replicaCount }}
    template:
      spec:
        containers:
        - name: my-container
          image: {{ .Values.image }}

  • Install with helm install my-release -f values.yaml .

• Kustomize: Use overlays to patch base YAML files.

  # base/deployment.yaml
  spec:
    replicas: 1
    template:
      spec:
        containers:
        - name: my-container
          image: my-image:latest
  
  # overlays/dev/replicas.yaml
  patchesStrategicMerge:
  - |-
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: my-deployment
    spec:
      replicas: 3

  • Apply with kubectl apply -k overlays/dev

4. ConfigMaps and Secrets:

• Store dynamic values in ConfigMaps or Secrets.

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
data:
  my-value: "Hello"

• Mount them as volumes or environment variables in your pods.

spec:
  containers:
  - name: my-container
    env:
    - name: GREETING
      valueFrom:
        configMapKeyRef:
          name: my-config
          key: my-value

5. Programming Languages:

• Generate YAML files dynamically using languages like Python.

import yaml

data = {
    'apiVersion': 'v1',
    'kind': 'Pod',
    'metadata': {'name': 'my-pod'},
    'spec': {'containers': [{'name': 'my-container', 'image': 'my-image'}]},
}

with open('pod.yaml', 'w') as f:
    yaml.dump(data, f)

Choose the method that best suits your needs and workflow.

This document provides code examples for managing dynamic values in Kubernetes YAML files using various methods. These methods include using environment variables, command substitution, templating tools like Helm and Kustomize, ConfigMaps and Secrets, and programming languages like Python. The examples demonstrate how to inject dynamic values for container images, replicas, database hosts, and other configuration parameters. The best approach depends on specific workflow and application complexity.

This document provides code examples for the methods described in the article.

1. Using Environment Variables:

# Set environment variable
export DB_HOST="my-database.example.com"

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: my-app-container
        image: my-app:latest
        env:
        - name: DB_HOST
          value: $(DB_HOST) 

2. Command Substitution:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: my-app-container
        image: $(cat image_tag.txt) # Assuming image_tag.txt contains the image tag

3. Templating Tools:

a) Helm:

# values.yaml
replicaCount: 3
image: my-app:latest
database:
  host: my-database.example.com

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}-deployment
spec:
  replicas: {{ .Values.replicaCount }}
  template:
    spec:
      containers:
      - name: {{ .Release.Name }}-container
        image: {{ .Values.image }}
        env:
        - name: DB_HOST
          value: {{ .Values.database.host }}

b) Kustomize:

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: my-app-container
        image: my-app:latest

# overlays/dev/replicas.yaml
patchesStrategicMerge:
- |-
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: my-app
  spec:
    replicas: 3

# overlays/dev/image.yaml
patchesStrategicMerge:
- |-
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: my-app
  spec:
    template:
      spec:
        containers:
        - name: my-app-container
          image: my-app:dev

4. ConfigMaps and Secrets:

# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
data:
  database_host: my-database.example.com

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: my-app-container
        image: my-app:latest
        env:
        - name: DB_HOST
          valueFrom:
            configMapKeyRef:
              name: my-config
              key: database_host

5. Programming Languages (Python):

import yaml

# Define dynamic values
image_tag = "latest"
replicas = 3

# Define deployment structure
deployment = {
    'apiVersion': 'apps/v1',
    'kind': 'Deployment',
    'metadata': {'name': 'my-app'},
    'spec': {
        'replicas': replicas,
        'template': {
            'spec': {
                'containers': [
                    {'name': 'my-app-container', 'image': f'my-app:{image_tag}'}
                ]
            }
        },
    },
}

# Generate YAML file
with open('deployment.yaml', 'w') as f:
    yaml.dump(deployment, f)

These are just a few examples of how to manage dynamic values in Kubernetes YAML files. The best approach for your specific needs will depend on your workflow and the complexity of your application.

General Considerations:

• Security: Be mindful of storing sensitive information like passwords and API keys directly in YAML files. Utilize Kubernetes Secrets for such data.
• Complexity vs Maintainability: Choose the method that strikes a balance between managing complexity and ensuring maintainability of your deployments. Simple use cases might only require environment variables, while complex applications might benefit from templating tools.
• Testing: Always test your YAML files thoroughly, especially when using dynamic values, to avoid deployment issues.

Specific to Each Method:

• Environment Variables:
  • Great for simple values and development environments.
  • Can be error-prone if an environment variable is not set or misspelled.
• Command Substitution:
  • Useful for dynamically fetching values like image tags from external sources.
  • Can make debugging difficult if the command fails or produces unexpected output.
• Templating Tools:
  • Helm: Offers a robust way to manage complex deployments with reusable charts and value overrides.
  • Kustomize: Provides a more GitOps-friendly approach with base configurations and overlays for customization.
• ConfigMaps and Secrets:
  • Suitable for storing configuration data and secrets separately from your application code.
  • Allows for centralized management and updates to configuration without redeploying pods.
• Programming Languages:
  • Provides the most flexibility for generating complex YAML files based on various conditions.
  • Requires additional tooling and might increase the complexity of your deployment pipeline.

Additional Tools and Techniques:

• Jsonnet: A data templating language that can be used to generate YAML files with complex logic.
• ytt: A YAML templating tool that uses a Python-like syntax for templating and data manipulation.
• CI/CD Pipelines: Integrate dynamic value management into your CI/CD pipelines for automated deployments with different configurations.

Remember that the best approach for managing dynamic values depends on your specific needs and the complexity of your Kubernetes deployments.

This table summarizes different methods for managing dynamic values in Kubernetes YAML files:

| Method | Description

By understanding and utilizing these methods, you can effectively manage dynamic values in your Kubernetes YAML files, leading to more adaptable, maintainable, and automated deployments. Each method has its own strengths and weaknesses, so carefully consider your specific needs and workflow to choose the most appropriate approach. Remember to prioritize security, testability, and maintainability when implementing dynamic value management in your Kubernetes deployments.

• How to set dynamic values with Kubernetes yaml file? : r/kubernetes | Posted by u/online2offline - 2 votes and 8 comments
• How to dynamically populate values into Kubernetes yaml files ... | Jun 22, 2018 ... This sounds like a perfect use case for Helm (www.helm.sh). Helm Charts helps you define, install, and upgrade Kubernetes applications.
• Passing File Reference Into Yaml File For K8 | Nutanix Community | I'm writing a yaml file to create a pod. And i have to pass multiple json files into the docker container. How can i pass these files into the yaml file dynamically?
• Dynamic Declarative Object Management - Discuss Kubernetes | Hi, with kubectl apply … deployments are defined in yaml files, which seems to be a very static approach. Basically I would like to dynamicallly create unique deployments per customer but based on the same deployment-yaml. Meaning I would need to pass eg. a customer id dynamically rather than defining everything in the yaml. Is there a way to accomplish this? Thx Robert
• Updating Dynamic Config in values.dynamic_config.yaml ... | We are trying to update the values.dynamic_config.yaml file as per the documentation with helm (here) But helm is getting stuck. So we are trying to apply manifest file kubectl apply -f dynamicconfigmap.yaml. Can someone help us with equivalent file?
• Values Files - Helm | Instructions on how to use the --values flag.
• Note about Traefik static and dynamic configuration on Kubenetes ... | Background
• Create a persistent volume with Azure Files in Azure Kubernetes ... | Learn how to create a static or dynamic persistent volume with Azure Files for use with multiple concurrent pods in Azure Kubernetes Service (AKS)
• Hi all, Is there a way to pass a dynamic value to a manifest? At the ... | gopi m: Hi all, Is there a way to pass a dynamic value to a manifest? At the time of running ‘kubectl apply’ I need a variable to be populated with the ‘env variable’ of the host where we are going to ‘apply’ the manifest. In the below sample example, instead of passing a fixed host ip, I need it to be populated with the latest host ip available as env variable. apiVersion: v1 kind: Pod metadata: name: print-greeting spec: containers: - name: env-print-demo image: bash env: - name: IN...