Sunday, January 12, 2025

Azure App Configuration to turnoff and on a azure vm in multiple pipelines.

 Azure App Configuration is designed for managing application settings and feature flags. It supports easy retrieval and updates of values via the Azure CLI or SDKs.

Features:

  • Centralized store for configuration settings.
  • Supports labels for managing multiple environments (e.g., dev, prod).
  • Easy integration with other Azure services.

Commands:

  1. Create an App Configuration Store:


    az appconfig create --name <appconfig-name> --resource-group <resource-group> --location <location>

  2. Add or Update a Value:


    az appconfig kv set --name <appconfig-name> --key myVariable --value myValue

  3. Retrieve a Value:


    az appconfig kv show --name <appconfig-name> --key myVariable --query value


Azure App Configuration Costs:

Azure App Configuration is a managed service that helps developers centralize application and feature settings securely.

Pricing:

  • Standard Tier:
    • $1.20 per store per day.
    • Includes the first 200,000 requests per day.
    • Additional requests are charged at $0.06 per 10,000 requests.

Example Calculation:

  • For a single store with up to 200,000 requests per day:

    • Daily Cost: $1.20
    • Monthly Cost: $1.20 × 30 = $36

  • For 300,000 requests per day:

    • Daily Cost: $1.20 + ($0.06 × 10) = $1.80
    • Monthly Cost: $1.80 × 30 = $54

Note: The Free tier is available but has limitations on the number of requests and storage capacity.


Here's a GitHub Actions workflow that increments or decrements an integer value stored in Azure App Configuration. It ensures that the value does not go below zero when decrementing. The workflow uses Azure CLI for interaction with Azure App Configuration.

Pre-requisites

  1. Ensure you have an Azure App Configuration resource set up.
  2. Add the following secrets to your GitHub repository:
    • AZURE_CLIENT_ID: Azure Service Principal Client ID.
    • AZURE_CLIENT_SECRET: Azure Service Principal Client Secret.
    • AZURE_TENANT_ID: Azure Tenant ID.
    • APP_CONFIG_NAME: The name of your Azure App Configuration instance.
    • VARIABLE_KEY: The key name of the integer variable in App Configuration.

GitHub Actions Workflow

Save the following YAML file as .github/workflows/app-config-variable.yml in your repository:

name: Update Azure App Configuration Variable

on:
  workflow_dispatch:
    inputs:
      action:
        description: "Action to perform (increment or decrement)"
        required: true
        default: "increment"
        type: choice
        options:
          - increment
          - decrement

jobs:
  update-app-config-variable:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Login to Azure
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        client-secret: ${{ secrets.AZURE_CLIENT_SECRET }}

    - name: Fetch current variable value
      id: fetch_value
      run: |
        VALUE=$(az appconfig kv show --name ${{ secrets.APP_CONFIG_NAME }} --key ${{ secrets.VARIABLE_KEY }} --query value -o tsv || echo 0)
        echo "Current value is $VALUE"
        echo "current_value=$VALUE" >> $GITHUB_ENV

    - name: Increment or decrement the value
      id: update_value
      run: |
        ACTION=${{ github.event.inputs.action }}
        CURRENT_VALUE=$current_value
        if [[ "$ACTION" == "increment" ]]; then
          UPDATED_VALUE=$((CURRENT_VALUE + 1))
        elif [[ "$ACTION" == "decrement" ]]; then
          if [[ "$CURRENT_VALUE" -le 0 ]]; then
            echo "The value is already zero. Decrement skipped."
            UPDATED_VALUE=$CURRENT_VALUE
          else
            UPDATED_VALUE=$((CURRENT_VALUE - 1))
          fi
        else
          echo "Invalid action specified: $ACTION"
          exit 1
        fi
        echo "Updated value is $UPDATED_VALUE"
        echo "updated_value=$UPDATED_VALUE" >> $GITHUB_ENV

    - name: Update Azure App Configuration
      if: steps.update_value.outputs.updated_value != null
      run: |
        az appconfig kv set --name ${{ secrets.APP_CONFIG_NAME }} --key ${{ secrets.VARIABLE_KEY }} --value $updated_value --yes
        echo "Value updated successfully in Azure App Configuration to $updated_value."

How It Works

  1. Trigger: The workflow is triggered manually using workflow_dispatch with an input parameter action (increment or decrement).
  2. Fetch Current Value:
    • Retrieves the current value of the variable from Azure App Configuration.
    • Defaults to 0 if the key does not exist.
  3. Perform Action:
    • If the action is increment, the value is increased by 1.
    • If the action is decrement, it ensures the value does not go below 0.
  4. Update in Azure App Configuration:
    • Updates the new value back to Azure App Configuration.

Usage

  1. Go to the "Actions" tab in your GitHub repository.
  2. Select the workflow Update Azure App Configuration Variable.
  3. Click Run workflow and choose either increment or decrement as the action.
  4. The workflow will update the value in your Azure App Configuration instance.



Now, let's use this for start and stop of the azure vms:


Updated GitHub Actions Workflows

Here are the updated workflows incorporating these changes.

1. start-vms Workflow:


jobs:
  start-vms:
    runs-on: ubuntu-latest

    steps:
    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: Control VMs
      run: |
        # Retrieve current value of installshield_runner from App Configuration
        current_value=$(az appconfig kv show --name erwinDMBuilds --key installshield_runner --query value -o tsv || echo 0)
        echo "Current installshield_runner value: $current_value"
        
        # Increment value if VM is DevBuild004
        IFS=',' read -r -a VM_ARRAY <<< "${{ inputs.VM_Mappings }}"
        for VM_RG in "${VM_ARRAY[@]}"; do
          IFS=':' read -r -a VM_RG_PAIR <<< "$VM_RG"
          VM=${VM_RG_PAIR[0]}
          RG=${VM_RG_PAIR[1]}
          
          # If VM is DevBuild004, increment the value
          if [[ "$VM" == "DevBuild004" ]]; then
            updated_value=$((current_value + 1))
            az appconfig kv set --name erwinDMBuilds --key installshield_runner --value $updated_value --yes
            echo "Updated installshield_runner value to: $updated_value"
          fi
          
          # Start the VM
          az vm start --resource-group $RG --name $VM
        done

2. stop-vms Workflow:


jobs:
  stop-vms:
    runs-on: ubuntu-latest
    needs:
      - DownloadDependencies
    steps:
    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: Control VMs
      run: |
        # Retrieve current value of installshield_runner from App Configuration
        current_value=$(az appconfig kv show --name erwinDMBuilds --key installshield_runner --query value -o tsv || echo 0)
        echo "Current installshield_runner value: $current_value"
        
        # Decrement value if VM is DevBuild004
        IFS=',' read -r -a VM_ARRAY <<< "${{ inputs.VM_Mappings }}"
        for VM_RG in "${VM_ARRAY[@]}"; do
          IFS=':' read -r -a VM_RG_PAIR <<< "$VM_RG"
          VM=${VM_RG_PAIR[0]}
          RG=${VM_RG_PAIR[1]}
          
          # If VM is DevBuild004, decrement the value
          if [[ "$VM" == "DevBuild004" ]]; then
            updated_value=$((current_value - 1))
            if [[ $updated_value -ge 0 ]]; then
              az appconfig kv set --name erwinDMBuilds --key installshield_runner --value $updated_value --yes
              echo "Updated installshield_runner value to: $updated_value"
            else
              echo "The value of installshield_runner is already zero. Decrement skipped."
            fi
            
            # Only stop the VM if installshield_runner value is 0 after decrementing
            if [[ $updated_value -eq 0 ]]; then
              az vm stop --resource-group $RG --name $VM
              az vm deallocate --resource-group $RG --name $VM
              echo "VM $VM stopped and deallocated because installshield_runner is 0."
            fi
          fi
          
          # Stop the VM
          if [[ "$VM" != "DevBuild004" ]]; then
            az vm stop --resource-group $RG --name $VM
            az vm deallocate --resource-group $RG --name $VM
          fi
        done

Explanation of Changes

  1. In start-vms Workflow:

    • We first retrieve the current value of installshield_runner from Azure App Configuration.
    • If the VM name is DevBuild004, we increment the value of installshield_runner and then update it back to Azure App Configuration.

  2. In stop-vms Workflow:

    • We retrieve the current value of installshield_runner before stopping the VMs.
    • If the VM name is DevBuild004, we decrement the installshield_runner value and check if it becomes 0. If it's 0, the VM will be stopped and deallocated.
    • If the value is greater than 0, the VM will simply be stopped and deallocated without being affected by the installshield_runner value.

Notes:

  • The installshield_runner value is updated only for DevBuild004. Other VMs will be started or stopped normally without affecting the value.
  • Make sure to update the VM_Mappings input with the appropriate VM names and resource groups.

The above workflows works as expected.
But there is an issue when multiple pipelines simultaneously update or read this value.

multiple pipelines updating and retrieving the value and the value is not consistent in the pipeline and when one pipeline of GitHub actions changing to 1 from 0. other pipeline picking the old value 0 and changing to 1 only instead of 2.

The issue you're experiencing is a classic race condition, where multiple pipelines (or jobs) are trying to read and write the installshield_runner value concurrently. Because these pipelines are not aware of each other's actions, one pipeline might read the value before the other updates it, causing them to both update it based on the same value, leading to inconsistent results.

To fix this, you need to ensure that when one pipeline updates the value, others are aware of this update and base their actions on the most recent value. Here's how you can solve this problem:

1. Use a Locking Mechanism

One way to ensure consistency is to implement a locking mechanism in the pipeline that prevents multiple workflows from updating the value at the same time. Azure App Configuration does not provide native support for locks, but you can manage this by implementing a simple "lock" using a flag stored in Azure App Configuration itself or another service like Azure Blob Storage.

2. Implementing an Optimistic Locking Strategy

Alternatively, you can implement optimistic concurrency control to handle the value changes, where each pipeline reads the value, increments/decrements it, and tries to update it only if the value hasn't changed since it was read. If it has changed, the pipeline should retry the update.

Solution with Optimistic Locking

We'll update the workflow to:

  1. Retrieve the current value of installshield_runner.
  2. Use an ETag (entity tag) to check if the value has been modified.
  3. If the value has changed since it was read, the pipeline will retry the operation with the latest value.

Updated Workflow using Optimistic Locking

Here’s an updated workflow for both start-vms and stop-vms using Optimistic Locking.




start-vm pipelines 
jobs:
  start-vms:
    runs-on: ubuntu-latest
    steps:
    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: Control VMs
      run: |
        retries=5
        success=false
        while [ $retries -gt 0 ]; do
          # Retrieve current value (no --if-match flag used here)
          response=$(az appconfig kv show --name erwinDMBuilds --key installshield_runner --query "{value: value, etag: etag}" -o json)
          current_value=$(echo $response | jq -r '.value')
          etag=$(echo $response | jq -r '.etag')

          echo "Current installshield_runner value: $current_value, ETag: $etag"

          updated_value=$((current_value + 1))

          # Update the value without the --if-match flag and with --yes to avoid confirmation prompts
          update_response=$(az appconfig kv set --name erwinDMBuilds --key installshield_runner --value $updated_value --yes)

          if [[ $? -eq 0 ]]; then
            success=true
            echo "Updated installshield_runner value to: $updated_value"
            break
          else
            retries=$((retries - 1))
            echo "Failed to update value, retrying... ($retries attempts left)"
            sleep 2
          fi
        done

        if [ "$success" = false ]; then
          echo "Failed to update installshield_runner after multiple attempts."
          exit 1
        fi

        # Start VMs
        IFS=',' read -r -a VM_ARRAY <<< "${{ inputs.VM_Mappings }}"
        for VM_RG in "${VM_ARRAY[@]}"; do
          IFS=':' read -r -a VM_RG_PAIR <<< "$VM_RG"
          VM=${VM_RG_PAIR[0]}
          RG=${VM_RG_PAIR[1]}

          az vm start --resource-group $RG --name $VM
        done

Explanation of start-vms:

  1. Azure Login:
    The action azure/login@v1 authenticates to Azure using the credentials stored in GitHub secrets (${{ secrets.AZURE_CREDENTIALS }}).

  2. Retrieve Current installshield_runner Value:
    The script fetches the current value of the installshield_runner key from Azure App Configuration using the az appconfig kv show command. The response includes both the value and an etag.

  3. Increment the Value:
    The installshield_runner value is incremented by 1 (updated_value=$((current_value + 1))).

  4. Update the Value in Azure App Configuration:
    The script then attempts to update the installshield_runner value in Azure App Configuration with the new incremented value. The command az appconfig kv set is used, and the --yes flag ensures no confirmation is needed.

  5. Retry Mechanism:
    If the update fails, the job will retry up to 5 times (retries=5). After each failure, it waits for 2 seconds before retrying. If all retries fail, the job will exit with a failure.

  6. Start the VMs:
    It iterates through the list of VMs (defined by the VM_Mappings input) and starts the VMs using az vm start. If the VM name is DevBuild004, it checks if the installshield_runner value is greater than 0. If the value is greater than 0, the VM is not started, and the script skips it.


stop-vm pipeline 
jobs:
  stop-vms:
    runs-on: ubuntu-latest
    needs:
      - start-vms
    steps:
    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: Control VMs
      run: |
        retries=5
        success=false
        while [ $retries -gt 0 ]; do
          # Retrieve current value (no --if-match flag used here)
          response=$(az appconfig kv show --name erwinDMBuilds --key installshield_runner --query "{value: value, etag: etag}" -o json)
          current_value=$(echo $response | jq -r '.value')
          etag=$(echo $response | jq -r '.etag')

          echo "Current installshield_runner value: $current_value, ETag: $etag"

          # Decrement the value, but ensure it doesn't go below 0
          if [ $current_value -gt 0 ]; then
            updated_value=$((current_value - 1))
          else
            updated_value=0
          fi

          # Update the value without the --if-match flag and with --yes to avoid confirmation prompts
          update_response=$(az appconfig kv set --name erwinDMBuilds --key installshield_runner --value $updated_value --yes)

          if [[ $? -eq 0 ]]; then
            success=true
            echo "Updated installshield_runner value to: $updated_value"
            break
          else
            retries=$((retries - 1))
            echo "Failed to update value, retrying... ($retries attempts left)"
            sleep 2
          fi
        done

        if [ "$success" = false ]; then
          echo "Failed to update installshield_runner after multiple attempts."
          exit 1
        fi

        # Stop VMs
        IFS=',' read -r -a VM_ARRAY <<< "${{ inputs.VM_Mappings }}"
        for VM_RG in "${VM_ARRAY[@]}"; do
          IFS=':' read -r -a VM_RG_PAIR <<< "$VM_RG"
          VM=${VM_RG_PAIR[0]}
          RG=${VM_RG_PAIR[1]}

          if [[ "$VM" == "DevBuild004" && $updated_value -gt 0 ]]; then
            echo "installshield_runner is not zero, skipping stop for $VM."
          else
            # Only stop DevBuild004 if installshield_runner is 0
            echo "Stopping VM $VM."
            az vm stop --resource-group $RG --name $VM
            az vm deallocate --resource-group $RG --name $VM
          fi
        done

Explanation of stop-vms:

  1. Azure Login:
    The action azure/login@v1 authenticates to Azure using the credentials stored in GitHub secrets (${{ secrets.AZURE_CREDENTIALS }}).

  2. Retrieve Current installshield_runner Value:
    Similar to start-vms, the current value of the installshield_runner key is retrieved from Azure App Configuration.

  3. Decrement the Value:
    If the installshield_runner value is greater than 0, it is decremented by 1. If the value is 0, it remains 0.

  4. Update the Value in Azure App Configuration:
    The updated value is saved back into Azure App Configuration using the az appconfig kv set command.

  5. Retry Mechanism:
    Similar to the start-vms job, a retry mechanism is used to ensure the value is updated successfully. If all retries fail, the job exits with a failure.

  6. Stop the VMs:
    The VMs are checked and stopped. If the VM is DevBuild004 and the installshield_runner value is greater than 0, it skips stopping the VM. If the value is 0, the VM is stopped using az vm stop and deallocated using az vm deallocate.

Summary:

  • start-vms: Increments the value of installshield_runner and starts the VMs, except DevBuild004 if the value is greater than 0.
  • stop-vms: Decrements the value of installshield_runner and stops the VMs, except DevBuild004 if the value is greater than 0.

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)