Get Notifications about Unwanted Changes in your Infrastructure

During recent projects I worked on, customers requested to receive automated notifications if unwanted changes in the IT Infrastructure occurred. 

In these projects, I mainly used  Terraform  and  Ansible  for deploying and configuring the Infrastructures – I noticed the power of Terraform  for detecting and even remediating unwanted changes to the initial deployments.

In this post, I want to introduce the possibility to check, notify, and remediate unwanted changes periodically.

Introducing the Config Drift Notification Script

Let´s assume you have deployed a Citrix Desktop-as-a-Service or Citrix Virtual Apps and Desktops environment according to your needs and your best practises – you want to keep it in this state.

As we all know, there are many settings, policies, and configurations in such a site – alterations may have serious consequences for stability, user experience, or even lead to a breakdown of the site.

Leveraging  Terraform  and an adjacent  PowerShell  script can help you to check, notify, and remediate unwanted changes periodically.

I used the initial version of the script as a foundation and added some more functionalities like:

• Correlating the detected changes with the entries in the Configuration Logging database
• Automatic notification using different notification methods like Webhooks, SMS Push, or e-Mail (default)
• Automatic remediation of the changes if wanted (disabled by default)
• Integrating the script in Azure DevOps

Prerequisites

To leverage the script, your environment needs to be fully mapped into  Terraform .
That implies, your Infrastructure must be fully managed by  Terraform .
If  Terraform  is not able to recognize an entity or configuration, it cannot check it for changes!

Terraform´s representation is the so-called  .tfstate  file – you can find more information about Terraform´s state file and how to create it from an existing infrastructure in our Citrix Tech Zone guide: Using Terraform for Daily Administrative Operations.

IMPORTANT:
Create a .tfstate  file as soon as your initial deployment is complete and all configurations fulfill your needs – it will be used as the foundation for all further steps.

If you have your  .tfstate  file ready, it is a valid representation of your Infrastructure as Infrastructure-as-Code ( IaC ).

Running the script

If you run the script, it creates an actual representation of your Infrastructure and compares it to the saved  .tfstate  file.
Any changes are detected and the notification process as well as the remediation process (if wanted) are triggered.

In this example, I changed the AutoScale settings of a Delivery Group and changed 2 settings in the Policy Set.

Let´s start the script manually – stay tuned for the next post, where I will showcase the implementation into Azure DevOps.

I used parameters to disable the Automatic Remediation of the changes and to use e-Mail as the Notification method.

After 10 seconds, I got the Notification mail:

You can now be sure to get notified, if (unwanted) changes in your Infrastructure occur.

If you set the parameter  -AutoRemediate $true , then the script will immediately remediate the changes.

CAUTION:
There is no way to stop the remediation after it has started!
Be sure you understand possible consequences of remediation in a production environment!
 

Summary

Combining  Terraform´s  power and the  APIs of Citrix DaaS/Citrix CVAD  enables you to periodically scan your environment for unwanted changes and remediate these if wanted.

Stay tuned for the next episode – Automating this process using  Azure DevOps .