Threax Provisioner

**WORK IN PROGRESS**  This is a work in progress with a huge number of spelling errors and other issues.

In DevOps we are presented with a huge variety of APIs, tools and operating environments. These range in functionality from bare bones cloud APIs to proprietary configuration environments like Azure ARM templates or even general purpose frameworks like Terraform that provide common functionality. There are container orchestration tools like Docker, Podman or Kubernetes and other tools to directly manage VMs. There are build and release pipeline creation tools like Azure DevOps or GitHub actions. The DevOps toolbox is large and each part of it requires its own individual configuration. I know the name of my service, now I need to duplicate it into 10 different config files in order to keep everything working. I know all the tech these vendors are pushing out. I can write a config file in YAML and JSON and even embed some programming actions into them. This whole thing is no problem at all.

Except it really kind of is. Take a step back for a second and ignore all the noise. What are we really creating? No matter the final format it might take: JSON, YAML, proprietary aren't these really just a whole bunch of objects that have been stored as strings? If that is all we are dealing with can't we create those objects in a different fashion than writing them into config files, copying and pasting them around everywhere and making small changes to a code base that is growing in an unbounded manner? Are we really requiring DevOps engineers to sit down and write what ultimately boils down to temporary data that is only needed in the moment?

Sit down and think about your app for a second. It doesn't really matter too much if it is microservices, a monolith or something in between. If you are doing services consider one of your services in isolation. I will refer to all of this as the app going forward. Think about your app's actual needs. You likely have some sort of compute somewhere that you need for the app to run. This could be a VM, a managed app service, functions as a service or some other place you can put some code an run it. It could be a raspberry pi sitting in an office or a huge Kubernetes cluster in a data center, it all depends on your needs. Next you might have a variety of data storage, such as a SQL server (in any flavor), MongoDB, a key value store etc. This goes on and on all the other stuff your apps use. However if you really boil all of it down there are probably only a few configuration items that really change between your apps and environments. Most everything else, no matter what form its stored in probably does not change very much.

With that in mind the goal of a provisioner is to take a very simple view of the configuration for your apps that is most important and turn it into the objects and commands to drive the other tools to do what you need. Nobody can make this for you since it will be extremely specific to the current project in the way that your application code is the same. If someone could do it they will have and it will be wrapped up in one of the millions of DevOps tools. The goal here is to eliminate the huge bundle of files and even GUI based web configuration screens that are needed to deploy apps down to a single config file per app that can be used to create all of that other stuff. This can then be used to create instances of your app much more easily and also helps with making changes since all the actual work goes through the application. You also gain the ability to version your infrastructure since you can manage the version of the program you are creating.

These pages will walk through an implementation of this I have done in C# for Azure. However, it is important to stress that this is a concept and should be possible in more technologies. I do believe that C# contains some features that make it really suitable for use this way. Also I will discuss each piece of what I did and why, but you could probably create something good if you left out some of them or added some things I didn't think of. Ultimately we should free ourselves from writing logic code in markup languages, which is inappropriate. Languages meant for programming not markup can provide a much richer environment.