Interacting with Other Applications
An important thing that is needed is a way to run other applications and get their results. In .Net there is a Process class that can start other apps and get their output. It can be a bit verbose to use this by itself so I created a little wrapper around it called Threax.ProcessHelper that makes it a bit easier to use. That project also includes a wrapper to write inline powershell code and execute it through the Process class in a safe way. This is helpful when you need to do more advanced actions to make linked programs work. Especially in the case of using a powershell based command set.
Next take these two libraries and start building out common libraries. The reference implementation uses Az Powershell, so a library was created to wrap the most common commands to make them easier to use inside our actual provisioner controllers. If you look at the reference implementation you will find the wrapped az powershell commands and a unit test project for them. You will want to be able to validate that your wrappers are working independently from running the actual provisioner, so a unit test project is helpful. Do note that you will probably have to run the tests by hand 1 at a time, since things may need to be created for other commands to work.
I've gone back and forth if it makes sense to have a wrapper around the commands. I like that it enforces standard usage, such as needing to always specify a resource group to look something up. Sometimes cloud cli tools are a bit "helpful" and infer data that you may wish to enforce be provided. Wrapping the functions does this. It also provides abstraction so you could potentially swap things out and traceability for breaking changes. I think its a good idea in the long term, but may not be worth the effort either initially or if time is a big factor. Please note that the better job you do on the lower levels improves the experience at higher ones.
Make sure to check return codes and throw exceptions when there are problems. Don't keep running commands that won't work now that something has gone wrong. Restarting after an issue can be handled on a higher level.