Skip to content

High Level Overview#

The AzureRM Provider is a Plugin which is invoked by Terraform (Core) and comprised of Data Sources and Resources.

Within the AzureRM Provider, these Data Sources and Resources are grouped into Service Packages - which are logical groupings of Data Sources/Resources based on the Azure Service they're related to.

Each of these Data Sources and Resources has both Acceptance Tests and Documentation associated with each Data Source/Resource - the Acceptance Tests are also located within this Service Package, however the Documentation exists within a dedicated folder.

Project Structure#

The Azure Provider is a large codebase which has evolved over time - but tends to follow consistent patterns for the most-part.

The Provider is split up into Service Packages (see terminology) - with some other logic sprinkled across several packages.

At a high-level, the Provider structure is:

  • ./examples
    • Contains more complete example usages of Data Sources and Resources offered by this Provider.
  • ./helpers
    • This package is deprecated (and so intentionally not documented) - new functionality should instead be added to either the Service Package or go-azure-helpers.
  • ./internal
    • ./internal/acceptance
      • Contains the Acceptance Test wrappers that we use in the Azure Provider, offering common patterns across the Provider to be reused.
    • ./internal/clients
      • Contains a reference to the Client from each Service Package, which is used in Data Sources and Resources to access the Azure API’s.
    • ./internal/common
      • Contains helper functions for registering Clients (for example, setting the user agent, configuring credentials etc).
    • ./internal/features
      • Contains Feature Toggles for Provider functionality and behaviour (for example, enabling Betas or changing a resource type's soft delete or purge protection). This also contains the struct and parsing of/default values for the features block (within the Provider block).
    • ./internal/locks
      • Provides common locking across resources where necessary to workaround API consistency issues.
    • ./internal/provider
      • Contains the Provider implementation itself, the Provider schema and a reference to each Service Registration so that Data Sources and Resources can be surfaced within the Provider.
    • ./internal/resourceid
      • This package is deprecated in favour of github.com/hashicorp/go-azure-helpers/resourcemanager/resourceids and will be removed in the future.
    • ./internal/resourceproviders
      • Contains the list of Resource Providers which should be auto-registered by the Provider.
    • ./internal/sdk
      • Contains the Typed Plugin SDK functionality used in this Provider.
    • ./internal/services
      • Contains a packages for each service that the provider supports (e.g. appconfiguration, compute) in which are each's Data Sources and Resources.
    • ./internal/tags
      • Contains helpers for parsing Tags from the Terraform Configuration and setting Tags into the Terraform State.
    • ./internal/tf
      • Contains helpers and abstractions on top of the Terraform Plugin SDK.
    • ./internal/timeouts
      • Contains helpers for computing the Timeouts for a Data Source / Resource - used in Untyped Data Sources and Untyped Resources.
    • ./internal/tools
      • This package contains tooling used to generate functionality within the Provider, for example for Resource ID’s and Website Documentation.
  • ./scripts
    • Contains various scripts used during testing, linting, and building the provider.
  • ./utils
    • This primarily contains helper functions for converting simple types (e.g. bool/int/strings) to pointers (e.g. utils.String(“someValue”).
    • We intend to deprecate this folder in time and new functionality should be added to individual service packages where possible. The existing functions will be gradually moved (via aliasing) into another repository.
  • ./vendor
    • Contains the vendored copies of the go modules the provider uses. For more information please refer to the official Go Documentation.
  • ./website
    • Contains the guides and documentation for each resource (in ./website/docs/r) and data source (in ./website/docs/d) that are published to the Terraform registry.

Note: Due to the size of the codebase and open Pull Requests - when functionality is moved we use aliasing to try and avoid breaking open Pull Requests / big-bang migrations. These aliases stick around for a few weeks to allow open PR’s to be merged without extra out of scope changes - at which point these aliases are removed.

Each Service Package consists of (to take appconfiguration as an example):

  • ./internal/services/appconfiguration
    • ./client
      • Contains a Client struct, with a reference to any SDK Clients used to access the Azure API’s within this Service Package.
    • ./parse
      • Contains Resource ID Formatters and Parsers.
    • ./validate
      • Contains Validation functions for this Service Package, including Resource ID Validators.
    • ./app_configuration_data_source.go - the Data Source azurerm_app_configuration
    • ./app_configuration_data_source_test.go - Acceptance tests for the Data Source azurerm_app_configuration
    • ./app_configuration_key_resource.go - the Resource azurerm_app_configuration_key
    • ./app_configuration_key_resource_test.go - Acceptance Tests for the Resource azurerm_app_configuration_key
    • ./app_configuration_resource.go - the Resource azurerm_app_configuration
    • ./app_configuration_resource_test.go - Acceptance tests for the Resource azurerm_app_configuration
    • ./registration.go - the Service Registration for this Service Package

Some Service Packages may also contain:

  • ./migration - any State Migrations used in Resources.
  • ./sdk - any Embedded SDK’s used to access the Azure API’s (either Resource Manager or Data Plane).
  • ./resourceids.go - used to generate Resource ID Formatters, Parsers and Validators.

—-

  • Data Sources use the filename format: {name}_data_source.go
  • Acceptance Tests for Data Sources use the filename format: {name}_data_source_test.go (note: Golang requires that Tests are contained within a test.go file)
  • Resources use the filename format: {name}_resource.go
  • Acceptance Tests for Resources use the filename format: {name}_resource_test.go (note: Golang requires that Tests are contained within a test.go file)

Note: there are a handful of exceptions to these to reduce stuttering (e.g. Resource Provider Registration Resource)

Types of Data Sources/Resources within the Provider#

Whilst the Azure Provider is built on-top of the Terraform Plugin SDK - as this is a large codebase with a number of behavioural similarities across the Provider, we've added an abstraction atop the Terraform Plugin SDK to make development easier.

This means that at this point in time, there are four types of Data Source/Resources which can be added in this Provider:

  1. (Untyped) Data Sources (based on the Terraform Plugin SDK) (example)).
  2. (Untyped) Resources (based on the Terraform Plugin SDK) (example)).
  3. Typed Data Sources (based on top of the Typed SDK within this Repository) (example)).
  4. Typed Resources (based on top of the Typed SDK within this Repository) (example)).

At this point in time the codebase uses a mixture of both (primarily the Untyped Data Sources/Resources) - in time we plan to migrate across to using Typed Data Sources/Resources instead.

Ultimately this approach will allow us to switch from using the Terraform Plugin SDK to Terraform Plugin Framework, enabling us to fix a number of long-standing issues in the Provider - whilst reducing the TLOC needed for each resource.

Interaction with Azure#

This Provider makes use of a number of SDKs to interact with both the Azure Resource Manager and a number of associated Data Plane APIs, these are:

There's also a number of Embedded SDKs within the provider for interaction with Resource Manager Services which are not supported by the Azure SDK for Go - generated from the Swagger files within the Azure/azure-rest-api-specs repository.

At this point in time, each of the SDKs mentioned above (excluding Hamilton) make use of Azure/go-autorest as a base layer (e.g. for sending requests/responses/handling retries from Azure).

Testing the Provider#

Since the behaviour of the Azure API can change over time, the Provider leans on Acceptance Tests over Unit Tests for asserting that the Data Sources and Resources within the Provider work as expected.

More details and guidance on how to test Data Sources/Resources can be found in the Acceptance Testing reference.