Azure Pipelines¶

Four pipelines carry the flow — each block below is titled with its filename and heavily commented so you can see which code is for which job:

Pipeline	Purpose	Runs on
`wiki-ci.yml`	Validate every documentation PR	PR to `main`
`quality-security.yml`	Dependency check · code quality · security scan	PR + `main`
`wiki-publish.yml`	Publish on merge	push to `main`
`docs-site/azure-pipelines.yml`	Build & deploy the MkDocs site	changes under `docs-site/**`

All pipelines pull secrets from Key Vault-linked variable groups — no secrets in YAML.

1. `wiki-ci.yml` — validate every PR¶

Runs the reference-implementation checks (front-matter contract, links/orphans, hand-edit guard, spellcheck). Set as required build validation on the wiki repo's main branch policy.

wiki-ci.yml

# Validates documentation pull requests before they can merge.
trigger: none                       # never runs on push...
pr:
  branches: { include: [main] }     # ...only on PRs targeting main

pool: { vmImage: ubuntu-latest }    # or a self-hosted internal pool

variables:
  AUTOMATION_AUTHOR_EMAIL: 'docbot@automation.local'  # identity allowed to edit /_generated

jobs:
  - job: Validate
    steps:
      - checkout: self
        fetchDepth: 0                # full history so the hand-edit guard can diff
        persistCredentials: true
      - task: UsePythonVersion@0
        inputs: { versionSpec: '3.12' }
      - script: pip install --quiet pyyaml
        displayName: Install deps
      # Every page must have valid front-matter (owner, last_reviewed, sources...)
      - script: python tools/validators/front_matter.py .
        displayName: Front-matter contract
      # No broken internal links, no orphan pages
      - script: python tools/validators/link_check.py .
        displayName: Links + orphans
      # Machine-owned pages under /_generated may only be changed by the bot identity
      - script: |
          BASE=$(git merge-base origin/$(System.PullRequest.TargetBranch) HEAD)
          python tools/validators/hand_edit_guard.py "$BASE" HEAD
        displayName: Hand-edit guard (/_generated)
        env: { AUTOMATION_AUTHOR_EMAIL: $(AUTOMATION_AUTHOR_EMAIL) }
      # Spelling is advisory — warns but does not block
      - script: npx --yes cspell "**/*.md" --no-progress --no-summary --gitignore || true
        displayName: Spellcheck (advisory)
        continueOnError: true

2. `quality-security.yml` — dependency check, code quality & security¶

This is the pipeline that runs dependency checking, code quality, and security scanning across the automation code, the docs, and the pipeline YAML. Each stage below is labelled with what it does and why.

quality-security.yml

# Dependency check + code quality + security scanning.
# Runs on PRs (gate) and on main (baseline). Fails the build on real problems;
# advisory-only steps are marked continueOnError.
trigger:
  branches: { include: [main] }
pr:
  branches: { include: [main] }

pool: { vmImage: ubuntu-latest }

stages:
  # ---------------------------------------------------------------------------
  # STAGE A — DEPENDENCY CHECK
  # Finds known-vulnerable / outdated dependencies in the Python automation code.
  # ---------------------------------------------------------------------------
  - stage: Dependencies
    displayName: Dependency check
    jobs:
      - job: deps
        steps:
          - task: UsePythonVersion@0
            inputs: { versionSpec: '3.12' }
          # pip-audit: scans deps against the PyPI advisory DB (CVEs).
          # The automation project uses uv/pyproject, so export a lockfile first.
          - script: |
              cd automation
              pip install --quiet pip-audit uv
              uv export --format requirements-txt --no-emit-project > /tmp/reqs.txt
              pip-audit -r /tmp/reqs.txt --strict
            displayName: pip-audit (Python CVEs)
          # Trivy filesystem scan: catches vulnerable deps + lockfiles across langs
          - script: |
              curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
              trivy fs --scanners vuln --severity HIGH,CRITICAL --exit-code 1 .
            displayName: Trivy dependency scan (HIGH/CRITICAL fail)

  # ---------------------------------------------------------------------------
  # STAGE B — CODE QUALITY
  # Linting/formatting for Python, YAML and Markdown. Keeps the codebase clean.
  # ---------------------------------------------------------------------------
  - stage: Quality
    displayName: Code quality
    dependsOn: []                     # run in parallel with Dependencies
    jobs:
      - job: quality
        steps:
          - task: UsePythonVersion@0
            inputs: { versionSpec: '3.12' }
          # Ruff: fast Python linter (style, bugs, imports)
          - script: |
              pip install --quiet ruff
              ruff check automation/
            displayName: Ruff (Python lint)
          # yamllint: validates pipeline + config YAML syntax/style
          - script: |
              pip install --quiet yamllint
              yamllint -d relaxed .azuredevops/ docs-site/ || true
            displayName: yamllint (YAML style)
            continueOnError: true
          # markdownlint: consistent Markdown across the wiki
          - script: npx --yes markdownlint-cli2 "**/*.md" "#node_modules" || true
            displayName: markdownlint (docs)
            continueOnError: true

  # ---------------------------------------------------------------------------
  # STAGE C — SECURITY SCAN
  # Secret detection, Python SAST, and IaC/config misconfig scanning.
  # ---------------------------------------------------------------------------
  - stage: Security
    displayName: Security scan
    dependsOn: []                     # run in parallel too
    jobs:
      - job: security
        steps:
          - checkout: self
            fetchDepth: 0             # full history for secret scanning
          # gitleaks: detects committed secrets/credentials in code + history
          - script: |
              curl -sfL https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks_linux_x64.tar.gz | tar xz
              ./gitleaks detect --source . --redact --exit-code 1
            displayName: gitleaks (secret detection)
          # Bandit: static application security testing for Python
          - script: |
              pip install --quiet "bandit[toml]"
              bandit -c automation/pyproject.toml -r automation/ -ll   # -ll = medium+ severity
            displayName: Bandit (Python SAST)
          # Trivy config: scans IaC / pipeline / Dockerfiles for misconfig
          - script: |
              trivy config --severity HIGH,CRITICAL --exit-code 1 . || \
                (echo "Install trivy first if not cached"; \
                 curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin && \
                 trivy config --severity HIGH,CRITICAL --exit-code 1 .)
            displayName: Trivy config (IaC misconfig)
          # Optional: GitHub Advanced Security / CodeQL if licensed
          # - task: AdvancedSecurity-Codeql-Init@1
          # - task: AdvancedSecurity-Codeql-Analyze@1

What each tool covers

Dependency check — pip-audit (Python CVEs), trivy fs (multi-language vulnerable deps).
Code quality — ruff (Python), yamllint (YAML), markdownlint (docs).
Security — gitleaks (secrets), bandit (Python SAST), trivy config (IaC/config misconfig). Fail-the-build steps use --exit-code 1 / --strict; style checks are advisory (continueOnError).

3. `wiki-publish.yml` — publish on merge¶

wiki-publish.yml

# Publishes documentation after a PR is merged to main.
trigger:
  branches: { include: [main] }     # runs on push to main (i.e. after merge)
pr: none                            # never runs on PRs

pool: { vmImage: ubuntu-latest }

variables:
  - group: wiki-publish             # Key Vault-linked variable group (no secrets in YAML)

stages:
  - stage: Publish
    jobs:
      - job: Sync
        steps:
          - checkout: self
          # Code wiki = no-op (Git IS the wiki). Project wiki = sync via REST here.
          - script: echo "If Project wiki, sync via REST. If Code wiki, no-op."
            displayName: Publish content
          # Kick off the docs-site build so the rendered site refreshes
          - task: TriggerBuild@4
            displayName: Trigger docs-site build
            inputs:
              definitionIsInCurrentTeamProject: true
              buildDefinition: 'docs-site-build'

4. `docs-site/azure-pipelines.yml` — build & deploy the site¶

docs-site/azure-pipelines.yml

# Builds the MkDocs Material site (strict), secret-scans, then deploys behind Entra ID.
trigger:
  branches: { include: [main] }
  paths: { include: [docs-site/**] }   # only when docs change
pr:
  branches: { include: [main] }
  paths: { include: [docs-site/**] }

pool: { vmImage: ubuntu-latest }

variables:
  - group: docs-site-secrets           # Key Vault-linked (may be empty for MVP)

stages:
  - stage: Build
    jobs:
      - job: build
        steps:
          - checkout: self
          - task: UsePythonVersion@0
            inputs: { versionSpec: '3.12' }
          - script: pip install -r docs-site/requirements.txt
            displayName: Install MkDocs Material
          # Defence-in-depth: block publish if anything secret-shaped is committed
          - script: |
              set -e
              if grep -RInaE '(BEGIN [A-Z ]*PRIVATE KEY|AKIA[0-9A-Z]{16}|xox[baprs]-)' docs-site/docs; then
                echo "##vso[task.logissue type=error]Potential secret in docs. Blocking."; exit 1
              fi
              echo "No secrets found."
            displayName: Secret scan
          # --strict = broken links / nav fail the build
          - script: cd docs-site && mkdocs build --strict --site-dir "$(Build.ArtifactStagingDirectory)/site"
            displayName: mkdocs build --strict
          - publish: $(Build.ArtifactStagingDirectory)/site
            artifact: site

  - stage: Deploy
    dependsOn: Build
    condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    jobs:
      - deployment: deploy
        environment: docs-prod           # attach approval checks to this environment
        strategy:
          runOnce:
            deploy:
              steps:
                - download: current
                  artifact: site
                # Recommended target: Azure Static Web Apps (cheap + Entra ID auth)
                # - task: AzureStaticWebApp@0
                #     inputs:
                #       app_location: "$(Pipeline.Workspace)/site"
                #       skip_app_build: true
                #       azure_static_web_apps_api_token: $(SWA_DEPLOY_TOKEN)  # from Key Vault

How access is secured¶

Concern	Control
Secrets in pipelines	Key Vault-linked variable groups; no secrets in YAML
Vulnerable dependencies	`pip-audit` + `trivy fs` (Stage A) fail on HIGH/CRITICAL
Code quality	`ruff` / `yamllint` / `markdownlint` (Stage B)
Committed secrets	`gitleaks` (Stage C) fails the build
Insecure code / IaC	`bandit` + `trivy config` (Stage C)
Who can merge	Branch policy: PR + required `wiki-ci` + `quality-security` + reviewer
Machine content integrity	`hand_edit_guard.py` blocks human edits under `/_generated`
Leaks into the site	Secret scan step in the build
Who can view the site	Deploy behind Entra ID (Static Web Apps / App Service Easy Auth)

Hosting recommendation

Azure Static Web Apps with Entra ID auth is the cheapest, most secure target: static output, per-user auth, free/low tier. App Service Easy Auth is the alternative if you need private-endpoint network isolation.

Azure Pipelines¶

1. wiki-ci.yml — validate every PR¶

2. quality-security.yml — dependency check, code quality & security¶

3. wiki-publish.yml — publish on merge¶

4. docs-site/azure-pipelines.yml — build & deploy the site¶