Allow accessing private terraform modules (github)

[frank-bee]

Currently in any github pipeline where you run terraform and access private modules , this is very ugly.
You have to do something like this in your pipeline

      - name: Set git config
        run: |
          /usr/bin/git config --global --add url."https://${{ secrets.GH_PAT_TOKEN }}@github.com/MY_ORG".insteadOf https://github.com/MY_ORG

Would be great to handle this "natively" in digger.

Related also to #219 ( when running digger locally you also need access to all module repos)

[motatoes]

Ah indeed this is a good case to handle. Just dropping this here since it seems related to the issue: #111 (comment)

[frank-bee]

@motatoes
Atlantis implementation for that was done in this PR: runatlantis/atlantis#711

[frank-bee]

I found a way to access private repos with digger ( or any other terraform github workflow ).
Just want to share it here (replace with your github org)

name: Digger (Terraform)

on:
  pull_request:
    branches: [ "main" ]
    types: [ closed, opened, synchronize, reopened ]
  issue_comment:
    types: [created]
    if: contains(github.event.comment.body, 'digger')
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      #To allow checkout this project
      contents: read

      #To allow Digger to access PRs
      issues: write
      pull-requests: write
    steps:
      # create a temp. token from the github app installed in the org.
      # (The app needs to have read access to all private terraform repos  in the org )
      # this token is used in the next step to allow cloning terraform module repos
      - name: Get Token
        id: get_workflow_token
        uses: peter-murray/workflow-application-token-action@v2
        with:
          application_id: ${{ secrets.DIGGER_APPLICATION_ID }}
          application_private_key: ${{ secrets.DIGGER_APPLICATION_PRIVATE_KEY }}
          organization: <my org>

      # allow terraform to download our private modules
      - name: Set git config
        run: |
          /usr/bin/git config --global --add url."https://github:$GITHUB_TOKEN_READ_REPOS@github.com/<my org>".insteadOf https://github.com/<my org>
        env:
          GITHUB_TOKEN_READ_REPOS: ${{ steps.get_workflow_token.outputs.token }}

      - name: digger run
        uses: diggerhq/digger@v0.1.21
        env:
          GITHUB_CONTEXT: ${{ toJson(github) }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # here the workflow token is used implicitly
          GITHUB_OWNER: <my org>

[motatoes]

Thank you so much for sharing this @frank-bee , we will adopt something similar when we get to implementing support for digger for sure!

[frank-bee]

Any news here?

[motatoes]

hey @frank-bee haven't had a look at this but it looks like the atlantis-inspired implementation is the way to go, I'll take a look at this but got alot on my plate right now so it might be a bit of time

[ebuildy]

You can use env variables to configure git credentials, I think this is even more secure, see my blog post at https://dev.to/ebuildy/use-terraform-module-from-a-private-git-project-with-atlantis-18ak

GIT_CONFIG_COUNT=1
GIT_CONFIG_KEY_0="credential.https://gitlab.com/company/terraform.helper"
GIT_CONFIG_VALUE_0=!f() { echo "username=${ATLANTIS_GITLAB_USER}"; echo "password=${ATLANTIS_GITLAB_TOKEN}"; }; f

[Roberdvs]

I found a way to access private repos with digger ( or any other terraform github workflow ). Just want to share it here (replace with your github org)

name: Digger (Terraform)

on:
  pull_request:
    branches: [ "main" ]
    types: [ closed, opened, synchronize, reopened ]
  issue_comment:
    types: [created]
    if: contains(github.event.comment.body, 'digger')
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      #To allow checkout this project
      contents: read

      #To allow Digger to access PRs
      issues: write
      pull-requests: write
    steps:
      # create a temp. token from the github app installed in the org.
      # (The app needs to have read access to all private terraform repos  in the org )
      # this token is used in the next step to allow cloning terraform module repos
      - name: Get Token
        id: get_workflow_token
        uses: peter-murray/workflow-application-token-action@v2
        with:
          application_id: ${{ secrets.DIGGER_APPLICATION_ID }}
          application_private_key: ${{ secrets.DIGGER_APPLICATION_PRIVATE_KEY }}
          organization: <my org>

      # allow terraform to download our private modules
      - name: Set git config
        run: |
          /usr/bin/git config --global --add url."https://github:$GITHUB_TOKEN_READ_REPOS@github.com/<my org>".insteadOf https://github.com/<my org>
        env:
          GITHUB_TOKEN_READ_REPOS: ${{ steps.get_workflow_token.outputs.token }}

      - name: digger run
        uses: diggerhq/digger@v0.1.21
        env:
          GITHUB_CONTEXT: ${{ toJson(github) }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # here the workflow token is used implicitly
          GITHUB_OWNER: <my org>

Thanks for the pointer to make this work. In our case the modules were hosted in the same private repository than the Terraform project that triggers the Digger workflow (but we use git tags for versioning and promoting changes between envs) so the config is a bit simpler since there's no need to get a GitHub App token for "cross-repo access" as we can just use the workflow token.

      - name: Set git config
        run: |
          /usr/bin/git config --global --add url."https://github:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository_owner }}".insteadOf https://github.com/${{ github.repository_owner }}

Would be great if Digger could leverage for this purpose the GitHub Token that's already passed as an input to the action.

[dixitnh]

Is there any information or planned timeline for when this feature might be introduced in Digger? Just checking in to understand if it's on the roadmap or under consideration.