GitHub Copilot Instructions for Infrastructure (infra) Folder

Overview

This folder contains Terraform infrastructure-as-code (IaC) for managing the Reports application infrastructure.

infra/ itself is the Spacelift root (Cloudflare, CloudFront, S3, IAM, ACM, Firebase, Datadog, the GitHub repository). The application environments are separate roots — production/, staging/ and reviews/ — applied by deploy.yml against the shared S3 backend. Everything below describes the Spacelift root unless it says otherwise.

Technology Stack

• Terraform: >= 1.5.7
• Providers:
  • GitHub (~> 6.0) - Repository and organization management
  • AWS (>= 6.0) - Cloud infrastructure (S3, ACM, IAM, CloudFront)
  • Cloudflare - DNS
  • Datadog (~> 4.0) - Monitors and Error Tracking
  • Google Beta (~> 7.0) - Firebase projects
  • SOPS (~> 0.5) - Secrets management

Setup Instructions

Prerequisites

1. Install Terraform (>= 1.5.7):

# Using tfenv (recommended)
tfenv install 1.5.7
tfenv use 1.5.7

# Or download from https://www.terraform.io/downloads

2. Install and configure required CLI tools:

# GitHub CLI (for authentication)
gh auth login

# AWS CLI (for AWS resources)
aws configure

# SOPS (for secrets management)
# Install from https://github.com/mozilla/sops

Initialize Terraform

cd infra

# Initialize Terraform and download providers
terraform init

# Validate configuration
terraform validate

# Plan changes (dry-run)
terraform plan

# Apply changes (with approval)
terraform apply

File Structure

• main.tf - Provider configurations and version requirements
• github.tf - GitHub repository and organization resources
• iam.tf - AWS IAM roles and policies
• s3.tf - AWS S3 buckets and configurations
• acm.tf - AWS Certificate Manager (SSL certificates)
• cloudflare.tf - DNS records
• cloudfront.tf - CloudFront distributions
• datadog.tf - Monitors and Error Tracking rules
• firebase.tf - Firebase projects
• production/, staging/, reviews/ - Cluster roots, one per environment
• modules/app - Shared app module the cluster roots instantiate
• *.env, *.json - SOPS-encrypted files with secrets
• .sops.yaml - SOPS configuration for encryption
• .terraform.lock.hcl - Terraform provider version lock file

Secrets Management

Sensitive data is encrypted using SOPS and stored in encrypted .env files:

• ci_secrets.env - CI/CD secrets
• ga-reports-common.env - Secrets shared by every environment
• ga-reports.env - Production-only secrets
• ga-reports-staging.env - Secrets for staging and the review apps
• firebase-service-account-*.json - Firebase service accounts

Working with Encrypted Files

# Edit encrypted file
sops ga-reports.env

# View encrypted file
sops -d ga-reports.env

# Encrypt a new file
sops -e file.env > encrypted.env

Coding Standards & Best Practices

Terraform Style

1. Formatting: Always format code with terraform fmt

terraform fmt -recursive

2. Validation: Validate configuration before committing

terraform validate

3. Code Organization:

   • Group resources by service/provider in separate files
   • Use descriptive resource names that indicate their purpose
   • Add comments for complex configurations
   • Use variables for repeated values

4. State Management:

   • Never commit .tfstate files
   • Use remote state backend for team collaboration
   • Always run terraform plan before apply

5. Security:

   • Never commit plain-text secrets
   • Always use SOPS for sensitive data
   • Use least-privilege IAM policies
   • Enable resource encryption where applicable

Resource Naming Conventions

• Use lowercase with hyphens: reports-staging-app
• Include environment: reports-production-bucket
• Be descriptive: github-actions-deploy-key

Common Tasks

Adding a New AWS Resource

1. Create or edit the appropriate .tf file (e.g., s3.tf for S3 buckets)
2. Define the resource with appropriate configuration
3. Format and validate:

terraform fmt
terraform validate

4. Plan and review changes:

terraform plan

5. Apply if changes look correct:

terraform apply

Updating an Application Environment

1. Edit the root under production/, staging/ or reviews/, or the shared module under modules/
2. Open a pull request — deploy.yml runs the plan
3. Merge. Staging applies it straight away (staging.yml also runs on the infra paths the staging root reads). Production applies it with the next release, or earlier by dispatching deploy.yml from main with the current release tag: see "Apply infra changes without a release, or roll back" in infra/production/README.md

Managing GitHub Resources

1. Edit github.tf
2. Add/modify repository settings, secrets, or webhooks
3. Authenticate with GitHub CLI: gh auth status
4. Apply changes with terraform

Rotating Secrets

1. Update the secret in the appropriate service
2. Update the encrypted .env file:

sops ga-reports.env
# Edit the value
# Save and exit

3. Run terraform plan to verify changes
4. Apply with terraform apply

Development Workflow

Before Making Changes

1. Ensure you're on the latest code:

git pull origin main

2. Initialize/update Terraform:

terraform init -upgrade

3. Verify current state:

terraform plan

Making Changes

1. Edit the appropriate .tf files
2. Format your code:

terraform fmt

3. Validate syntax:

terraform validate

4. Review planned changes:

terraform plan

5. Test in staging environment if possible
6. Apply changes:

terraform apply

After Changes

1. Commit changes including .terraform.lock.hcl if providers updated
2. Document significant infrastructure changes
3. Update this documentation if adding new patterns

CI/CD Integration

The GitHub Actions workflow .github/workflows/copilot-setup-steps.yml should include terraform setup for infrastructure validation:

- name: Setup Terraform
  uses: hashicorp/setup-terraform@v3
  with:
    terraform_version: 1.5.7

- name: Terraform Init
  run: |
    cd infra
    terraform init

Troubleshooting

Common Issues

1. Provider Authentication Failures

   • Verify CLI authentication: gh auth status, aws sts get-caller-identity
   • Check environment variables for API tokens
   • Ensure SOPS can decrypt files

2. State Lock Issues

   • Wait for other operations to complete
   • Use terraform force-unlock only if necessary

3. Plan Shows Unexpected Changes

   • Check if manual changes were made outside Terraform
   • Review recent commits for configuration drift
   • Consider importing resources with terraform import

4. SOPS Decryption Errors

   • Verify you have the correct decryption keys
   • Check .sops.yaml configuration
   • Ensure age or PGP keys are properly configured

Important Notes

1. State Files: Never commit .tfstate files - they contain sensitive information
2. Provider Versions: Keep provider versions pinned to avoid unexpected changes
3. Testing: Always run terraform plan before apply
4. Backup: Keep backups of state files if using local state
5. Documentation: Update this file when adding new infrastructure patterns

Security Considerations

1. Use SOPS for all secrets - never commit plain-text credentials
2. Enable encryption at rest for S3 buckets and databases
3. Use least-privilege IAM policies
4. Regularly rotate access keys and tokens
5. Enable MFA for production infrastructure access
6. Review and audit infrastructure changes regularly

Getting Help

• Terraform Documentation: https://www.terraform.io/docs
• AWS Provider Docs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs
• GitHub Provider Docs: https://registry.terraform.io/providers/integrations/github/latest/docs
• SOPS Documentation: https://github.com/mozilla/sops

When Working with Infrastructure Code

1. Always Plan First: Run terraform plan before any apply
2. Small Changes: Make incremental changes and test frequently
3. Documentation: Document complex configurations inline
4. State Awareness: Be aware of state management and locking
5. Security First: Never compromise on security for convenience
6. Validate: Use terraform validate and terraform fmt before committing
7. Review: Carefully review plan output before applying
8. Backup: Ensure state files are backed up (if not using remote state)