GitHub Copilot Instructions for Infrastructure (infra) Folder

Overview

This folder contains Terraform infrastructure-as-code (IaC) for managing the Reports application infrastructure. The configuration manages resources across multiple providers including AWS, Heroku, GitHub, and New Relic.

Technology Stack

• Terraform: >= 1.5.7
• Providers:
  • GitHub (~> 6.0) - Repository and organization management
  • AWS (~> 5.35) - Cloud infrastructure (S3, ACM, IAM)
  • Heroku (~> 5.2) - Application hosting
  • New Relic - Application monitoring
  • SOPS (~> 0.5) - Secrets management

Setup Instructions

Prerequisites

1. Install Terraform (>= 1.5.7):

# Using tfenv (recommended)
tfenv install 1.5.7
tfenv use 1.5.7

# Or download from https://www.terraform.io/downloads

2. Install and configure required CLI tools:

# GitHub CLI (for authentication)
gh auth login

# AWS CLI (for AWS resources)
aws configure

# Heroku CLI (for Heroku resources)
heroku login

# SOPS (for secrets management)
# Install from https://github.com/mozilla/sops

Initialize Terraform

cd infra

# Initialize Terraform and download providers
terraform init

# Validate configuration
terraform validate

# Plan changes (dry-run)
terraform plan

# Apply changes (with approval)
terraform apply

File Structure

• main.tf - Provider configurations and version requirements
• github.tf - GitHub repository and organization resources
• heroku.tf - Heroku application and add-ons configuration
• iam.tf - AWS IAM roles and policies
• s3.tf - AWS S3 buckets and configurations
• acm.tf - AWS Certificate Manager (SSL certificates)
• newrelic.tf - New Relic monitoring configuration
• *.env - SOPS-encrypted environment files with secrets
• .sops.yaml - SOPS configuration for encryption
• .terraform.lock.hcl - Terraform provider version lock file

Secrets Management

Sensitive data is encrypted using SOPS and stored in encrypted .env files:

• newrelic.env - New Relic API credentials
• ci_secrets.env - CI/CD secrets
• ga-reports.env - Production environment variables
• ga-reports-staging.env - Staging environment variables

Working with Encrypted Files

# Edit encrypted file
sops newrelic.env

# View encrypted file
sops -d newrelic.env

# Encrypt a new file
sops -e file.env > encrypted.env

Coding Standards & Best Practices

Terraform Style

1. Formatting: Always format code with terraform fmt

terraform fmt -recursive

2. Validation: Validate configuration before committing

terraform validate

3. Code Organization:

   • Group resources by service/provider in separate files
   • Use descriptive resource names that indicate their purpose
   • Add comments for complex configurations
   • Use variables for repeated values

4. State Management:

   • Never commit .tfstate files
   • Use remote state backend for team collaboration
   • Always run terraform plan before apply

5. Security:

   • Never commit plain-text secrets
   • Always use SOPS for sensitive data
   • Use least-privilege IAM policies
   • Enable resource encryption where applicable

Resource Naming Conventions

• Use lowercase with hyphens: reports-staging-app
• Include environment: reports-production-bucket
• Be descriptive: github-actions-deploy-key

Common Tasks

Adding a New AWS Resource

1. Create or edit the appropriate .tf file (e.g., s3.tf for S3 buckets)
2. Define the resource with appropriate configuration
3. Format and validate:

terraform fmt
terraform validate

4. Plan and review changes:

terraform plan

5. Apply if changes look correct:

terraform apply

Updating Heroku Configuration

1. Edit heroku.tf
2. Update app configuration, add-ons, or environment variables
3. Run terraform plan to preview changes
4. Apply changes with terraform apply

Managing GitHub Resources

1. Edit github.tf
2. Add/modify repository settings, secrets, or webhooks
3. Authenticate with GitHub CLI: gh auth status
4. Apply changes with terraform

Rotating Secrets

1. Update the secret in the appropriate service (AWS, Heroku, etc.)
2. Update the encrypted .env file:

sops newrelic.env
# Edit the value
# Save and exit

3. Run terraform plan to verify changes
4. Apply with terraform apply

Development Workflow

Before Making Changes

1. Ensure you're on the latest code:

git pull origin main

2. Initialize/update Terraform:

terraform init -upgrade

3. Verify current state:

terraform plan

Making Changes

1. Edit the appropriate .tf files
2. Format your code:

terraform fmt

3. Validate syntax:

terraform validate

4. Review planned changes:

terraform plan

5. Test in staging environment if possible
6. Apply changes:

terraform apply

After Changes

1. Commit changes including .terraform.lock.hcl if providers updated
2. Document significant infrastructure changes
3. Update this documentation if adding new patterns

CI/CD Integration

The GitHub Actions workflow .github/workflows/copilot-setup-steps.yml should include terraform setup for infrastructure validation:

- name: Setup Terraform
  uses: hashicorp/setup-terraform@v3
  with:
    terraform_version: 1.5.7

- name: Terraform Init
  run: |
    cd infra
    terraform init

Troubleshooting

Common Issues

1. Provider Authentication Failures

   • Verify CLI authentication: gh auth status, aws sts get-caller-identity
   • Check environment variables for API tokens
   • Ensure SOPS can decrypt files

2. State Lock Issues

   • Wait for other operations to complete
   • Use terraform force-unlock only if necessary

3. Plan Shows Unexpected Changes

   • Check if manual changes were made outside Terraform
   • Review recent commits for configuration drift
   • Consider importing resources with terraform import

4. SOPS Decryption Errors

   • Verify you have the correct decryption keys
   • Check .sops.yaml configuration
   • Ensure age or PGP keys are properly configured

Important Notes

1. State Files: Never commit .tfstate files - they contain sensitive information
2. Provider Versions: Keep provider versions pinned to avoid unexpected changes
3. Testing: Always run terraform plan before apply
4. Backup: Keep backups of state files if using local state
5. Documentation: Update this file when adding new infrastructure patterns

Security Considerations

1. Use SOPS for all secrets - never commit plain-text credentials
2. Enable encryption at rest for S3 buckets and databases
3. Use least-privilege IAM policies
4. Regularly rotate access keys and tokens
5. Enable MFA for production infrastructure access
6. Review and audit infrastructure changes regularly

Getting Help

• Terraform Documentation: https://www.terraform.io/docs
• AWS Provider Docs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs
• Heroku Provider Docs: https://registry.terraform.io/providers/heroku/heroku/latest/docs
• GitHub Provider Docs: https://registry.terraform.io/providers/integrations/github/latest/docs
• SOPS Documentation: https://github.com/mozilla/sops

When Working with Infrastructure Code

1. Always Plan First: Run terraform plan before any apply
2. Small Changes: Make incremental changes and test frequently
3. Documentation: Document complex configurations inline
4. State Awareness: Be aware of state management and locking
5. Security First: Never compromise on security for convenience
6. Validate: Use terraform validate and terraform fmt before committing
7. Review: Carefully review plan output before applying
8. Backup: Ensure state files are backed up (if not using remote state)