Deployment Strategy

Overview

Zeron uses a multi-stage deployment strategy to ensure stability and enable safe testing before production releases.

Branch-Based Deployment Model

🚀 Production Branch

• Purpose: Stable production releases only
• Deployment: Manual promotion after validation
• Protection: Strict branch protection rules, required reviews
• Environment: Full production configuration

🧪 Staging Branch

• Purpose: Integration testing and pre-production validation
• Deployment: Automatic on merge
• Source: Merge target for all feature branches
• Environment: Staging configuration with isolated resources

🔧 Development Branch

• Purpose: Continuous integration for active development
• Deployment: Local/development only
• Function: Integration point for feature development
• Environment: Local development configuration

Deployment Workflow

Feature Branches → Staging → Production
     ↓              ↓           ↓
  Development   Testing     Live
   (Local)    (Validation) (Customers)

Development Workflow

1. Feature Development

• Start from staging branch
• Create isolated feature branch
• Develop and test locally
• Submit PR targeting staging

2. Staging Validation

• Automatic deployment to staging environment
• Integration testing with other systems
• QA validation and testing
• Performance verification

3. Production Deployment

• Manual promotion after staging validation
• Controlled rollout with monitoring
• Rollback capability maintained
• Health checks and verification

Environment Configuration

Environment Separation

• Production: Full production credentials and resources
• Staging: Isolated staging resources (separate databases, APIs)
• Development: Local development configuration

Database Migration Workflow

Managing database schema changes across environments is critical for maintaining stability and preventing data loss. This section outlines the standardized approach for database migrations in Zeron applications.

Three-Environment Database Architecture

CRITICAL: Each environment MUST use a completely separate database instance:

1. Local Development (Docker/Local)

   • Purpose: Development and testing of schema changes
   • Database: Local PostgreSQL via Docker Compose
   • Connection: Local connection string (never committed to Git)
   • Migrations: Created and tested here first

2. Staging (Cloud Database)

   • Purpose: Integration testing with production-like data
   • Database: Separate cloud database instance (e.g., Prisma Cloud, Neon, RDS)
   • Connection: Staging-specific connection string (stored in Vercel/deployment platform)
   • Migrations: Applied manually after staging deployment

3. Production (Cloud Database)

   • Purpose: Live customer data
   • Database: Separate production cloud database instance
   • Connection: Production-specific connection string (stored in Vercel/deployment platform)
   • Migrations: Applied manually after production deployment

Why Separate Databases?

• Multi-tenancy isolation: Prevents cross-environment data leaks
• Safe testing: Staging failures don't affect production
• Realistic testing: Staging can use production-like data volumes
• Security: Production credentials never touch development environments

Migration Workflow Principles

Manual Migrations Are Intentional

Database migrations MUST be run manually after code deployment, not automatically. This is a safety feature:

• ✅ Allows verification before applying schema changes
• ✅ Prevents cascading failures from bad migrations
• ✅ Enables emergency rollback without re-deploying code
• ✅ Provides clear audit trail of when migrations ran
• ❌ Automatic migrations can cause downtime during deployment
• ❌ Failed migrations can break entire deployment process

Never Use Development Migration Commands in Cloud

For Prisma-based projects:

• ❌ migrate dev - Only for local development (creates AND applies migrations)
• ✅ migrate deploy - For staging/production (applies existing migrations only)
• ❌ migrate reset - NEVER in cloud (destructive, deletes all data)

Golden Rules

1. Migrations are created and tested locally FIRST
2. Migrations are committed to Git alongside code changes
3. Code deploys automatically (via branch strategy)
4. Migrations deploy manually (via CLI commands)
5. Always verify migration status before running migrations

Standard Migration Workflow

Step 1: Create Migration Locally

# Create new migration
npm run db:migrate:create add_user_preferences

# Edit the generated migration SQL if needed
# Apply migration to local database
npm run db:migrate:dev

# Test your changes thoroughly
npm run dev

What's Happening:

• Prisma generates migration SQL based on schema.prisma changes
• Migration is applied to your local Docker database
• Migration files are created in prisma/migrations/ directory
• You verify the changes work correctly

Step 2: Commit and Push to Staging

# Commit migration files with your code changes
git add prisma/migrations/
git add prisma/schema.prisma
git add src/  # Your code changes
git commit -m "feat: Add user preferences table (ZER-123)"

# Push to staging branch
git push origin staging

What's Happening:

• Migration files are now in version control
• Vercel/deployment platform automatically deploys your code to staging
• Database schema has NOT changed yet (code deployment ≠ migration deployment)

Step 3: Apply Migrations to Staging

# Manually run migrations against staging database
npm run db:migrate:staging

# Verify migrations applied successfully
npm run db:migrate:staging:status

What's Happening:

• Command reads .env.staging for database connection string
• Prisma applies pending migrations to staging database
• Schema changes are now live in staging environment
• Your code can now interact with the new schema

Step 4: Test Staging (24-48 Hour Soak)

# Create test data if needed
npm run db:test-data:create

# Test thoroughly at staging URL
# - Integration tests
# - Manual QA
# - Performance verification
# - Edge cases

# Clean up test data when done
npm run db:test-data:cleanup

What's Happening:

• Staging environment runs with new schema for 24-48 hours
• Catch issues before they reach production
• Verify migrations work with production-like data volumes
• Test rollback procedures if needed

Step 5: Deploy to Production

# Merge staging into production branch
git checkout master
git merge staging
git push origin master

# Vercel/deployment platform automatically deploys code
# Wait for deployment to complete...

# Manually run migrations against production database
npm run db:migrate:production

# Verify migrations applied successfully
npm run db:migrate:production:status

What's Happening:

• Production code is now deployed (but using old schema)
• You manually apply migrations when ready
• Schema changes are now live in production
• Monitor for issues immediately after migration

Essential Migration Scripts

Every Zeron application with a database should implement these scripts in package.json:

Local Development Scripts

{
  "db:migrate:create": "Create new migration without applying",
  "db:migrate:dev": "Create and apply migration locally",
  "db:migrate:reset": "Reset local database (DESTRUCTIVE)",
  "db:migrate:status": "Check local migration status",
  "db:backup": "Backup local database before risky operations"
}

Staging Environment Scripts

{
  "db:migrate:staging": "Apply migrations to staging database",
  "db:migrate:staging:status": "Check staging migration status",
  "db:test-data:create": "Generate test data for staging",
  "db:test-data:cleanup": "Remove all test data from staging"
}

Production Environment Scripts

{
  "db:migrate:production": "Apply migrations to production database",
  "db:migrate:production:status": "Check production migration status"
}

Utility Scripts

{
  "db:verify": "Pre-deployment verification (CI/CD)",
  "db:migrate:rollback": "Interactive emergency rollback tool"
}

Implementation Notes:

• Use dotenv-cli to load environment-specific connection strings
• Staging scripts read from .env.staging (never committed)
• Production scripts read from .env.production (never committed)
• Example: "db:migrate:staging": "dotenv -e .env.staging -- prisma migrate deploy"

Handling Destructive Changes

Dropping columns or tables requires a 3-phase approach to prevent downtime and data loss:

Phase 1: Make Column Nullable (Migration 1)

// Update schema.prisma
model User {
  id       String  @id
  oldField String? // Add ? to make nullable
}

Actions:

1. Create migration: npm run db:migrate:create make_old_field_nullable
2. Deploy to staging → test → deploy to production
3. Wait for full deployment cycle (code in staging and production)

Why: Old code can still write to the field, new code ignores it

Phase 2: Remove Code References (No Migration)

// Remove all code that uses oldField
model User {
  id       String  @id
  oldField String? // Still in schema, but no code uses it
}

Actions:

1. Remove code references in your application
2. Deploy to staging → test → deploy to production
3. Verify no code accesses the field anymore

Why: Code is ready for field to disappear

Phase 3: Drop Column (Migration 2)

// Update schema.prisma - remove field entirely
model User {
  id String @id
  // oldField is gone
}

Actions:

1. Create migration: npm run db:migrate:create remove_old_field
2. Deploy to staging → test → deploy to production
3. Field is permanently deleted

Why: Safe to drop now - no code references it

Benefits of 3-Phase Approach:

• ✅ Zero downtime during migration
• ✅ Safe rollback at any phase
• ✅ Old and new code can coexist temporarily
• ✅ No race conditions between deployment and migration

Migration Rollback Procedures

Emergency Rollback (Critical Production Issues)

1. Revert Code Deployment

   • Use Vercel/deployment platform dashboard
   • Redeploy previous working version
   • Restores old code immediately (schema still changed)

2. Mark Migration as Rolled Back

   • Run: npm run db:migrate:rollback
   • Follow interactive prompts
   • Mark the problematic migration as "rolled back"

3. Create Reversal Migration

   • Run: npm run db:migrate:create reversal_of_problematic_change
   • Manually edit SQL to undo schema changes
   • Example: ALTER TABLE "users" DROP COLUMN "new_field";

4. Test Reversal in Staging

   • Deploy reversal migration to staging first
   • Verify old code works with reverted schema
   • Test thoroughly before production

5. Apply to Production

   • Merge to production branch
   • Run: npm run db:migrate:production
   • Monitor closely

Important: Migration rollback does NOT automatically undo schema changes. You must create a new migration that reverses the changes manually.

Planned Rollback (Testing/Development)

For local development only:

# Reset entire local database
npm run db:migrate:reset

# Restore from backup
npm run db:backup:restore <backup_file>

Never use migrate reset in staging or production!

Test Data Management

Creating Test Data (Staging Only)

# Generate realistic test data in staging
npm run db:test-data:create

What It Should Create:

• Multiple test accounts with different plan types
• Sample data for integration testing
• Edge cases (empty states, maximum limits)
• All records marked as isDemoAccount: true or similar flag

Safety Features:

• Production detection: Refuse to run if production database detected
• Clear labeling: All test data clearly marked as demo/test
• Easy cleanup: Simple command to remove all test data

Cleaning Up Test Data

# Remove all demo/test accounts and related data
npm run db:test-data:cleanup

Implementation:

• Query for all records with isDemoAccount: true
• Cascade delete all related data
• Require confirmation: User types "DELETE" to proceed
• Never touches real user data

Production Rule

NEVER seed or generate data in production!

• ❌ No test accounts in production
• ❌ No demo data in production
• ❌ No seed scripts in production
• ✅ Production only receives real user data
• ✅ Test everything in staging first

Best Practices for Database Migrations

DO These Things

1. Always Create Migrations Locally First

   • Never create migrations directly in cloud environments
   • Test locally with Docker before committing
   • Verify migration SQL is correct

2. Use Descriptive Migration Names

   ✅ npm run db:migrate:create add_user_preferences_table
   ✅ npm run db:migrate:create update_order_status_enum
   ✅ npm run db:migrate:create add_supplier_email_index
   ❌ npm run db:migrate:create changes
   ❌ npm run db:migrate:create fix

3. Test in Staging for 24-48 Hours

   • Don't rush to production
   • Catch performance issues with production-like data volumes
   • Verify migrations work under load

4. Commit Migration Files to Git

   • All files in prisma/migrations/ directory
   • Updated prisma/schema.prisma
   • Migration lock file (ensures same database provider)
   • Never edit migration files after committing

5. Verify Before Deploying

   # Check migration status before production deployment
   npm run db:verify
   npm run db:migrate:status

6. Backup Before Risky Operations

   # Local database backup
   npm run db:backup
   
   # Cloud databases have automatic backups
   # Check your cloud provider dashboard

7. Use Migration Lock Files

   • Commit prisma/migrations/migration_lock.toml
   • Prevents accidental database provider mismatches
   • Ensures PostgreSQL in all environments

DON'T Do These Things

1. Never Use migrate dev in Cloud Environments

   • It creates AND applies migrations (dangerous in production)
   • Use migrate deploy for cloud environments

2. Never Edit Migration Files After Committing

   • Migration history must be immutable
   • Create new migration to fix issues

3. Never Skip Staging

   • Direct to production is dangerous
   • Staging catches 90% of migration issues

4. Never Auto-Run Migrations on Deployment

   • Manual migrations provide safety and control
   • Bad migrations shouldn't block code deployment

5. Never Commit .env Files with Credentials

   • Database connection strings stay in deployment platform
   • Use .env.example for templates only

6. Never Drop Columns/Tables Immediately

   • Use 3-phase approach (see above)
   • Prevents downtime and data loss

7. Never Fallback to Default Data

   • If shopId/userId/tenantId is missing, fail loudly
   • Don't silently use sample data
   • Prevents data leaks in multi-tenant systems

Integration with Branch Strategy

Database migrations follow the same branch-based deployment model:

Feature Branch (Local)
  ├─ Create migration: db:migrate:create
  ├─ Apply locally: db:migrate:dev
  ├─ Test changes
  └─ Commit migration files
           ↓
Staging Branch (Cloud)
  ├─ Code auto-deploys (Vercel)
  ├─ Manually run: db:migrate:staging
  ├─ Test for 24-48 hours
  └─ Verify with test data
           ↓
Production Branch (Cloud)
  ├─ Code auto-deploys (Vercel)
  ├─ Manually run: db:migrate:production
  ├─ Monitor closely
  └─ Maintain rollback capability

Key Timing:

1. Code deployment: Automatic (branch strategy handles this)
2. Migration deployment: Manual (you run the command)
3. Gap between deployment and migration: Usually minutes, but you control timing

When to Run Migrations:

• ✅ After Vercel deployment completes
• ✅ During low-traffic periods (if possible)
• ✅ When you can monitor immediately after
• ❌ Not automatically as part of build process
• ❌ Not during high-traffic periods (if avoidable)

CI/CD Integration

Add migration verification to your CI pipeline:

# .github/workflows/ci.yml
- name: Setup test database
  run: |
    docker run -d --name test-postgres \
      -e POSTGRES_PASSWORD=test \
      -e POSTGRES_DB=app_test \
      -p 5433:5432 postgres:16-alpine

- name: Verify migrations
  run: npm run db:verify
  env:
    DATABASE_URL: postgresql://postgres:test@localhost:5433/app_test

- name: Run migrations
  run: npx prisma migrate deploy
  env:
    DATABASE_URL: postgresql://postgres:test@localhost:5433/app_test

Benefits:

• Catches migration issues before they reach staging
• Verifies migrations apply cleanly to fresh database
• Tests that schema matches Prisma client generation
• Fails PR if migrations are broken

Quick Reference Checklist

Setting Up New Zeron App:

• [ ] Create three separate database instances (local, staging, production)
• [ ] Configure connection strings in deployment platform (never commit)
• [ ] Set up Docker Compose with PostgreSQL for local development
• [ ] Add migration scripts to package.json
• [ ] Install dotenv-cli for environment-specific commands
• [ ] Create migration lock file (commit to Git)
• [ ] Add migration verification to CI/CD pipeline
• [ ] Document environment setup in project README

For Each Migration:

• [ ] Create migration locally: npm run db:migrate:create
• [ ] Apply to local database: npm run db:migrate:dev
• [ ] Test changes thoroughly locally
• [ ] Commit migration files with code changes
• [ ] Push to staging branch (code auto-deploys)
• [ ] Apply to staging database: npm run db:migrate:staging
• [ ] Test in staging for 24-48 hours
• [ ] Merge to production branch (code auto-deploys)
• [ ] Apply to production database: npm run db:migrate:production
• [ ] Monitor production closely after migration

Red Flags (Stop Immediately):

• ❌ Migration modifying production data unexpectedly
• ❌ Migration taking longer than expected (>30 seconds)
• ❌ Migration errors in logs
• ❌ Application errors immediately after migration
• ❌ Performance degradation after migration

If Red Flags Appear:

1. Do NOT continue with deployment
2. Rollback code deployment via platform dashboard
3. Create reversal migration
4. Test reversal in staging
5. Investigate root cause before retrying

Benefits

• ✅ Controlled releases: No accidental production updates
• ✅ Safe testing: Validate changes before customer impact
• ✅ Rollback capability: Easy revert when issues arise
• ✅ Parallel development: Multiple features tested simultaneously
• ✅ Security: Production environment fully protected
• ✅ Confidence: Thorough validation before release

Best Practices

Branch Protection

• Require reviews before merging
• Enforce passing tests and checks
• Restrict direct pushes to critical branches
• Maintain clean commit history

Deployment Safety

• Always deploy to staging first
• Monitor staging for issues
• Use feature flags for gradual rollouts
• Document rollback procedures

Rollback Strategy

Emergency Rollback

Quick revert to last known good state when critical issues arise:

• Identify last stable deployment
• Revert production to that commit
• Verify system health
• Investigate root cause

Planned Rollback

Controlled rollback for identified issues:

• Revert specific changes
• Test in staging first
• Deploy rollback during maintenance window
• Communicate with stakeholders

Key Principles

1. Never skip staging - All changes go through staging first
2. Automate validation - Use automated checks wherever possible
3. Monitor actively - Watch metrics after every deployment
4. Document everything - Maintain clear deployment history
5. Test rollbacks - Ensure rollback procedures actually work