CI/CD

Production-Ready Testing Pipelines

This project demonstrates sophisticated CI/CD patterns for Svelte testing with vitest-browser-svelte and Playwright, supporting the Client-Server Alignment Strategy in automated environments.

Overview

The CI/CD setup uses:

Playwright containers for consistent browser environments
Separate workflows for unit tests and E2E tests
Automatic version synchronization between dependencies and containers
Optimized caching for fast pipeline execution
Coverage reporting for unit tests

Workflow Architecture

Unit Tests Workflow (`unit-tests.yaml`)

1name: CI/CD
2on:
3  pull_request:
4    branches: [main]
5  push:
6    branches: [main]
7
8jobs:
9  test:
10    name: Unit Tests & Coverage
11    runs-on: ubuntu-24.04
12    container:
13      image: mcr.microsoft.com/playwright:v1.52.0-noble
14      options: --user 1001
15    timeout-minutes: 10

Key Features:

Runs vitest-browser-svelte tests in Playwright container
Generates coverage reports
Fast execution with 10-minute timeout
Proper user permissions with --user 1001

E2E Tests Workflow (`e2e.yaml`)

1name: E2E Tests
2on:
3  pull_request:
4    branches: [main]
5  push:
6    branches: [main]
7
8jobs:
9  e2e:
10    name: End-to-End Tests
11    runs-on: ubuntu-24.04
12    container:
13      image: mcr.microsoft.com/playwright:v1.52.0-noble
14      options: --user 1001
15    timeout-minutes: 15

Key Features:

Validates complete Client-Server integration
Longer timeout for full user journeys
Same container for consistency with unit tests

Playwright Container Strategy

Why Containers?

Using Playwright containers ensures:

Consistent browser environments across local dev and CI
Pre-installed browser dependencies (no installation time)
Reproducible test results regardless of runner environment
Faster pipeline execution with cached browser binaries

Version Synchronization

Both workflows compare the installed Playwright versions with the container tag. Checking only the version range in package.json misses lockfile updates and catalog references:

1playwright_version=$(node -p "require('./apps/website/node_modules/playwright/package.json').version")
2test_version=$(node -p "require('./apps/website/node_modules/@playwright/test/package.json').version")
3container_version=$(grep -o 'playwright:v[0-9.]*' .github/workflows/unit-tests.yaml | sed 's/playwright:v//')
4
5if [ "$playwright_version" != "$container_version" ] || [ "$test_version" != "$container_version" ]; then
6  echo "Update the installed Playwright packages and CI images together."
7  exit 1
8fi

Pin Playwright’s package versions and group their Renovate updates with the container images. A lockfile refresh should not silently select a browser revision absent from the CI image.

Caching Strategy

pnpm Store Caching

1- name: Get pnpm store directory
2  shell: bash
3  run: |
4    echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
5
6- name: Setup pnpm cache
7  uses: actions/cache@v4
8  with:
9    path: ${{ env.STORE_PATH }}
10    key:
11      ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml')
12      }}
13    restore-keys: |
14      ${{ runner.os }}-pnpm-store-

Benefits:

Faster dependency installation
Reduced bandwidth usage
Consistent package versions

Concurrency Controls

1concurrency:
2  group: ${{ github.workflow }}-${{ github.ref }}
3  cancel-in-progress: true

Purpose:

Cancels redundant runs on new pushes
Saves CI resources
Faster feedback on latest changes

Environment Configuration

Environment Variables and Secrets

1env:
2  API_SECRET: ${{ secrets.API_SECRET }}

Note: The API_SECRET is specific to this project’s requirements. Your project may not need secrets, or may need different ones.

When you MUST configure secrets in CI:

Build-time environment variables - Build will fail without them
Server-side API testing with authentication
E2E tests that require login flows
External service integrations (databases, APIs)

⚠️ Critical: If your project requires environment variables to build (like API_SECRET in this project), you MUST configure them in GitHub Secrets or your CI build will fail.

When you DON’T need secrets:

Simple component testing only
Static sites without build-time environment dependencies
Public demo applications with no external services
Projects that use only public APIs or mock data

This project uses API_SECRET for:

Server-side API testing
Authentication flows in E2E tests
Production-like environment simulation

Build Process

1- name: Build application
2  run: pnpm build
3  env:
4    API_SECRET: ${{ secrets.API_SECRET }}

Builds are tested in CI to catch:

TypeScript compilation errors
Build-time configuration issues
Missing environment variables

Test Execution Patterns

Unit Tests with Coverage

1- name: Coverage
2  run: pnpm test:unit --run --coverage

Features:

Runs all vitest-browser-svelte tests
Generates coverage reports
Fails on coverage thresholds (if configured)

E2E Test Execution

1- name: Run E2E tests
2  run: pnpm test:e2e
3  env:
4    API_SECRET: ${{ secrets.API_SECRET }}

Features:

Full application testing
Real browser interactions
Complete Client-Server validation

Best Practices

Container User Permissions

1container:
2  image: mcr.microsoft.com/playwright:v1.52.0-noble
3  options: --user 1001

Why --user 1001?

Matches GitHub Actions runner user
Prevents permission issues with file creation
Ensures consistent behavior

Timeout Management

Unit Tests: 10 minutes (fast feedback)
E2E Tests: 15 minutes (allows for full user journeys)

Workflow Separation

Benefits of separate workflows:

Independent failure isolation
Different timeout requirements
Parallel execution capability
Clear responsibility separation

Local Development Alignment

Matching CI Environment Locally

1# Run tests in same Playwright container locally
2docker run --rm -it \
3  -v $(pwd):/workspace \
4  -w /workspace \
5  mcr.microsoft.com/playwright:v1.52.0-noble \
6  /bin/bash
7
8# Inside container
9npm install -g pnpm
10pnpm install
11pnpm test:unit --run

Version Consistency

Keep these synchronized:

package.json Playwright version
Container image version in workflows
Local Playwright installation

Troubleshooting CI Issues

Common Problems

Playwright Version Mismatch:

1❌ ERROR: Playwright versions don't match!

Solution: Update either package.json or workflow container version

Permission Errors:

1EACCES: permission denied

Solution: Ensure --user 1001 is set in container options

Test Timeouts:

1Test timeout of 10000ms exceeded

Solution: Increase workflow timeout or optimize slow tests

Missing Environment Variables:

1❌ Error: Environment variable API_SECRET is not defined
2❌ Build failed: Missing required environment variables

Solution: Configure required secrets in GitHub repository settings:

Go to repository Settings → Secrets and variables → Actions
Add API_SECRET (or your required variables) to Repository secrets
Ensure workflow files reference the correct secret names

Debugging Failed Tests

Check workflow logs for specific error messages
Verify environment variables are properly set
Test locally with same container image
Check for flaky tests with multiple runs

Advanced Patterns

Matrix Testing (Future Enhancement)

1strategy:
2  matrix:
3    browser: [chromium, firefox, webkit]
4    node-version: [20, 22]

Artifact Collection

1- name: Upload test results
2  uses: actions/upload-artifact@v4
3  if: failure()
4  with:
5    name: test-results
6    path: test-results/

Parallel Test Execution

1strategy:
2  matrix:
3    shard: [1, 2, 3, 4]
4steps:
5  - run: pnpm test:unit --shard=${{ matrix.shard }}/4

Security Considerations

Secret Management

Use GitHub Secrets for sensitive data
Never log secret values
Rotate secrets regularly

Container Security

Use official Microsoft Playwright images
Pin specific versions (avoid latest)
Regular security updates

This CI/CD setup ensures your Client-Server Alignment Strategy works reliably in production environments, catching integration issues before they reach users.