> ## Documentation Index > Fetch the complete documentation index at: https://www.activepieces.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Canary Deployment Canary deployment lets us run a separate app instance on a newer (or experimental) build and route a specific subset of platforms to it, while all other traffic continues hitting the primary app. This gives us real-world validation with live data before a full rollout. ## Architecture Overview ``` Browser / External Webhook │ ▼ Primary App (prod) ┌─────────────────────────────┐ │ preHandler hook │ │ canaryRoutingMiddleware │ │ ├─ platformId ∈ canary? │──── No ──▶ handle normally │ └─ Yes │ │ ▼ │ │ HTTP proxy to canary app │──────────▶ Canary App └─────────────────────────────┘ │ ▼ Canary Worker (polls canaryWorkerJobs queue) ``` All traffic enters the primary app. The `canaryRoutingMiddleware` runs as a Fastify `preHandler` hook on every request. If the resolved `platformId` is in the canary list, the request is proxied in-process to the canary app and the response is returned directly to the caller — the primary app never processes it further. ## Canary membership: DB-backed with Redis cache Canary platform membership is stored in the `platform_plan.canary` boolean column. On each lookup, the list of canary platform IDs is fetched from Redis (`canary-platform-ids` key). On cache miss the list is read from the database and cached. When the canary flag is changed via the API, the cache is invalidated immediately. This removes the need to keep `AP_CANARY_PLATFORM_IDS` in sync across services and allows runtime changes without a redeploy. ## Components ### Canary App A second instance of the server API running a different image tag. No special env vars are needed for canary membership — it is driven entirely by the DB. Configure it with: | Env var | Value | | - | - | | `AP_FRONTEND_URL` | canary app's public URL | The canary app runs as a full app instance. Global queue consumers (`runsMetadata`, `system-job-queue`) run on both the primary and canary app — this is safe because BullMQ ensures each job is processed by exactly one consumer. ### Canary Workers Dedicated workers that connect to the canary app instead of the primary app: | Env var | Value | | - | - | | `AP_WORKER_GROUP_ID` | `canary` | Workers use `AP_FRONTEND_URL` for their Socket.IO RPC channel and for posting engine results back to the app. Canary workers are registered with the canary app, so the RPC path is fully isolated. ### Primary App Config The primary app only needs to know where to proxy canary requests: | Env var | Description | | - | - | | `AP_CANARY_APP_URL` | Internal URL of the canary app | ## WebSocket / Real-time Updates Socket.IO is configured with a **Redis adapter** (`@socket.io/redis-adapter`). Events emitted on any app instance (primary or canary) are broadcast through Redis pub/sub to all connected instances. This means: * Users connected to the primary app receive real-time flow run updates even when the execution happened on the canary app. * No WebSocket proxying is required. ## Queue Isolation | Queue | Primary App | Canary App | | - | - | - | | `workerJobs` | ✅ Consumed by primary workers | Not consumed | | `runsMetadata` | ✅ Consumed | ✅ Consumed | | `system-job-queue` | ✅ Consumed | ✅ Consumed | | `canaryWorkerJobs` | Not consumed | ✅ Consumed by canary workers | Jobs for canary platforms are enqueued to `canaryWorkerJobs`, which only canary workers poll — fully isolated from the primary worker fleet. ## Deploying a New Canary Build The canary has no workflow of its own. The `deploy` job of the Cloud gate (`continuous-delivery-cloud.yml`) deploys it, after the approval and a few minutes before the rest of production. There is no daily schedule and no manual dispatch: the canary shares the production database and runs migrations at boot, so a daily build of `main` ran newer migrations under the older fleet. The `deploy` job: 1. Checks if any migration that the canary does not have yet is breaking. It fails if one is, unless the Cloud run sets `skip_migration_check`. 2. Deploys the candidate image to the canary app (`config/app-canary.yml`) and canary workers. 3. Deploys the same image to the fleet. The job holds the `promote-production` lock from step 1 to step 3, so no other gate run or rollback can run between the canary and the fleet. Required GitHub secrets: `AP_API_KEY` (the primary app's `AP_API_KEY` value). ## Rolling Back a Canary Deployment The next Cloud deploy sets the canary image. The Cloud rollback workflow does not touch the canary hosts, so after a Cloud rollback the canary keeps the newer image until the next Cloud deploy. There is no separate canary rollback, because running migrations down on the shared database can drop data the newer code wrote. ## Promoting canary build to production Canary is not a promotion path. The Cloud gate (`continuous-delivery-cloud.yml`) deploys each approved candidate to the canary and then to the fleet in the same run. ## Managing Platform Routing ### Enable canary routing for a platform ```bash theme={null} curl -X POST https://cloud.activepieces.com/api/v1/admin/platforms/canary \ -H "api-key: " \ -H "Content-Type: application/json" \ -d '{"platformId": "", "canary": true}' ``` ### Disable canary routing for a platform ```bash theme={null} curl -X POST https://cloud.activepieces.com/api/v1/admin/platforms/canary \ -H "api-key: " \ -H "Content-Type: application/json" \ -d '{"platformId": "", "canary": false}' ``` Both calls update `platform_plan.canary` in the database and invalidate the Redis cache (`canary-platform-ids`). The change takes effect on the next request — no restart required. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.