Configuring CI Using Bitbucket Pipelines and Nx
There are two general approaches to setting up CI with Nx - using a single job or distributing tasks across multiple jobs. For smaller repositories, a single job is faster and cheaper, but once a full CI run starts taking 10 to 15 minutes, using multiple jobs becomes the better option. Nx Cloud's distributed task execution allows you to keep the CI pipeline fast as you scale. As the repository grows, all you need to do is add more agents.
Process Only Affected Projects With One Job on Bitbucket Pipelines
Below is an example of an Bitbucket Pipelines setup that runs on a single job, building and testing only what is affected. This uses the nx affected command to run the tasks only for the projects that were affected by that PR.
1image: node:20
2pipelines:
3 pull-requests:
4 '**':
5 - step:
6 name: 'Build and test affected apps on Pull Requests'
7 caches: # optional
8 - node
9 script:
10 - npm ci
11 - npx nx format:check
12 - npx nx affected -t lint,test,build --base=origin/master --head=HEAD --configuration=ci
13
14 branches:
15 main:
16 - step:
17 name: "Build and test affected apps on 'main' branch changes"
18 caches: # optional
19 - node
20 script:
21 - npm ci
22 - npx nx format:check
23 - npx nx affected -t lint,test,build --base=HEAD~1 --configuration=ci
24The pull-requests and main jobs implement the CI workflow.
Distribute Tasks Across Agents on Bitbucket Pipelines
To set up Distributed Task Execution (DTE), you can run this generator:
❯
npx nx g ci-workflow --ci=bitbucket-pipelines
Or you can copy and paste the workflow below:
1image: node:20
2
3clone:
4 depth: full
5
6definitions:
7 steps:
8 - step:
9 name: Agent
10 script:
11 - export NX_BRANCH=$BITBUCKET_PR_ID
12
13 - npm ci
14 - npx nx-cloud start-agent
15
16pipelines:
17 pull-requests:
18 '**':
19 - parallel:
20 - step:
21 name: CI
22 script:
23 - export NX_BRANCH=$BITBUCKET_PR_ID
24
25 - npm ci
26 - npx nx-cloud start-ci-run --stop-agents-after="build" --agent-count=3
27 - npx nx-cloud record -- npx nx format:check
28 - npx nx affected --target=lint,test,build --parallel=2
29 - step:
30 - step:
31 - step:
32This configuration is setting up two types of jobs - a main job and three agent jobs.
The main job tells Nx Cloud to use DTE and then runs normal Nx commands as if this were a single pipeline set up. Once the commands are done, it notifies Nx Cloud to stop the agent jobs.
The agent jobs set up the repo and then wait for Nx Cloud to assign them tasks.
Two Types of ParallelizationThe agents and the --parallel flag both parallelize tasks, but in different ways. The way this workflow is written, there will be 3 agents running tasks and each agent will try to run 2 tasks at once. If a particular CI run only has 2 tasks, only one agent will be used.