radicle/radicle-ci-broker
1
0
Fork 0
mirror of https://seed.radicle.dev/rad:zwTxygwuz5LDGBq255RA2CbNGrz8.git synced 2026-10-18 19:30:29 +08:00
Code Issues Packages Projects Releases Wiki Activity

tests: add scenario to verify behavior when adapter fails

Browse Source 
Verify behavior for both a successful and a failed CI run, but with an
adapter that returns non-zero.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
This commit is contained in:
Lars Wirzenius
2025-08-05 08:01:31 +03:00
parent a3d8d1aa20
commit 5439018503
1 changed files with 155 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ci-broker.md
+155 -1
View File
@@ -141,6 +141,73 @@ fi
) 1>&2
~~~
## A failing adapter with a successful run
This adapter does nothing, just reports a run ID and a successful run,
but then fails.
~~~{#failing-on-success.sh .file .sh}
#!/bin/sh
set -eu
cat > /dev/null
echo '{"response":"triggered","run_id":{"id":"xyzzy"}}'
echo '{"response":"finished","result":"success"}'
(
echo "This is an adapter error: Mordor"
echo "Environment:"
env
if [ "${RADICLE_NATIVE_CI:-}" != "" ]; then
echo "Adapter config:"
nl "$RADICLE_NATIVE_CI"
fi
) 1>&2
exit 1
~~~
## A failing adapter with a failed run
This adapter does nothing, just reports a run ID and a failed run,
but then fails.
~~~{#failing-on-failure.sh .file .sh}
#!/bin/sh
set -eu
cat > /dev/null
echo '{"response":"triggered","run_id":{"id":"xyzzy"}}'
echo '{"response":"finished","result":"failure"}'
(
echo "This is an adapter error: Mordor"
echo "Environment:"
env
if [ "${RADICLE_NATIVE_CI:-}" != "" ]; then
echo "Adapter config:"
nl "$RADICLE_NATIVE_CI"
fi
) 1>&2
exit 1
~~~
## List job COBs
Job COBs are a way for the CI broker to record that it's run CI for a
change. This script lists the job COBs in a given repository.
~~~{#list-jobs.sh .file .sh}
#!/bin/bash
set -euo pipefail
RID="$(rad ls --all | awk -v R="$1" '$2 == R { print $3 }')"
if [ -z "$RID" ]; then
echo "Unknown repository $1" 1>&2
exit 1
fi
rad cob list --repo "$RID" --type xyz.radworks.job
~~~
# Custom scenario steps
In this document we use scenarios to show how to verify that the CI
@@ -312,7 +379,6 @@ This is a very rudimentary check, but if the event file is incorrect,
then Radicle code will reject it. We don't want to duplicate the logic
to do that verification in detail.
# Acceptance criteria
## Shows config as JSON
@@ -408,6 +474,11 @@ given a Git repository xyzzy in the Radicle node
given the Radicle node emits a refsUpdated event for xyzzy
when I run ./env.sh synthetic-events synt.sock event.json --log log.txt
given a directory reports
given file list-jobs.sh
when I run bash list-jobs.sh xyzzy
then stdout is exactly ""
when I run ./env.sh cib --config broker-with-triggers.yaml process-events
then stderr contains "CibStart"
@@ -417,6 +488,89 @@ then file reports/index.html exists
then file reports/status.json exists
then file reports/index.rss exists
when I run bash list-jobs.sh xyzzy
then stdout isn't exactly ""
when I run cibtool --db ci-broker.db event list
then stdout is empty
when I run cibtool --db ci-broker.db run list --json
then stdout contains ""id": "xyzzy""
~~~
## Handles adapter failing on a successful run
_Want:_ If the adapter fails, the CI broker creates a job COB and
report pages anyway.
_Why:_ This is necessary for the CI broker to be robust.
_Who:_ `cib-devs`
~~~scenario
given a Radicle node, with CI configured with broker-with-triggers.yaml and adapter failing-on-success.sh
given a Git repository xyzzy in the Radicle node
given the Radicle node emits a refsUpdated event for xyzzy
when I run ./env.sh synthetic-events synt.sock event.json --log log.txt
given a directory reports
given file list-jobs.sh
when I run bash list-jobs.sh xyzzy
then stdout is exactly ""
when I run ./env.sh cib --config broker-with-triggers.yaml process-events
then stderr contains "CibStart"
then stderr contains "CibConfig"
then stderr contains "CibEndSuccess"
then file reports/index.html exists
then file reports/status.json exists
then file reports/index.rss exists
when I run bash list-jobs.sh xyzzy
then stdout isn't exactly ""
when I run cibtool --db ci-broker.db event list
then stdout is empty
when I run cibtool --db ci-broker.db run list --json
then stdout contains ""id": "xyzzy""
~~~
## Handles adapter failing on a failed run
_Want:_ If the adapter fails, the CI broker creates a job COB and
report pages anyway.
_Why:_ This is necessary for the CI broker to be robust.
_Who:_ `cib-devs`
~~~scenario
given a Radicle node, with CI configured with broker-with-triggers.yaml and adapter failing-on-failure.sh
given a Git repository xyzzy in the Radicle node
given the Radicle node emits a refsUpdated event for xyzzy
when I run ./env.sh synthetic-events synt.sock event.json --log log.txt
given a directory reports
given file list-jobs.sh
when I run bash list-jobs.sh xyzzy
then stdout is exactly ""
when I run ./env.sh cib --config broker-with-triggers.yaml process-events
then stderr contains "CibStart"
then stderr contains "CibConfig"
then stderr contains "CibEndSuccess"
then file reports/index.html exists
then file reports/status.json exists
then file reports/index.rss exists
when I run bash list-jobs.sh xyzzy
then stdout isn't exactly ""
when I run cibtool --db ci-broker.db event list
then stdout is empty