The test suite didn't work on Macs. A fundamental problem was using "sleep 0.1" to wait for a background process to do something: this is never a good idea. Either the wait time is too short, resulting in spurious test failures, or it's so long it makes test take unnecessarily long. I should have known better. A better approach is to loop for a sufficiently long time and check if the background process has done its thing every so often, and stop looping when it has. There were some other, more incidental incompatibility problems too. To fix all this, replace use of "nc", a common tool with many incompatible variants, by adding a "client mode" to synthetic-events.rs that connects to the daemon and closes the connection neatly, then waits for the daemon to delete the socket. Also, change how it does what it does so that it works both Linux and Mac. This includes removing setting a read timeout that failed on Mac, but doesn't seem to be necessary now. Also, change synthetic-events to log more and more detailed messages to aid remote debugging of test suite failures in the future. The daemon logs errors from fallible operations, instead of discarding them and terminating, and the client outputs the daemon log if the client fails, while waiting for the daemon to remove the socket. Finally, during the furious debugging, refactor the code into smaller functions so its easier to understand when nothing makes sense. Signed-off-by: Lars Wirzenius <liw@liw.fi>
radicle-ci-broker
Add integration to CI engines/systems/services to Radicle, a distributed git hosting and collaboration system.
This is not quite production ready code yet, but it will eventually become a thing that listens for changes in a Radicle node, and triggers CI on the relevant ones.
Architecture
See the doc directory for architecture documentation. Quick summary:
the CI broker gets events from the Radicle node, filters them based on
its own config, and for any event that gets past the filter, runs the
configured adapter executable. The broker and adapter use a simple
JSON based message protocol over stdin/stdout. Each CI system has its
own adapter.
To build the documentation, run make in the doc directory. You'll
need some tools installed: Pandoc, graphviz (dot), PlantUML,
pikchr-cli. The others are widely packaged, pikchr-cli is a Rust
crate, so you can install it with cargo install pikchr-cli
Build and publish the documentation like this:
RADICLE_CI_BROKER_WEBROOT=/tmp/ci make -C doc publish
Binaries
The crate contains several binaries:
cib--- the actual CI broker- this is the only one you need to care about, the rest are for debugging
cibtool--- management tool for node operators, for managing the CI brokersynthetic-events--- test tool for CI broker developers
Packaging
There is simple, simplistic, rudimentary, personal-use-only packaging
for Debian in the debian directory, used by Lars to build packages
for his own use.
Running from the source tree
To run cib or cibtool from the source tree:
cargo run --bin cib -- --config config.yaml process-events
cargo run --bin cibtool -- --db ci-broker.db event list
Note the -- argument. It tells cargo run that all the arguments
that follow are to be passed to the program being run, and not used by
cargo itself.
Running tests
To run the test suite for the CI broker:
cargo test
(The usual way, for a Rust program.)
Configuration
The configuration file is named on the command line. It is a YAML file, for example:
default_adapter: native
db: ci-broker.sqlite
adapters:
native:
command: radicle-native-ci
env:
RADICLE_NATIVE_CI: /home/liw/radicle/radicle-native-ci/x/config.yaml
sensitive_env:
some_secret: some_secret_value_that_is_not_logged
filters:
- !And
- !Repository "rad:z2e6URdt1we1iG1BCVqtx8QVgsX4a"
- !Or
- !Branch "main"
- !AnyPatchRef
Or if you only want to filter for patch COB updates (e.g. comments), to the specified repository:
default_adapter: native
db: ci-broker.sqlite
adapters:
native:
command: radicle-native-ci
env:
RADICLE_NATIVE_CI: /home/liw/radicle/radicle-native-ci/x/config.yaml
sensitive_env:
some_secret: some_secret_value_that_is_not_logged
filters:
- !And
- !Repository "rad:z2e6URdt1we1iG1BCVqtx8QVgsX4a"
- !And
- !AnyPatch
- !Not
- !AnyPatchRef
This runs the native CI engine as an adapter, on any repository events
that pass the filter. The filter allows any changes to the main
branch or any Radicle patch, on the specified repository.
License
Radicle CI broker is distributed under the terms of both the MIT license and the Apache License (Version 2.0).
See LICENSE-APACHE and LICENSE-MIT for details.