Commit Graph
193 Commits
Author SHA1 Message Date
Lars Wirzenius 0e5c1d7dc9 refactor(ci-broker.md): rename trigger.json to be more correct
It's a "refsFetched" node event, not a trigger message. The name was
the last remaining remnant of an earlier approach to verifying the CI
broker.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:35:06 +03:00
Lars Wirzenius 0d86c048b9 refactor: drop now-useless sed manipulation of node config
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:31:26 +03:00
Lars Wirzenius 5478af8f41 fix(ci-broker.md): replace only the DUMMYNID part
Change the "smoke test" scenario so it passes.

Fix set-rid (embedded in ci-broker.md). Previously the input ref name
was just prefixed with "refs/namespaces/{nid}", but this resulted in
an input value that referenced a namespace to become a ref to a
namespace in a namespace, which isn't sensible for what we're doing.
Change this to replace only the namespace NID in the input with the
NID we're testing with. For simplicity, assume the input NID is
"DUMMYNID".

If I was sensible, I'd write a helper library to make this better.

Signed-off-by: Lars Wirzenius <liw@liw.fi>

ci-broker.md: fix smoke test scenario

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:26:23 +03:00
Lars Wirzenius 93e17fe9ec fix(cibtool): make verbose listing events be explicit when not found
Previously, we output an Option, but now we output what's contained in
the option, or an explicit "no such event". This helps output to be
easier to understand for users, and simplifies testing, too.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:25:39 +03:00
Lars Wirzenius 8850f18a75 ci-broker.md: verify that event is processed and run is inserted
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:22:38 +03:00
Lars Wirzenius 70538ba31b ci-broker.md: verify there is a run (broken to show output)
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:20:34 +03:00
Lars Wirzenius 441030205a cibtool: show contents of run, not just id
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:20:17 +03:00
Lars Wirzenius 99f0124aa1 ydb.rs: fix list and get runs
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:19:25 +03:00
Lars Wirzenius cb05ca9e65 partial debug, abandonded
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:34:48 +03:00
Lars Wirzenius f4e9ba5470 nothing is inserted into list of runs in db
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:32:07 +03:00
Lars Wirzenius 7a1b7e0a9f Revert "ci-broker.md: don't set rid in trigger event for smoke test"
This reverts commit 63dd3f92ab.
2024-07-03 18:08:57 +03:00
Lars Wirzenius 384041506b actually run CI for queued events
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:08:40 +03:00
Lars Wirzenius 63dd3f92ab ci-broker.md: don't set rid in trigger event for smoke test
it breaks for mysterious reasons

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:01:33 +03:00
Lars Wirzenius d4e4091509 add dep on env_logger
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:01:13 +03:00
Lars Wirzenius c62a4bee95 event.rs: add trace logging to parse_ref, and enable logging in its tests
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:00:47 +03:00
Lars Wirzenius b24692c6d1 drop status page argument to Broker::execute_ci
We'll insert them into the db soon enugh.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 17:25:23 +03:00
Lars Wirzenius b3638a2342 drop Adapter::run status argument harder
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 17:21:30 +03:00
Lars Wirzenius 67b66b1f91 drop status page argument for Adapter::run
We'll not update StatusPage in memory with runs any more soon.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 17:15:28 +03:00
Lars Wirzenius e4573ec3fb feat: add a thread for adding events to queue
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 14:01:13 +03:00
Lars Wirzenius 0db9dfe34c test: update the subplot scenarios to verify the changes
Add a couple of helper scripts to make scenarios a little less
repetitive. These aren't great for understanding, and we may need to
add more understandable steps later.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:56:05 +03:00
Lars Wirzenius 003e901279 feat: rewrite ci-broker.rs as cib.rs
Drop the old ci-broker binary. New binary is cib.

New tool has a command line interface that is more extensible and
nicer to use.
2024-07-02 19:56:05 +03:00
Lars Wirzenius 86884cbe43 feat: change cibtool to list runs
Drop old list_runs binary.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:56:05 +03:00
Lars Wirzenius 21ef36aa23 feat: add new CI broker management tool
This will be useful for the node operator to manage CI broker event
queue, and for other purposes.
2024-07-02 19:56:05 +03:00
Lars Wirzenius 1fbb4ccbd5 chore: add new modules to src/lib.rs 2024-07-02 19:56:05 +03:00
Lars Wirzenius 56c212a8ca feat: module to process events in event queue 2024-07-02 19:56:05 +03:00
Lars Wirzenius 77d08d3fcc feat: module to insert events into event queue
Make BrokerEvent de-serializable so that we can store them as JSON in
the database in the persistent event queue.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:55:42 +03:00
Lars Wirzenius 6e5c0d0290 fix: adapt src/broker.rs to new database abstraction 2024-07-02 19:55:35 +03:00
Lars Wirzenius 1386c94189 feat: rewrite database abstraction
Add a "counter" table, to enable a rudimentary concurrency test. Used
this to verify that multiple processes can safely update the same
database. Multi-process concurrency can result in temporary, short
lived lock conflicts in transactions: deal with this by re-trying for
a short duration.

Add a persistent event queue. Add methods to the Db type to deal with
counters and event queue.

Add a new error type to allow Db methods to return errors that do not
come from SQLite.

Restructure the internals a bit to make them easier for me to follow.
2024-07-02 19:55:35 +03:00
Lars Wirzenius 969f4fb2fb feat: serialize a configuration as JSON
This will allow the CI broker to show the full actual configuration to
the user.
2024-07-02 19:55:35 +03:00
Lars Wirzenius fc5a466426 feat: teach synthetic-event to repeat events
This allows for simplistic performance testing.
2024-07-02 19:55:35 +03:00
Lars Wirzenius 61cbf9e236 feat: log the name of the adapter binary
This turned out to be helpful when debugging a problem.
2024-07-02 19:55:35 +03:00
Lars Wirzenius 74862206fb fix: parsing of ref strings for NID; also add tests
I don't understand how the old could ever have worked. Maybe I don't
understand what's going on. The tests should help, in the future.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:55:06 +03:00
Lars Wirzenius 0266df7e21 feat! remove BrokerEvent::is_allowed from its public API
Nobody in the CI broker was using it, so no other changes.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:52:55 +03:00
Lars Wirzenius 1010100c29 refactor: remove unnecessary variable
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius c69c4c44ca refactor: rename variables for clarity
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius b1820f3c2a refactor: simplify creating Result::Trigger
I find this easier to follow than keeping track of so many variables.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius 3a9f8ff3e2 refactor: simplify handling of parse_ref result
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius e842e01ac1 refactor: make variable have an initial value
Clearer for me.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius b4484b946d refactor: move PatchEvent creation to its own function
For clarity.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius f38697e2d9 refactor: move creation of PushEvent to its own function
This is clearer for me.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius d84eccdcb6 refactor: drop unnecessary PathAction conversion from string
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius 5fa43fc325 refactor: simplify how common fields Repository is created
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius a216b84bd1 refactor: drop debug logging, to make code easier to grok
I want to debug the trigger message creation, and the `build_trigger`
method is long, and hard to follow. So I'll refactor it to be easier
for me to understand. First step: remove the `debug!` calls to make
the code shorter.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:35:14 +03:00
Lars Wirzenius 96499aa1ba chore: bump version number
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-06 18:08:28 +03:00
Lars Wirzenius 4f1036970c docs: add NEWS.md
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-06 18:07:16 +03:00
Lars Wirzenius c82fc3f657 fix: use the latest commit in a patch or push change
The commits in the trigger event are newest-first. When I wrote the
code I thought they were oldest-first. Mea culpa.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-06 14:40:02 +03:00
Lars Wirzenius 645c5436cb feat: improve logging of event filtering
This makes it a little easier for me to read what's happening in the
filtering.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-06 14:39:45 +03:00
Lars Wirzenius e16308414b chore: update dependency on radicle to 0.11.0
Small fixes to adapt to API changes in the radicle crate.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-06 14:39:45 +03:00
Michalis Zampetakis e3a90dfc19 Provide a parser function for parsing reference.
Currently supported handlers for puch and patch events.
2024-05-24 22:26:39 +03:00
Michalis Zampetakis 8adca82fe3 Change the way that the broker perceives a patch
Instead of checking for changes in patch cob, check if push is at
`/refs/heads/patches/`.

Adds AnyPushRef filter.
2024-05-23 21:15:43 +03:00