1003 Commits
Author SHA1 Message Date
Lars Wirzenius 08706d1d4e docs(README.md): drop mention of list_runs binary
It's gone and replaced by "cibtool run list".

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-11 12:45:33 +03:00
Lars Wirzenius 8245df4722 fix(cibtool): use node alias in reports
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-11 12:45:22 +03:00
Lars Wirzenius bbf65cb29a refactor: add Db::get_all_runs
This is needed in a few places, and it's nominally more efficient to
get all runs with one SELECT, than first get all run ids and then each
run one by one.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-10 12:43:33 +03:00
Lars Wirzenius 968d0e091b feat(cibtool): add command to produce HTML report from database
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-10 12:43:14 +03:00
Lars Wirzenius 6cd968d480 feat(cibtool): allow user to add information about a run
This is primarily going to be useful for setting up runs in the
database for testing report generation, which will happen in the near
future.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-10 12:41:37 +03:00
Lars Wirzenius 5f1ce255c0 refactor: rename *Cmd to *SubCmd, Counter/EventRun to *Cmd
This will allow use to import "crate::run::Run" later, without any
hoops to jump though.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-10 12:37:18 +03:00
Lars Wirzenius d4cbcc81ed refactor: use BrokerError only in broker.rs
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 19:30:31 +03:00
Lars Wirzenius 9f5db36e44 refactor(filter-events.rs): drop use of BrokerError
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 18:15:09 +03:00
Lars Wirzenius e4b6897037 refactor: store run id from Run (i.e., from adapter) in db
Previously we invented a new run id that was meaningless.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 18:00:07 +03:00
Lars Wirzenius 50ea2eb1e5 fix(src/event.rs): allow branch names to have slashes
A branch name "liw/fix" if perfectly fine, so "parse_ref" should allow that.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:49:54 +03:00
Lars Wirzenius 0e5c1d7dc9 refactor(ci-broker.md): rename trigger.json to be more correct
It's a "refsFetched" node event, not a trigger message. The name was
the last remaining remnant of an earlier approach to verifying the CI
broker.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:35:06 +03:00
Lars Wirzenius 0d86c048b9 refactor: drop now-useless sed manipulation of node config
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:31:26 +03:00
Lars Wirzenius 5478af8f41 fix(ci-broker.md): replace only the DUMMYNID part
Change the "smoke test" scenario so it passes.

Fix set-rid (embedded in ci-broker.md). Previously the input ref name
was just prefixed with "refs/namespaces/{nid}", but this resulted in
an input value that referenced a namespace to become a ref to a
namespace in a namespace, which isn't sensible for what we're doing.
Change this to replace only the namespace NID in the input with the
NID we're testing with. For simplicity, assume the input NID is
"DUMMYNID".

If I was sensible, I'd write a helper library to make this better.

Signed-off-by: Lars Wirzenius <liw@liw.fi>

ci-broker.md: fix smoke test scenario

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:26:23 +03:00
Lars Wirzenius 93e17fe9ec fix(cibtool): make verbose listing events be explicit when not found
Previously, we output an Option, but now we output what's contained in
the option, or an explicit "no such event". This helps output to be
easier to understand for users, and simplifies testing, too.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-08 17:25:39 +03:00
Lars Wirzenius 8850f18a75 ci-broker.md: verify that event is processed and run is inserted
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:22:38 +03:00
Lars Wirzenius 70538ba31b ci-broker.md: verify there is a run (broken to show output)
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:20:34 +03:00
Lars Wirzenius 441030205a cibtool: show contents of run, not just id
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:20:17 +03:00
Lars Wirzenius 99f0124aa1 ydb.rs: fix list and get runs
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 19:19:25 +03:00
Lars Wirzenius cb05ca9e65 partial debug, abandonded
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:34:48 +03:00
Lars Wirzenius f4e9ba5470 nothing is inserted into list of runs in db
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:32:07 +03:00
Lars Wirzenius 7a1b7e0a9f Revert "ci-broker.md: don't set rid in trigger event for smoke test"
This reverts commit 63dd3f92ab.
2024-07-03 18:08:57 +03:00
Lars Wirzenius 384041506b actually run CI for queued events
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:08:40 +03:00
Lars Wirzenius 63dd3f92ab ci-broker.md: don't set rid in trigger event for smoke test
it breaks for mysterious reasons

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:01:33 +03:00
Lars Wirzenius d4e4091509 add dep on env_logger
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:01:13 +03:00
Lars Wirzenius c62a4bee95 event.rs: add trace logging to parse_ref, and enable logging in its tests
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 18:00:47 +03:00
Lars Wirzenius b24692c6d1 drop status page argument to Broker::execute_ci
We'll insert them into the db soon enugh.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 17:25:23 +03:00
Lars Wirzenius b3638a2342 drop Adapter::run status argument harder
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 17:21:30 +03:00
Lars Wirzenius 67b66b1f91 drop status page argument for Adapter::run
We'll not update StatusPage in memory with runs any more soon.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 17:15:28 +03:00
Lars Wirzenius e4573ec3fb feat: add a thread for adding events to queue
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-03 14:01:13 +03:00
Lars Wirzenius 0db9dfe34c test: update the subplot scenarios to verify the changes
Add a couple of helper scripts to make scenarios a little less
repetitive. These aren't great for understanding, and we may need to
add more understandable steps later.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:56:05 +03:00
Lars Wirzenius 003e901279 feat: rewrite ci-broker.rs as cib.rs
Drop the old ci-broker binary. New binary is cib.

New tool has a command line interface that is more extensible and
nicer to use.
2024-07-02 19:56:05 +03:00
Lars Wirzenius 86884cbe43 feat: change cibtool to list runs
Drop old list_runs binary.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:56:05 +03:00
Lars Wirzenius 21ef36aa23 feat: add new CI broker management tool
This will be useful for the node operator to manage CI broker event
queue, and for other purposes.
2024-07-02 19:56:05 +03:00
Lars Wirzenius 1fbb4ccbd5 chore: add new modules to src/lib.rs 2024-07-02 19:56:05 +03:00
Lars Wirzenius 56c212a8ca feat: module to process events in event queue 2024-07-02 19:56:05 +03:00
Lars Wirzenius 77d08d3fcc feat: module to insert events into event queue
Make BrokerEvent de-serializable so that we can store them as JSON in
the database in the persistent event queue.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:55:42 +03:00
Lars Wirzenius 6e5c0d0290 fix: adapt src/broker.rs to new database abstraction 2024-07-02 19:55:35 +03:00
Lars Wirzenius 1386c94189 feat: rewrite database abstraction
Add a "counter" table, to enable a rudimentary concurrency test. Used
this to verify that multiple processes can safely update the same
database. Multi-process concurrency can result in temporary, short
lived lock conflicts in transactions: deal with this by re-trying for
a short duration.

Add a persistent event queue. Add methods to the Db type to deal with
counters and event queue.

Add a new error type to allow Db methods to return errors that do not
come from SQLite.

Restructure the internals a bit to make them easier for me to follow.
2024-07-02 19:55:35 +03:00
Lars Wirzenius 969f4fb2fb feat: serialize a configuration as JSON
This will allow the CI broker to show the full actual configuration to
the user.
2024-07-02 19:55:35 +03:00
Lars Wirzenius fc5a466426 feat: teach synthetic-event to repeat events
This allows for simplistic performance testing.
2024-07-02 19:55:35 +03:00
Lars Wirzenius 61cbf9e236 feat: log the name of the adapter binary
This turned out to be helpful when debugging a problem.
2024-07-02 19:55:35 +03:00
Lars Wirzenius 74862206fb fix: parsing of ref strings for NID; also add tests
I don't understand how the old could ever have worked. Maybe I don't
understand what's going on. The tests should help, in the future.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:55:06 +03:00
Lars Wirzenius 0266df7e21 feat! remove BrokerEvent::is_allowed from its public API
Nobody in the CI broker was using it, so no other changes.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-07-02 19:52:55 +03:00
Lars Wirzenius 1010100c29 refactor: remove unnecessary variable
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius c69c4c44ca refactor: rename variables for clarity
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius b1820f3c2a refactor: simplify creating Result::Trigger
I find this easier to follow than keeping track of so many variables.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius 3a9f8ff3e2 refactor: simplify handling of parse_ref result
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius e842e01ac1 refactor: make variable have an initial value
Clearer for me.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius b4484b946d refactor: move PatchEvent creation to its own function
For clarity.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00
Lars Wirzenius f38697e2d9 refactor: move creation of PushEvent to its own function
This is clearer for me.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2024-06-18 20:36:22 +03:00