Commit Graph
471 Commits
Author SHA1 Message Date
Lars Wirzenius 4577682fd6 refactor(src/pull_queue.rs): drop "end" method, add Drop impl
This makes it harder to forget to close the producer end.

Suggested-by: Fintan
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 13:17:20 +03:00
Lars Wirzenius 19ede6dfed refactor(src/pull_queue.rs): avoid asserts
I didn't like having asserts, and didn't want debug_asserts. With
plain asserts, the program would have panicked. With debug_asserts, the
program would have silently overwritten the item in the queue.
Instead, change the method to return whether it was able to set, and
have the caller cope with not being able to set. The caller needs to
be able handle that (and was already doing so).

Found-by: Fintan
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 08:45:34 +03:00
Lars Wirzenius 582bd54b73 refactor: drop commented-out constants
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 08:20:05 +03:00
Lars Wirzenius e5723448d4 feat(src/queueproc.rs): log reason why waiting for adapter fails
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 08:16:01 +03:00
Lars Wirzenius e494961227 refactor: drop unnecessary pragma for allowing dead code
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 08:06:48 +03:00
Lars Wirzenius 62faeeec02 refactor: drop now-unnecessary pragma to allow unused imports
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 08:00:21 +03:00
Lars Wirzenius c493ec4928 fix(src/queueproc.rs): rewrite to avoid unimplemented!
This should never trigger, but just in case, we don't ever want it to
trigger in production.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 07:55:44 +03:00
Lars Wirzenius fd845f7e44 refactor: use more idiomatic Rust to check result of operation
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 07:52:14 +03:00
Lars Wirzenius 1f8978186a refactor: use a type to capture non-zero count of concurrent adapters
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-14 07:44:42 +03:00
Lars Wirzenius 06449445ec feat: enable adapter concurrency in cib
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius 55023f2e4f refactor: rewrite the timeoutcmd module for simplicity
Feed child process stdin via a temporary file. This is simpler than
opening a pipe and spawning a thread that feeds the pipe, when it's
not full.

Capture child stderr into a buffer without trying to let the
caller (another thread) process it line by line. We don't seem to have
use for that, and this simplifies things a fair bit.

Simplify the capture of child stdout to use a simpler thread and data
structure, compared to the previous multi-thread setup. Splitting
stdout into lines is now done by the thread that consumes the lines,
making the code that captures it simpler.

Also update the timeout subcommand to work with the changes. Other
callers of this code aren't yet updated.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius b73ab44962 refactor(src/broker.rs): add accessors, drop unnecessary mut
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius 0cf160d3d1 feat: implement concurrently running adapters
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius c16e9b0af7 feat: add log messages for thread start/end, log exit status in full
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius ccb82c8890 feat: implement cloning of a NotififcationSender
This will be useful later.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius c7dbff55e5 feat: add a "pull queue" data type
This will be useful for inter-thread synchronization in situations
where the producer thread should not produce anything until at least
one consumer is ready for it.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Lars Wirzenius 5ce95bc360 feat: add "concurrent_adapters" to configuration file
This doesn't yet affect anything, but it's a step towards implementing
concurrent adapters. The default is 1, which will be no change from
previous behavior, once I add support.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-13 09:02:12 +03:00
Tshepang Mbambo 0e746060fe process-events help: add missing space 2025-05-13 08:59:38 +03:00
Lars Wirzenius 22451b4b3a refactor: drop unused Broker::all_runs method
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-05-02 12:22:35 +03:00
Lars Wirzenius 8d7989cc6a Revert "fix: "cibtool run list" shows ids without loading run info"
This reverts commit 900a457157.
2025-04-17 17:04:46 +03:00
Lars Wirzenius 904f4bb249 refactor: make a proper new type for TagName
Make TagName into a new type (struct), instead of a type alias for
RefString. This provide stronger type safety, for the cost of a little
more implementation work. For the needs of the CI broker, that's two
methods and two trait impls.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-07 16:02:03 +03:00
Lars Wirzenius 0bda319b54 refactor: add a TagName type alias
Slightly better self-documentation in code, even if not actually
stronger typing.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-07 16:02:03 +03:00
Lars Wirzenius 765e6b606c fix: "cibtool run remove" removed only by broker run id by default
We need to parse the run info in the database as JSON to get at the
adapter run info. This fails if the run info can't be parsed. That can
happen if the database has run info from a development version. Change
`cibtool run remove` so that it takes only broker run ids by default
so that we can delete using those even if the run info is broken.
Allow removing by adapter run id with the `--adapter` option.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-07 15:35:47 +03:00
Lars Wirzenius 900a457157 fix: "cibtool run list" shows ids without loading run info
If the run info can't be parsed as JSON, loading it will fail. Change
`cibtool run list` so that it only lists the run ids by default.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-07 15:34:57 +03:00
Lars Wirzenius a7d02c18c4 feat: make Tag event filter take a regular expression
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-04 11:42:11 +03:00
Lars Wirzenius b783ef65d7 feat: events for created, updated, deleted Git tags
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-04 10:59:07 +03:00
Lars Wirzenius daa73ab7c2 refactor: use new Git ref parser to simplify from_node_event constructor
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-02 19:51:22 +03:00
Lars Wirzenius 504f13262c feat(src/ci_event.rs): add parser for Git refs for patch, branch, tag
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-04-02 19:21:05 +03:00
Lars Wirzenius 7cab201b41 feat: version number now includes Git commit short id
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-31 15:04:06 +03:00
Lars Wirzenius 5038769a41 feat(src/msg.rs): allow logging to stderr in AdminLog
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-26 07:56:43 +02:00
Lars Wirzenius 68a7a596bf feat: if Adminlog is null, don't log to stderr either
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-25 16:35:20 +02:00
Lars Wirzenius 2450f7b764 feat: log a warning if report directory does not exist
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-25 08:31:19 +02:00
Lars Wirzenius 33b4b65b0f fix(src/pages.rs): if report dir does not exist, silently do nothing
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 18:11:36 +02:00
Lars Wirzenius 88d819d556 fix(src/bin/cib.rs): log failure of page updater thread
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 18:07:38 +02:00
Lars Wirzenius 47fc131186 feat: log thread start/end at info level normally
This makes them more obvious than at the debug level. These are not
messages that repeat, so they won't be log spam.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 18:02:19 +02:00
Lars Wirzenius 192af400ed feat(src/logger.rs): report thread ending with error at error level
Previously, errors were reported at debug level. The error level makes
errors more obvious.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 17:57:34 +02:00
Lars Wirzenius 6e0d8853d0 fix(src/ci_event.rs): don't error if namespaced ref is not branch
When we get a CI event, we parse the namespaced name in the event to
see if it's a patch, or branch update. However, if might not be a
either: it might, for example, be a sigrefs update. Create a
branch event only if it's a branch event. Previously we would assume
it's branch event if not a patch event, and return an error if it's
not actually a branch.

If it's not a patch or branch change, ignore the event, as there's
nothing useful we can do about it, at least for now.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:42:27 +02:00
Lars Wirzenius 041e736a23 feat: when CI event source closes, log the error
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:40:02 +02:00
Lars Wirzenius 6ef5675a2a feat: reduce log spam
Don't log unstructured messages from various places in processing
events and the execution of a CI run. I think these are unnecessary
messages anyway, by now. If it turns out we need them, we'll add them
back as structured log messages.

Drop the now-unused logger::debug, debug2, trace, and trace2
functions, may the never return.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:37:59 +02:00
Lars Wirzenius 3e51c02d04 feat: make event queue processing use structured log messages
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:37:16 +02:00
Lars Wirzenius e17b467b37 fix(src/queueproc.rs): always drop a picked event from queue
Previously, we might not remove the event from the queue if something
went wrong while processing it. This could result in the CI broker
picking again in the future, and it might fail again, and repeat ad
nauseam.

It's better to remove the event from the queue at once, and log any
problems while processing it, and let humans deal with failures, and
decide if re-processing is needed.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:34:25 +02:00
Lars Wirzenius 11bb07ddf7 feat: always log thread start and result for page updating thread
This means that even if the thread ends due to an error, the error
gets logged. This will help with diagnosing problems.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:34:25 +02:00
Lars Wirzenius fb03457460 feat: always log the start and end result in the queueproc thread
This means that even if the thread ends due to an error, the error
gets logged. This will help with diagnosing problems.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:34:25 +02:00
Lars Wirzenius 923a4f1637 feat: always log thread start, end, result of queueadd thread
This means that even if the thread ends due to an error, the error
gets logged. This will help with diagnosing problems.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:34:23 +02:00
Lars Wirzenius b73d00df9d refactor: drop unused function ci_event_source_eof
Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:30:56 +02:00
Lars Wirzenius c22a12f085 feat: add a "kind" field to log messages that don't have it
Only queueproc_filter_decision and queueproc_filter_predicate seemed
to be missing it.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-24 11:30:56 +02:00
Lars Wirzenius 0d1d3291cc fix: use new Git refs module, fix bugs found
This is a huge change, but it's mostly a few kinds of similar changes
in a lot of places. Unfortunately, they were not easy to tease apart
into small, easily reviewed commits.

Use the Git ref types created by the new refs module. Fix problems
this exposes.

The CiEvent enum variants for branch events now use the BranchName
type for branch names. In other words, they have a plain branch name.
The CiEvent::from_node_event constructor also always sets the branch
name to a plain branch name, instead of a name spaced ref for updated
or deleted branches. The ci_event.rs module's unit tests have been
fixed to use the correct kind of Git ref in each test. (It used to be
a mess.)

The CI event filter "Branch" now contains a BranchName, previously a
RefString. This should not break anything, as a RefString will always
work as a BranchName.

The DefaultBranch filter now works, because it compares plain branch
names, where it previously would compare a plain and a name spaced
branch name, and those are never equal.

"cibtool trigger" and "cibtool event add" now add branch events with a plain
branch instead of a name spaced ref.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-20 18:33:39 +02:00
Lars Wirzenius 9222f6d332 feat: module for manipulating Git refs
Add helper functions to construct plain branch names ("main"), full
branch refs ("refs/heads/main"), and namespaced full branch
names ("refs/namespaces/$NID/refs/heads/main").

Keeping track of while only using RefStrings is too hard to get right.
We need to use the types provided by the radicle crate: BranchName,
Qualified, and Namespaced. Even if I find the type names less than
obvious, they're much better than plain RefStrings.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-20 17:57:29 +02:00
Lars Wirzenius 4f98127322 feat: RSS feeds now only contain up to ten entries
Very long RSS feeds can cause problems for some readers. Apparently
they forget they've already seen very old entries and show them as
new. To work around this, only include the ten newest CI runs.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-13 12:05:14 +02:00
Lars Wirzenius 746445cb86 feat: "event filter --explain" shows why an event was allowed or not
Also refactor the code that makes the decision for simplicity and
clarity.

Signed-off-by: Lars Wirzenius <liw@liw.fi>
2025-03-10 19:42:27 +02:00