Each COB change names an identity commit in its `Rad-Resource`
trailer. Patches and issues check the rights of the author against the
document in that commit. Nothing checked that the commit is a revision
of the repository identity. Any user could write a commit with a
document that names them as a delegate, cite it, and act as a delegate.
- Add `Identity::is_accepted`, which follows each revision's parent
back from `rad/id`, so a change that cites the head costs one blob
read
- Add `ReadRepository::identity_revision_doc`, which returns the
document of an accepted identity revision only
- Use it in `Op::identity_doc`, so a change with any other resource
gets no identity document and is rejected
- Document that `identity_doc_at` does not check the commit
- Test forged resources on patches and issues, and the accepted chain
Accepted revisions form a chain, because a revision is only adopted if
its parent is the current revision. Callers synchronize `rad/id` with
the current revision after identity writes, so the chain holds every
accepted revision.
Every caller of `IdentityMut` had to move `rad/id` itself. `rad id
accept` only moved it when the accepted revision became current. When
an accept also adopted a queued child, `rad/id` stayed behind.
- Move `rad/id` to the current revision after each identity write
- Route `IdentityMut::update` through the same commit helper
- Remove the now redundant updates from `rad id` and `rad publish`
- Test that an update and a cascading accept move `rad/id`
Only the first peer to send us a version of an announcement was
recorded as its relayer, as the `FIXME` in `handle_announcement`
notes. Inventory announcements are relayed with a delay, so peers that
sent the same version in the meantime were sent it back, which kept
`announcement_message_amplification` close to the worst case. Node and
refs announcements are relayed as soon as they arrive, so this does
not affect them.
The gossip store does not report which row a duplicate belongs to, so
key the relayers by announcer, type and repository instead, together
with the version they relayed. Having `Store::announced` report the
row of a duplicate would also work, but would change what its result
means for its other callers. As before, entries are never pruned, but
there is now at most one entry per announcer, type and repository,
rather than one per row ever created.
With this, 1000 runs of `announcement_message_amplification` measured
between 2.525 and 2.975, with a mean of 2.72, down from a mean of 3.98,
against a best case of 2.5 for five fully connected nodes. Require it
to stay below 3.25, which leaves room for the variation between runs.
The relayers of an announcement are keyed by its row in the gossip
store, which is the same for every version of the announcement, and
are only ever added to. A peer that relayed any version of it to us
is never sent a later one.
Keep only the relayers of the latest version. This also keeps each
list of relayers from growing with every version.
The amplification that `announcement_message_amplification` measures
rises slightly, from a mean of 3.90 to 3.98 over 300 runs each, since
peers that relayed an earlier version of the announcement are now sent
the new one as well. In one traced simulation, for example, Dan now
relays Amy's new inventory to Bob, which had relayed Amy's earlier
inventory to Dan.
A node does not relay a new version of an announcement to a peer that
relayed an earlier version of it. If the link between the announcer
and that peer drops, the peer stops receiving the announcer's updates
through other paths until it reconnects to the announcer.
Add a simulation in which Cid learns about Amy's inventory through
Bob, after which Amy drops Bob and connects to Cid. Bob does not learn
about Amy's new inventory. Assert this, and mark it to be fixed.
The simulator scheduled a wakeup relative to the simulation's time,
even when the node's clock is ahead of it, for example after
`Peer::elapse`. Tests that stagger nodes this way, such as
`announcement_message_amplification`, ended up running every node's
periodic tasks at the same moment.
Schedule the wakeup relative to the node's clock, or to the
simulation's time if the node's clock is behind. Other outputs are
still scheduled relative to the simulation's time.
With the nodes staggered, `announcement_message_amplification` still
measures close to the worst case: a node does not record the sender
of a duplicate as a relayer, as the `FIXME` in `handle_announcement`
notes, and so relays the message back to it.
The simulator skipped a wakeup whenever any node was already due to
wake at the same time, so of several nodes asking to wake at the same
moment, only the first was ever woken. With nodes started together,
most of them never ran their periodic tasks, relaying included.
Only skip a wakeup if the same node is already due to wake at that
time, which includes the later time slots that colliding inputs are
moved to.
`announcement_message_amplification` passed only because of this: in
every simulation, only Cid, whose node ID sorts first, ever woke, so
only Cid relayed the announcement. With every node waking at the same
moment, all of them relay it before receiving it from each other,
which brings the amplification close to the worst case. Assert that it
stays above 3, and mark the assertion to be lowered.
A simulation that fails under a random seed can only be investigated
if it can be run again with the same seed. Replaying from a seed is
also a first step towards the deterministic simulation testing
proposed [on Zulip][dst], and one that works with the simulator as it
is.
The simulator seeds its generator randomly and only logs the seed, so
there is no way to run a simulation with it again. Even with a fixed
seed, runs would not reliably replay, because partitions were decided
on the absolute time, and simulations start at `LocalTime::now()`.
Seed simulations from `RAD_SIM_SEED`, mention it next to the logged
seed, and decide partitions by the elapsed time. A fixed seed in every
test would also make runs reproducible, but would give up the
variation that random seeds provide.
Message timestamps still differ between runs of tests that start at
`LocalTime::now()`, and tests that draw from their own generator, such
as `announcement_message_amplification`, are not reproducible from the
seed alone yet.
[dst]: <https://radicle.zulipchat.com/#narrow/channel/369277-heartwood/topic/DST.3A.20Deterministic.20Simulation.20Testing/near/567515594>
Clippy reports `clippy::missing_transmute_annotations` in the
macro expansions for `refname!` and `pattern!`, including their
uses in `radicle-surf` tests.
Specify `String` as the source type in both calls to `transmute`.
Signing keys generated by `r#gen::<SigningKey>(n)` may collide,
leading to failure.
`SigningKey::mock(n)` derives the key from `n`, so keys
are distinct and stable across runs.
Bob may fetch before Alice creates the issue, because the
clone may join an in-flight fetch.
Create the issue before connecting the nodes to avoid the race.
Various small fixes:
1. Disable conversion of CRLFs for exact comparison of fixtures.
2. Do not assume that the path separator is `/`.
3. Skip a test that cannot work due to a backslash in a file name.
4. Enable a workaround for long paths.
On Windows, these tests sometimes produced a different error message.
The message itself, and the hint that follows, should be good enough to
match on.
We would like the diff output of changes to the identity document to not
collapse into separate hunks, but show all of the identity document in
the diff. The way to do this is to set the number of context lines
sufficiently high. It turns out that `u32::MAX` is too high for `xdiff`,
which is commonly used on Windows. `xdiff` stores the context lines in
a signed integer of 32 bits. Also, it appears to double line numbers
when it merges hunks. Thus, a safe value seems to be half of `i32::MAX`.
Maintaining tests for Windows and Unix separately, just because they use
different characters to separate paths is annoying.
This simple normalization function that in essence replaces backslashes
with forward slashes eats all cases in our tests.
Because tests are written "Unix-style", it actually does more harm
than good to use `winsplit` on Windows, because of differences in
behavior compared to `shlex`. For example, empty arguments are
dropped by `winsplit`.
The unauthorized clone in `rad-init-private-clone` keeps a 1s timeout
so it still fails quickly. The success paths used 1s and 3s, which is
not enough on slower hosts: `rad sync --announce` reports "All seeds
timed out" after Bob has already fetched, and `rad seed`/`rad clone`
miss the fetch handshake.
Raise those success timeouts to 30s, matching `DEFAULT_TIMEOUT`. The
wait ends as soon as the target is met, so passing runs are not slower.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`test_large_fetch` populates the repository with `scale.max(3)` but
waits only `9 * scale` seconds for `RefsFetched`. At the default scale
of 1 that is about 9MB of incompressible data with a 9s budget.
On slower hosts the wait times out before the fetch finishes. Use
`scale.max(3)` for the timeout as well, matching the payload size.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`git_push_canonical` clones the repository as Bob and then immediately
runs `git push` as Alice, expecting "Synced with 1 seed(s)".
`clone` announces Bob's inventory, but Alice only treats Bob as a seed
after that announcement is applied to her routing table. On a slower
host the reactor can still be busy when Alice looks up seeds, so she
reports "No seeds found".
Wait until Alice's routing table contains Bob for the repository before
pushing. Apply the same wait to the lightweight-tags variant, which
uses the same setup.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`fn program_reports_version` skips optional tools such as `jq` and
`jj` when they are missing. It currently only treats an error returned
by `std::process::Command::status` of kind
`std::io::ErrorKind::NotFound` as absence.
Rust's `std::command::Command` uses glibc's `posix_spawn` on Unix.
When `execve` fails after `spawn` has already returned, POSIX requires
the child to exit with code 127 rather than reporting `errno` to the
parent. glibc usually copies that failure back through a
`CLONE_VM|CLONE_VFORK` shared-stack channel, but not every
implementation preserves it. qemu-user, OpenBSD, and HPPA surface a
successful `spawn` and an exit code of 127 instead.
On those platforms the helper panics, so tests that are meant to skip
without `jq` or `jj` fail. Treat exit status 127 the same as
`NotFound`.
See `man:posix_spawn(3)` and POSIX.1.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
Instead of iterating over *all references* and filtering for those that
look like a COB of the expected type, use a glob so that Git can do more
lifting for us.
Valid COB refs always point to commits. However, any peer can write
an invalid COB ref that points to a blob. When the node fetches
that peer's refs and it tries to load them, it would fail the whole
`objects` or `types` call. The targeted COB could not be opened and every
listing of that type from storage failed, until the ref was removed.
Any peer could thus create invalid COB refs for existing COBs, causing
any other node that fetched those refs (e.g. a permissive node) to no
longer be able to list / display the COB in question. A malicious peer
could thus prevent permissive nodes from displaying any issues or patches,
e.g. on popular projects.
Skip the ref with a warning instead. A valid COB ref always points to a
commit, so honest data evaluates the same as before. Other conversion
errors still fail the call as before.
Tests cover opening and listing an issue next to such a ref, and
opening an issue whose only ref is such a ref.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
Any connected peer can send a `Subscribe` with `since` after `until`.
The service passes the range to `Store::filtered`, which asserts
`from <= to`. The panic stops the service thread.
Remove the assertion. The query already returns no rows when
`from > to`, so the peer gets no history.
There is no apparent reason why announcements with a timestamp of zero
cannot be stored. Avoid panicking.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
To determine whether a patch has been merged to the default branch, we
tally merges based on the default branch quorum: allow set and
threshold. With the introduction of custom branches, the tallying was
not updated to use the quorum rules for the target branch, instead
merging on the identity document threshold and delegate set.
This fix uses the `allow` set and the `threshold` of the most specific
rule that matches the target branch of patch.
Note that a reference that no rule matches never becomes canonical, so a
patch that targets one can never reach a quorum.
The crates `radicle` and `radicle-node` need the default hasher for
`IndexMap<K, V>` and `IndexSet<T>`, which currently gets enabled
indirectly by `serde_json`, which depends on `indexmap` with `std`
enabled by default.
Make the dependency on the `std` feature of `indexmap` explicit.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
`sync_routing` calls `get_inventory(&from)` once per inbound inventory
announcement, to work out which of a peer's previously-announced repos are
no longer in its inventory:
// crates/radicle-protocol/src/service.rs
synced.removed.extend(
self.db.routing().get_inventory(&from)?
.into_iter()
.filter(|rid| !included.contains(rid)),
);
That is `SELECT repo FROM routing WHERE node = ?`. The table's only index is
the implicit one for the primary key `(repo, node)`, whose leading column is
`repo`, so the lookup cannot use it and scans instead. The index fixes that.
`disconnect_unresponsive_peers` drops any session whose `last_active` is
older than STALE_CONNECTION_TIMEOUT (hard-coded to 2 minutes).
`last_active` is written in only two places: `to_connected`, and
`handle_message` (the gossip message handler).
Pack data never reaches `handle_message` which is what marks . It arrives as `FrameData::Git`
and goes straight to the worker thread over a channel. So a peer busy
sending us a large pack appears completely idle, and
after two minutes we hang up on it mid-transfer. gitoxide then sees a
truncated stream and the fetch fails with:
Clone of rad:<rid> from <nid> failed with
'gix fetch error: Failed to consume the pack sent by the remote'
This only affects large repositories. Small ones finish inside the
window. Neither `limits.fetchPackReceive` nor `limits.fetchTimeout`
bounds it, and it reproduces against every peer, since any peer sending a
big pack goes gossip-quiet for the duration.
We now treat arriving pack bytes as liveness, symmetrically with the
`FrameData::Gossip` arm immediately above, which already calls
`received_message`.
Note that a remote that spends more than STALE_CONNECTION_TIMEOUT
server-side (e.g. enumerating or compressing a large pack before sending
its first byte), is still reaped, and `limits.fetchTimeout` does not help.
This is not a change in behaviour. Gossip silence would have reaped it anyway.
`limits.routingMaxSize` is documented as "Number of routing table entries
before we start pruning", but the implementation diverges from this.
DELETE FROM routing WHERE node <> ?1 AND rowid IN
(SELECT rowid FROM routing WHERE timestamp < ?2 ORDER BY timestamp LIMIT ?3)
`?2` is `now - routingMaxAge`. So if nothing is older than routingMaxAge,
nothing is deleted, however far over routingMaxSize the table has grown. On a
seed following a busy network, entries are refreshed by gossip faster than
they age out, and the cap is never reachable.
For example, on a permissive node with routingMaxSize=1000, routingMaxAge=7d,
I observed:
routing rows 94,254
older than routingMaxAge, ie. prunable 16
Dropping routingMaxAge to 1 day cleared the backlog the 7-day setting had
accumulated -- from 94,254 down to 82,400 rows -- but then stopped, with
8 rows prunable and the table still ... 82x over the cap. The `routing` table
size is therefore decided by arrival rate; the configured max size has no
effect on it.
Split the two evictions so each does what its name says:
- size eviction removes the oldest entries down to `routingMaxSize`,
regardless of age;
- age eviction then removes anything still older than
`routingMaxAge`.
Size runs first: it takes the oldest rows, so the age pass only fires when
the table was near its limit.
All released versions of Radicle are affected by vulnerabilities [1] in the
network protocol: traffic is unencrypted and peers are not authenticated.
A network observer can read the contents of private repositories, and an
attacker can impersonate a node that to fetch them. No fix has been released
yet.
Since users may rely on private repositories being confidential, `rad
init` now warns about this before the interactive "Visibility" prompt,
so the warning is printed before the choice is made. The warning is printed
again in the epilogue whenever the created repository is private. This
also covers `--private`, which skips the prompt.
Update the `rad-init-private` and `rad-init-private-no-seed` examples
to match the new output.
[1]: https://radicle.dev/2026/09/23/disclosure-of-vulnerability-in-network-protocol
`VarInt` can get large enough to exhaust available memory on many
machines. Instead of allocating directly, check the size of the
allocation against some limit.
The limit chosen for frames is 128 KiB, which is roughly twice as much
as actual gossip and Git protocol frames are expected to be. However,
128 KiB is arguably short enough to not burden a standard consumer
machine too much.
This redundancy does not add much. These functions are already marked
via `#[test]`, and often times they are in a separate module named
"test" or "tests".