Both `cargo deny` and `cargo audit` can check for advisories. We want
to use `cargo audit` for advisories and `cargo deny` for its other
checks.
Notify users of `cargo deny` by providing an obviously non-existing
database path to `cargo deny`, which is printed as part of the error
message.
Signing keys generated by `r#gen::<SigningKey>(n)` may collide,
leading to failure.
`SigningKey::mock(n)` derives the key from `n`, so keys
are distinct and stable across runs.
Bob may fetch before Alice creates the issue, because the
clone may join an in-flight fetch.
Create the issue before connecting the nodes to avoid the race.
Various small fixes:
1. Disable conversion of CRLFs for exact comparison of fixtures.
2. Do not assume that the path separator is `/`.
3. Skip a test that cannot work due to a backslash in a file name.
4. Enable a workaround for long paths.
On Windows, these tests sometimes produced a different error message.
The message itself, and the hint that follows, should be good enough to
match on.
We would like the diff output of changes to the identity document to not
collapse into separate hunks, but show all of the identity document in
the diff. The way to do this is to set the number of context lines
sufficiently high. It turns out that `u32::MAX` is too high for `xdiff`,
which is commonly used on Windows. `xdiff` stores the context lines in
a signed integer of 32 bits. Also, it appears to double line numbers
when it merges hunks. Thus, a safe value seems to be half of `i32::MAX`.
Maintaining tests for Windows and Unix separately, just because they use
different characters to separate paths is annoying.
This simple normalization function that in essence replaces backslashes
with forward slashes eats all cases in our tests.
Because tests are written "Unix-style", it actually does more harm
than good to use `winsplit` on Windows, because of differences in
behavior compared to `shlex`. For example, empty arguments are
dropped by `winsplit`.
The unauthorized clone in `rad-init-private-clone` keeps a 1s timeout
so it still fails quickly. The success paths used 1s and 3s, which is
not enough on slower hosts: `rad sync --announce` reports "All seeds
timed out" after Bob has already fetched, and `rad seed`/`rad clone`
miss the fetch handshake.
Raise those success timeouts to 30s, matching `DEFAULT_TIMEOUT`. The
wait ends as soon as the target is met, so passing runs are not slower.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`test_large_fetch` populates the repository with `scale.max(3)` but
waits only `9 * scale` seconds for `RefsFetched`. At the default scale
of 1 that is about 9MB of incompressible data with a 9s budget.
On slower hosts the wait times out before the fetch finishes. Use
`scale.max(3)` for the timeout as well, matching the payload size.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`git_push_canonical` clones the repository as Bob and then immediately
runs `git push` as Alice, expecting "Synced with 1 seed(s)".
`clone` announces Bob's inventory, but Alice only treats Bob as a seed
after that announcement is applied to her routing table. On a slower
host the reactor can still be busy when Alice looks up seeds, so she
reports "No seeds found".
Wait until Alice's routing table contains Bob for the repository before
pushing. Apply the same wait to the lightweight-tags variant, which
uses the same setup.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`fn program_reports_version` skips optional tools such as `jq` and
`jj` when they are missing. It currently only treats an error returned
by `std::process::Command::status` of kind
`std::io::ErrorKind::NotFound` as absence.
Rust's `std::command::Command` uses glibc's `posix_spawn` on Unix.
When `execve` fails after `spawn` has already returned, POSIX requires
the child to exit with code 127 rather than reporting `errno` to the
parent. glibc usually copies that failure back through a
`CLONE_VM|CLONE_VFORK` shared-stack channel, but not every
implementation preserves it. qemu-user, OpenBSD, and HPPA surface a
successful `spawn` and an exit code of 127 instead.
On those platforms the helper panics, so tests that are meant to skip
without `jq` or `jj` fail. Treat exit status 127 the same as
`NotFound`.
See `man:posix_spawn(3)` and POSIX.1.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
Instead of iterating over *all references* and filtering for those that
look like a COB of the expected type, use a glob so that Git can do more
lifting for us.
Valid COB refs always point to commits. However, any peer can write
an invalid COB ref that points to a blob. When the node fetches
that peer's refs and it tries to load them, it would fail the whole
`objects` or `types` call. The targeted COB could not be opened and every
listing of that type from storage failed, until the ref was removed.
Any peer could thus create invalid COB refs for existing COBs, causing
any other node that fetched those refs (e.g. a permissive node) to no
longer be able to list / display the COB in question. A malicious peer
could thus prevent permissive nodes from displaying any issues or patches,
e.g. on popular projects.
Skip the ref with a warning instead. A valid COB ref always points to a
commit, so honest data evaluates the same as before. Other conversion
errors still fail the call as before.
Tests cover opening and listing an issue next to such a ref, and
opening an issue whose only ref is such a ref.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
Any connected peer can send a `Subscribe` with `since` after `until`.
The service passes the range to `Store::filtered`, which asserts
`from <= to`. The panic stops the service thread.
Remove the assertion. The query already returns no rows when
`from > to`, so the peer gets no history.
There is no apparent reason why announcements with a timestamp of zero
cannot be stored. Avoid panicking.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
This reverts commit a87e0bd1c8, which
changed from downloading Zig to installing it via `apk`.
The reason to do this back then was because the download failed after
the version bump in 8d50b75826.
However, the reason for the download to fail was that the file name
pattern for the tarballs changed, see
<https://ziggit.dev/t/zig-0-14-1-released/10233/2> and that no
indirection via <https://ziglang.org/download/index.json> was done.
Now, as 0.17.0 is released, and the package in the Alpine repositories
is taking time to be updated, we revert the change, and rewrite the
download step to be more resilient.
To determine whether a patch has been merged to the default branch, we
tally merges based on the default branch quorum: allow set and
threshold. With the introduction of custom branches, the tallying was
not updated to use the quorum rules for the target branch, instead
merging on the identity document threshold and delegate set.
This fix uses the `allow` set and the `threshold` of the most specific
rule that matches the target branch of patch.
Note that a reference that no rule matches never becomes canonical, so a
patch that targets one can never reach a quorum.
The crates `radicle` and `radicle-node` need the default hasher for
`IndexMap<K, V>` and `IndexSet<T>`, which currently gets enabled
indirectly by `serde_json`, which depends on `indexmap` with `std`
enabled by default.
Make the dependency on the `std` feature of `indexmap` explicit.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
`sync_routing` calls `get_inventory(&from)` once per inbound inventory
announcement, to work out which of a peer's previously-announced repos are
no longer in its inventory:
// crates/radicle-protocol/src/service.rs
synced.removed.extend(
self.db.routing().get_inventory(&from)?
.into_iter()
.filter(|rid| !included.contains(rid)),
);
That is `SELECT repo FROM routing WHERE node = ?`. The table's only index is
the implicit one for the primary key `(repo, node)`, whose leading column is
`repo`, so the lookup cannot use it and scans instead. The index fixes that.
`disconnect_unresponsive_peers` drops any session whose `last_active` is
older than STALE_CONNECTION_TIMEOUT (hard-coded to 2 minutes).
`last_active` is written in only two places: `to_connected`, and
`handle_message` (the gossip message handler).
Pack data never reaches `handle_message` which is what marks . It arrives as `FrameData::Git`
and goes straight to the worker thread over a channel. So a peer busy
sending us a large pack appears completely idle, and
after two minutes we hang up on it mid-transfer. gitoxide then sees a
truncated stream and the fetch fails with:
Clone of rad:<rid> from <nid> failed with
'gix fetch error: Failed to consume the pack sent by the remote'
This only affects large repositories. Small ones finish inside the
window. Neither `limits.fetchPackReceive` nor `limits.fetchTimeout`
bounds it, and it reproduces against every peer, since any peer sending a
big pack goes gossip-quiet for the duration.
We now treat arriving pack bytes as liveness, symmetrically with the
`FrameData::Gossip` arm immediately above, which already calls
`received_message`.
Note that a remote that spends more than STALE_CONNECTION_TIMEOUT
server-side (e.g. enumerating or compressing a large pack before sending
its first byte), is still reaped, and `limits.fetchTimeout` does not help.
This is not a change in behaviour. Gossip silence would have reaped it anyway.
`limits.routingMaxSize` is documented as "Number of routing table entries
before we start pruning", but the implementation diverges from this.
DELETE FROM routing WHERE node <> ?1 AND rowid IN
(SELECT rowid FROM routing WHERE timestamp < ?2 ORDER BY timestamp LIMIT ?3)
`?2` is `now - routingMaxAge`. So if nothing is older than routingMaxAge,
nothing is deleted, however far over routingMaxSize the table has grown. On a
seed following a busy network, entries are refreshed by gossip faster than
they age out, and the cap is never reachable.
For example, on a permissive node with routingMaxSize=1000, routingMaxAge=7d,
I observed:
routing rows 94,254
older than routingMaxAge, ie. prunable 16
Dropping routingMaxAge to 1 day cleared the backlog the 7-day setting had
accumulated -- from 94,254 down to 82,400 rows -- but then stopped, with
8 rows prunable and the table still ... 82x over the cap. The `routing` table
size is therefore decided by arrival rate; the configured max size has no
effect on it.
Split the two evictions so each does what its name says:
- size eviction removes the oldest entries down to `routingMaxSize`,
regardless of age;
- age eviction then removes anything still older than
`routingMaxAge`.
Size runs first: it takes the oldest rows, so the age pass only fires when
the table was near its limit.