It's pretty annoying to have untracked (or even ignored!) stray files
checked when committing.
Script checks treat placeholders as filenames.
Remove the find-specific arguments from the xargs command so
ShellCheck receives only tracked shell script paths.
Only the first peer to send us a version of an announcement was
recorded as its relayer, as the `FIXME` in `handle_announcement`
notes. Inventory announcements are relayed with a delay, so peers that
sent the same version in the meantime were sent it back, which kept
`announcement_message_amplification` close to the worst case. Node and
refs announcements are relayed as soon as they arrive, so this does
not affect them.
The gossip store does not report which row a duplicate belongs to, so
key the relayers by announcer, type and repository instead, together
with the version they relayed. Having `Store::announced` report the
row of a duplicate would also work, but would change what its result
means for its other callers. As before, entries are never pruned, but
there is now at most one entry per announcer, type and repository,
rather than one per row ever created.
With this, 1000 runs of `announcement_message_amplification` measured
between 2.525 and 2.975, with a mean of 2.72, down from a mean of 3.98,
against a best case of 2.5 for five fully connected nodes. Require it
to stay below 3.25, which leaves room for the variation between runs.
The relayers of an announcement are keyed by its row in the gossip
store, which is the same for every version of the announcement, and
are only ever added to. A peer that relayed any version of it to us
is never sent a later one.
Keep only the relayers of the latest version. This also keeps each
list of relayers from growing with every version.
The amplification that `announcement_message_amplification` measures
rises slightly, from a mean of 3.90 to 3.98 over 300 runs each, since
peers that relayed an earlier version of the announcement are now sent
the new one as well. In one traced simulation, for example, Dan now
relays Amy's new inventory to Bob, which had relayed Amy's earlier
inventory to Dan.
A node does not relay a new version of an announcement to a peer that
relayed an earlier version of it. If the link between the announcer
and that peer drops, the peer stops receiving the announcer's updates
through other paths until it reconnects to the announcer.
Add a simulation in which Cid learns about Amy's inventory through
Bob, after which Amy drops Bob and connects to Cid. Bob does not learn
about Amy's new inventory. Assert this, and mark it to be fixed.
The simulator scheduled a wakeup relative to the simulation's time,
even when the node's clock is ahead of it, for example after
`Peer::elapse`. Tests that stagger nodes this way, such as
`announcement_message_amplification`, ended up running every node's
periodic tasks at the same moment.
Schedule the wakeup relative to the node's clock, or to the
simulation's time if the node's clock is behind. Other outputs are
still scheduled relative to the simulation's time.
With the nodes staggered, `announcement_message_amplification` still
measures close to the worst case: a node does not record the sender
of a duplicate as a relayer, as the `FIXME` in `handle_announcement`
notes, and so relays the message back to it.
The simulator skipped a wakeup whenever any node was already due to
wake at the same time, so of several nodes asking to wake at the same
moment, only the first was ever woken. With nodes started together,
most of them never ran their periodic tasks, relaying included.
Only skip a wakeup if the same node is already due to wake at that
time, which includes the later time slots that colliding inputs are
moved to.
`announcement_message_amplification` passed only because of this: in
every simulation, only Cid, whose node ID sorts first, ever woke, so
only Cid relayed the announcement. With every node waking at the same
moment, all of them relay it before receiving it from each other,
which brings the amplification close to the worst case. Assert that it
stays above 3, and mark the assertion to be lowered.
A simulation that fails under a random seed can only be investigated
if it can be run again with the same seed. Replaying from a seed is
also a first step towards the deterministic simulation testing
proposed [on Zulip][dst], and one that works with the simulator as it
is.
The simulator seeds its generator randomly and only logs the seed, so
there is no way to run a simulation with it again. Even with a fixed
seed, runs would not reliably replay, because partitions were decided
on the absolute time, and simulations start at `LocalTime::now()`.
Seed simulations from `RAD_SIM_SEED`, mention it next to the logged
seed, and decide partitions by the elapsed time. A fixed seed in every
test would also make runs reproducible, but would give up the
variation that random seeds provide.
Message timestamps still differ between runs of tests that start at
`LocalTime::now()`, and tests that draw from their own generator, such
as `announcement_message_amplification`, are not reproducible from the
seed alone yet.
[dst]: <https://radicle.zulipchat.com/#narrow/channel/369277-heartwood/topic/DST.3A.20Deterministic.20Simulation.20Testing/near/567515594>
Clippy reports `clippy::missing_transmute_annotations` in the
macro expansions for `refname!` and `pattern!`, including their
uses in `radicle-surf` tests.
Specify `String` as the source type in both calls to `transmute`.
Both `cargo deny` and `cargo audit` can check for advisories. We want
to use `cargo audit` for advisories and `cargo deny` for its other
checks.
Notify users of `cargo deny` by providing an obviously non-existing
database path to `cargo deny`, which is printed as part of the error
message.
Signing keys generated by `r#gen::<SigningKey>(n)` may collide,
leading to failure.
`SigningKey::mock(n)` derives the key from `n`, so keys
are distinct and stable across runs.
Bob may fetch before Alice creates the issue, because the
clone may join an in-flight fetch.
Create the issue before connecting the nodes to avoid the race.
Various small fixes:
1. Disable conversion of CRLFs for exact comparison of fixtures.
2. Do not assume that the path separator is `/`.
3. Skip a test that cannot work due to a backslash in a file name.
4. Enable a workaround for long paths.
On Windows, these tests sometimes produced a different error message.
The message itself, and the hint that follows, should be good enough to
match on.
We would like the diff output of changes to the identity document to not
collapse into separate hunks, but show all of the identity document in
the diff. The way to do this is to set the number of context lines
sufficiently high. It turns out that `u32::MAX` is too high for `xdiff`,
which is commonly used on Windows. `xdiff` stores the context lines in
a signed integer of 32 bits. Also, it appears to double line numbers
when it merges hunks. Thus, a safe value seems to be half of `i32::MAX`.
Maintaining tests for Windows and Unix separately, just because they use
different characters to separate paths is annoying.
This simple normalization function that in essence replaces backslashes
with forward slashes eats all cases in our tests.
Because tests are written "Unix-style", it actually does more harm
than good to use `winsplit` on Windows, because of differences in
behavior compared to `shlex`. For example, empty arguments are
dropped by `winsplit`.
The unauthorized clone in `rad-init-private-clone` keeps a 1s timeout
so it still fails quickly. The success paths used 1s and 3s, which is
not enough on slower hosts: `rad sync --announce` reports "All seeds
timed out" after Bob has already fetched, and `rad seed`/`rad clone`
miss the fetch handshake.
Raise those success timeouts to 30s, matching `DEFAULT_TIMEOUT`. The
wait ends as soon as the target is met, so passing runs are not slower.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`test_large_fetch` populates the repository with `scale.max(3)` but
waits only `9 * scale` seconds for `RefsFetched`. At the default scale
of 1 that is about 9MB of incompressible data with a 9s budget.
On slower hosts the wait times out before the fetch finishes. Use
`scale.max(3)` for the timeout as well, matching the payload size.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`git_push_canonical` clones the repository as Bob and then immediately
runs `git push` as Alice, expecting "Synced with 1 seed(s)".
`clone` announces Bob's inventory, but Alice only treats Bob as a seed
after that announcement is applied to her routing table. On a slower
host the reactor can still be busy when Alice looks up seeds, so she
reports "No seeds found".
Wait until Alice's routing table contains Bob for the repository before
pushing. Apply the same wait to the lightweight-tags variant, which
uses the same setup.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
`fn program_reports_version` skips optional tools such as `jq` and
`jj` when they are missing. It currently only treats an error returned
by `std::process::Command::status` of kind
`std::io::ErrorKind::NotFound` as absence.
Rust's `std::command::Command` uses glibc's `posix_spawn` on Unix.
When `execve` fails after `spawn` has already returned, POSIX requires
the child to exit with code 127 rather than reporting `errno` to the
parent. glibc usually copies that failure back through a
`CLONE_VM|CLONE_VFORK` shared-stack channel, but not every
implementation preserves it. qemu-user, OpenBSD, and HPPA surface a
successful `spawn` and an exit code of 127 instead.
On those platforms the helper panics, so tests that are meant to skip
without `jq` or `jj` fail. Treat exit status 127 the same as
`NotFound`.
See `man:posix_spawn(3)` and POSIX.1.
Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
Instead of iterating over *all references* and filtering for those that
look like a COB of the expected type, use a glob so that Git can do more
lifting for us.
Valid COB refs always point to commits. However, any peer can write
an invalid COB ref that points to a blob. When the node fetches
that peer's refs and it tries to load them, it would fail the whole
`objects` or `types` call. The targeted COB could not be opened and every
listing of that type from storage failed, until the ref was removed.
Any peer could thus create invalid COB refs for existing COBs, causing
any other node that fetched those refs (e.g. a permissive node) to no
longer be able to list / display the COB in question. A malicious peer
could thus prevent permissive nodes from displaying any issues or patches,
e.g. on popular projects.
Skip the ref with a warning instead. A valid COB ref always points to a
commit, so honest data evaluates the same as before. Other conversion
errors still fail the call as before.
Tests cover opening and listing an issue next to such a ref, and
opening an issue whose only ref is such a ref.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
Any connected peer can send a `Subscribe` with `since` after `until`.
The service passes the range to `Store::filtered`, which asserts
`from <= to`. The panic stops the service thread.
Remove the assertion. The query already returns no rows when
`from > to`, so the peer gets no history.
There is no apparent reason why announcements with a timestamp of zero
cannot be stored. Avoid panicking.
Co-authored-by: Lorenz Leutgeb <lorenz.leutgeb@radicle.dev>
This reverts commit a87e0bd1c8, which
changed from downloading Zig to installing it via `apk`.
The reason to do this back then was because the download failed after
the version bump in 8d50b75826.
However, the reason for the download to fail was that the file name
pattern for the tarballs changed, see
<https://ziggit.dev/t/zig-0-14-1-released/10233/2> and that no
indirection via <https://ziglang.org/download/index.json> was done.
Now, as 0.17.0 is released, and the package in the Alpine repositories
is taking time to be updated, we revert the change, and rewrite the
download step to be more resilient.
Richard Levitte