Commit Graph
100 Commits
Author SHA1 Message Date
Fintan Halpenny 96637aca7e chore: update rust-toolchain
Update the rust toolchain version to 1.85.

Note that this is not the latest, however, it is the latest available in nixpgs,
so this version is good enough for now.
2025-04-17 10:39:34 +02:00
Fintan Halpenny 2af090ea1e cli: fallible comment selector
The usage of the `comment_select` used `unwrap`. It would be better to have the
function be fallible instead and provide a somewhat helpful message.
2025-04-16 16:02:58 +02:00
Fintan Halpenny 538648c542 cli: document disallow and edit options in manpages
There were missing descriptions for the `--disallow` and `--edit` options in the
`rad-id` manpages.
2025-04-15 10:44:00 +02:00
Fintan Halpenny 9ef9c5d59a radicle: add Op::load method
Adds an `Op::load` helper method for loading an `Entry`, given its `Oid` and the
`store` it is stored in.

This can be useful for downstream consumers to inspect operations on COBs given
a single `Oid`, without having to load the entire object and/or graph.

It can also be useful when looking at implementing COB stream primitives.
2025-04-11 12:52:09 +02:00
Fintan Halpenny c847a16e14 radicle: return iterator types for db policies
Instead of collecting the results and returning the collection as an
iterator, introduce `FollowPolicies` and `SeedPolicies` types, which
implement `Iterator`.

Note that a call in `service` changed due to the lifetime borrow. It
simply `collect`s into a `Vec` for the time being.
2025-04-11 12:35:55 +02:00
Fintan Halpenny 7de82b5025 radicle: use Alias in follow
Ensure that the validated `Alias` type is used for inserting into the
policy database.
2025-04-11 12:35:55 +02:00
Fintan Halpenny 6940ac42be radicle: reverse lookup for AliasStore
Provide functionality to lookup the `NodeId`s associated with a given
`Alias`.

The use for this functionality is to allow callers to use aliases as
shorthands for `NodeId`s, but still refer to the exact identifier.

The implementation allows for searching using the `LIKE '%<pattern>%'`
queries in sqlite, so searches do not have to be exact matches.
2025-04-11 12:18:35 +02:00
Fintan Halpenny 3dba4fbc91 cli: cargo fmt
Missed a `cargo fmt` for the feedback print statements
2025-04-10 16:17:01 +02:00
Fintan Halpenny dd5f739630 radicle: generic Transaction::initial
The use of `Transaction` outside of the `radicle` crate requires that a newtype is used, if the upstream code wants to write methods, such as:

```
pub struct Transaction<R>(store::Transaction<MyCob, R>);

impl Transaction<R> {
  pub fn add_foo(&mut self, foo: Foo) -> Result<(), store::Error> { /* code */ }
  pub fn add_bar(&mut self, bar: Bar) -> Result<(), store::Error> { /* code */ }}
```

This works up until the point that it tries to re-use the
`Transaction::initial` method.

To make this possible, the method needs to know that it can be
converted to the upstream `Transaction` and back to `radicle`
`Transaction`.
2025-04-10 15:37:08 +02:00
Fintan Halpenny 170915ffcf nix: Nix flake maintenance
The nix flake's checks were failing for various reasons. The following fixes are
applied to get it up to date again.

- Update to release-24.11, since `crane` requires it – also to keep up-to-date
- Add `.json` to the source filters to allow for testing patch migration
- Ensure that the `rad` and `git-remote-rad` binaries are built for the nextest
  check. N.b. the binaries need to be in the target dir of `radicle-cli` since
  that is where the nextest check runs from.
- Add `cargo-audit` and `cargo-deny`, to the devShell, to allow for running those
  tools locally
- Bump `gix-path` to 0.10.11 to avoid a vulnerability[^0]

[^0]: https://rustsec.org/advisories/RUSTSEC-2024-0371
2025-04-10 15:29:08 +02:00
Fintan Halpenny c2863c08ab radicle: extend emoji support
Extend the ranges of emojis that are supported, listed
[below](#new-emojis-supported).

To ensure that single `char` emojis are supported, the `emojis` crate
is added as a dev dependency, using it to check that a `Reaction` can
be constructed for the two groups `SmileysAndEmotion` and
`PeopleAndBody`.

### New Emojis Supported

'🩵' 0x1FA75
'🩶' 0x1FA76
'🩷' 0x1FA77
'🫀' 0x1FAC0
'🫁' 0x1FAC1
'🫂' 0x1FAC2
'🫃' 0x1FAC3
'🫄' 0x1FAC4
'🫅' 0x1FAC5
'🫆' 0x1FAC6
'🫠' 0x1FAE0
'🫡' 0x1FAE1
'🫢' 0x1FAE2
'🫣' 0x1FAE3
'🫤' 0x1FAE4
'🫥' 0x1FAE5
'🫦' 0x1FAE6
'🫨' 0x1FAE8
'🫩' 0x1FAE9
'🫰' 0x1FAF0
'🫱' 0x1FAF1
'🫲' 0x1FAF2
'🫳' 0x1FAF3
'🫴' 0x1FAF4
'🫵' 0x1FAF5
'🫶' 0x1FAF6
'🫷' 0x1FAF7
'🫸' 0x1FAF8
2025-04-10 15:18:31 +02:00
Fintan Halpenny a723068246 cli: provide error context for ssh-agent connect
Provide more context to the ssh-agent error for easier debugging when `rad auth`.
fails.
2025-04-10 14:48:37 +02:00
Fintan Halpenny 62d000f7cb radicle: fix parse_ref_* documentation
The `parse_ref` and `parse_ref_namespaced` documentation used 4 space
indentation for the reference names. Unfortunately, `cargo test` attempts to
interpret this as code.

Use triple-backtick code fencing, with `text` and `no_run`, instead.
2025-01-16 11:08:35 +00:00
Fintan Halpenny ee0d19f2eb man: make a note on draft patches
Include a note on making draft patches – whether via the remote helper, or via
the CLI – in the section on opening patches.
2025-01-14 14:05:21 +01:00
Fintan Halpenny 4cced3ddb8 cli: add remaining patch actions
Add the remaining `Patch` actions to the CLI interface. These include:

- React to a revision
- Edit a comment
- React to a comment
- Redact a comment

The `react` subcommand is added in a straight-forward manner.

The comment commands are slightly less straight-forward. In order to
keep to one layer of subcommands, these variants are done through the
`rad patch comment` subcommand. They are accessed via options:
`--edit`, `--react`, and `--redact`, respectively. Each of these
options takes the `CommentId` for which the action is affecting. The
`message`, `react` (introduced for `rad patch react`), and `undo`
variables repurposed for this subset of actions.
2025-01-14 14:00:57 +01:00
Fintan Halpenny f0390e00d9 cli: print success to console on rad issue state
Indicate that the `rad issue state` operation was successful to the
user by printing a message to the console.

Solves:

    rad issue show 71020de847
2025-01-14 13:41:13 +01:00
Fintan Halpenny ffbdabe432 cli: add comment edit for rad issue
Add the ability to edit an issue comment to the `rad issue` command.

This is achieved by adding an `--edit` option to `rad issue
comment`. Note that if `--reply-to` and `--edit` are both specified,
then this results in an error.
2025-01-14 13:39:09 +01:00
Fintan Halpenny 6598243465 radicle: document parse_ref and parse_ref_namespaced
Provide better documentation for the two methods.

Solves the issue:
```
rad issue show 4603f55d97
```

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2025-01-14 13:34:13 +01:00
Fintan Halpenny d9c76893a1 cli: bump version to 0.12.1 2024-12-04 12:47:00 +01:00
Fintan Halpenny 19bbdbca24 cli: fix syntax highlighting
There was mishap in updating the parameters of highlight configuration – so many
string types!

This is fixed by adding the correct `name` parameter, reusing the `language`
variable that is being matched on.
2024-12-04 12:47:00 +01:00
Fintan Halpenny 61865b5b5a cob: fix documentation
`no_run` will compile but not run the code. Use `ignore` instead, and also added
`rust` for syntax highlighting.
2024-12-04 12:40:40 +01:00
Fintan Halpenny 259adf7dc2 cob: mutable stable time for testing
The `radicle-cob` crate provides a feature flag, `stable-commit-ids` to allow
for `Oid`s to remain stable for testing by always using the same author and
timestamp when creating commits.

This can be an issue when it interplays with COB related tests that are relying
on the ordering for the DAG walk. If the `Oid` values change and get re-ordered
then the test will fail. The DAG for COBs are now ordered via the `Oid` and
`Timestamp`.

In this change, a global timestamp is introduced that is initialised
per thread. The value is read, via `read_timestamp`, and can be
advanced using `with_advanced_timestamp` – while performing an action
that creates a new commit.
2024-11-18 16:11:44 +01:00
Fintan Halpenny 91914d9345 radicle: improve quorum copy
The `no quorum found` error can be opaque and lead to a lot of confusion. It is
better to give more information for this error so that it leads to easier
debugging, since it can be quite a common warning/error.

Instead of a single error, there are two cases provided: `NoCandidates` and
`Diverging`, to capture the two different variants of errors that can occur.

`Display` implementations are provided for both to provide more information.
2024-11-18 12:32:44 +01:00
Fintan Halpenny e412168be3 node: do not fail on set_head
The failure to set a HEAD ref, due to quourum, during a fetch should not fail
the fetch entirely.

Instead, log the success and failure, or return an error in other cases.

In fact, any failure to set the head could have resulted in notifications not
being sent, nor caching to occur.
2024-11-18 12:32:40 +01:00
Fintan Halpenny 159d3fce93 dag: test contains
Test coverage for the `Dag::contains` method were not covered. Add tests for the
method to cover an empty and filled `Dag`.
2024-11-18 11:21:17 +01:00
Fintan Halpenny 088339850b radicle: introduce identity document version
This change introduces versioning to the identity document format.

The `Version` type is introduced to capture this. Importantly, it is very
restricted in its construction and API. `Version::new` provides a smart
constructor to ensure that it is only ever constructed within the expected
bounds of `0 < v <= IDENTITY_VERSION`, where `IDENTITY_VERSION` is the constant
for the current version.

Note that serializing of the `Doc::version` is skipped to preserve the `Oid`
outputs of all the tests – which means that the first `version` is always
implicit and is not really expected to exist in the serialized forms of the
identity document.
2024-11-14 12:09:28 +01:00
Fintan Halpenny ccc0297b5a cli: test deletion via git push -d
Ensure that remote branches can be delete via the `-d` flag. This
includes the fact that one can delete the `refs/heads/patches`
branches, as long as they are owned by the current operator.
2024-11-11 12:00:45 +01:00
Fintan Halpenny 43e08a8e70 cli: rad id update --edit
Introduce an `--edit` option to `rad id update` to allow a user to open up their
text editor for changing the document's contents.
2024-11-11 11:58:09 +01:00
Fintan Halpenny c6d975799a cli: verification of project for json errors only
An `xyz.radicle.project` payload is allowed to not be defined, so report the
error if the project resulted in a JSON error.
2024-11-11 11:58:09 +01:00
Fintan Halpenny f9c3523183 e2e: improve flake in missing_delegate_default_branch
The `missing_delegate_default_branch` test was flakey and would fail checking if
Bob had the `rid` in storage.

This can be improved by waiting to ensure that Bob has a `RefsFetched` event
before performing the check.
2024-11-11 11:36:09 +01:00
Fintan Halpenny 0d402647cb term: allow Editor to be reusable
The `Editor` is very useful for correctly opening a text editor and making changes
to some initial input. The current use of `Editor` is only for getting input for
commets.

However, it would also be useful for opening up a text editor on some other
existing files or text, for example, in the command `rad config edit`.

The `Editor` struct was changed to have two new options, `truncate` and
`cleanup`, to allow the user of the struct to dictate whether existing text is
truncated, and if the underlying file should be remove.

The original `new` method is now named `comment`, mimicing the existing
construction of a temporary `RAD_COMMENT` file.

The new version of `new` accepts any file for the `Editor` and will open it
without truncating or removing the file.

`Editor` is now used for the `rad config edit` command.
2024-11-11 11:27:33 +01:00
Fintan Halpenny 23f8cf0d84 cli: Option for caching COBs for all repositories
To make it easier for caching COBs across all the repositories in storage, add
an option `--storage` that iterates over all the repositories and performs the
cache operation for each.
2024-11-07 16:56:38 +01:00
Fintan Halpenny eb095c109b fetch: allow missing default branch
When fetching, the validation logic would prune a delegate if it was missing the
default branch. Originally, this was done to prevent any broken state, but later
it was realised that this can be too restrictive and would not allow nodes to
recover into an improved state.

Remove this check so thtat nodes can still fetch from each other when the
default branch is missing.

An e2e test is added to show that nodes can still fetch from each other and
still receive other references, even when a delegate is missing their default
branch.
2024-11-06 16:47:01 +01:00
Fintan Halpenny b4f2614d6c cob: chronological ordering of concurrent values
The underlying `Dag` for a COB will use the `Ord` implementation of
whatever is provided as the key. This means that the lexicographical
ordering of the `Oid` SHA will be used, which in turn means that if
the SHAs change in our tests, then the ordering will be broken for
concurrent updates.

To prevent this from happening, a variant of the `prune` method is
introduced that takes an ordering of the keys and values – this method
being called `prune_by`.

This allows `prune` to be replaced with `prune_by` in the
`ChangeGraph::evaluate` method. The ordering used is first comparing
the `Entry::timestamp` and then the `EntryId` (`Oid`).
2024-10-23 13:02:55 +01:00
Fintan Halpenny de1958fab0 radicle: refactor doc
The aim of this change is to make the `Doc` type more safe to use by approaching
the design via [Parse don't validate][[0]] approach.

The problem with the previous approach was that all field were `pub` and thus a
`Doc<Verified>` could easily be mutated and serialized. Granted, the code that
used the serialization would tend to verify the `Doc` first, however, this
approach *ensures* that only a verified `Doc` can be serialized. It also meant
that trying to add new data that would follow the parse approach would require
more generic parameters on top of the existing `PhantomData` parameter, i.e. we
need to do something like: `Doc<RawField, V> -> Doc<ValidField, V>`.

The new approach splits the type into two separate types: `RawDoc` and `Doc`.
The former is allowed to be mutated at will, and uses types that are less
strict. The latter is the valid type that can only be constructed by validating
a `RawDoc` (or the `initial` constructor). The `Doc` type's fields can then only
be accessed by read-only methods.

Solves the problems above by only allowing mutations to `RawDoc`, as well as,
new fields being added to `RawDoc` which are then validated via
`RawDoc::verified`.

[0]: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/
2024-10-22 14:58:12 +01:00
Fintan Halpenny 46c2637f77 radicle: add tags fetch refspec
Add a fetch refspec to fetch tags from a Radicle remote. The
`--no-tag` option is also included. This is to ensure that it is easy
to collaborate by having tags placed in the remote namespace of
another peer. However, if the namespace is the default `rad` remote,
then tags are expected to be allowed -- this is to pave way for the
feature of canonical tags.

In the tests, the `--tags` option is removed -- and is generally
recommended that it is not used. This is to ensure that it does not
override the `--no-tag` option.
2024-10-21 14:30:37 +02:00
Fintan Halpenny e130b4dc06 radicle: custom upstream remote for patches
When creating a new upstream for the patch workflow, the remote name is always
assumed to be `rad`.

It's possible for users to use another remote name but still use a `rad://` URL
for pushing. To allow for this, the function that creates the upstream entry now
takes a remote name. In the remote helper, `rad patch checkout`, and `rad patch
set` the remote can be optionally specified and fall back to the `rad` remote
name as a default.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-09-12 17:59:57 +02:00
Fintan Halpenny f244d89e56 node: check policy before visibility
In the `is_authorized` helper, the logic checks the policy and the visibility of
the repository.

If the policy is set to block, the function can return before getting the
repository and the identity document. This improves the check, since the
repository and identity document may be missing if the repository is blocked, so
it would return a different error other than the expected unauthorized error.
2024-09-12 17:47:55 +02:00
Fintan Halpenny eb432b9bc1 cli: announce on issue edit and comment
The `rad issue edit` and `rad issue comment` subcommands were not included in
the announce check. Since these are considered write commands, they should also
announce when executed.
2024-08-20 16:32:02 +02:00
Fintan Halpenny 86a0443916 crdt: run cargo fmt 2024-08-09 12:41:01 +02:00
Fintan Halpenny e1470fccd7 fetch: update gix-pack and gix-odb
The `gix-fs` crate was flagged as vulnerable[^0]. To update it to `>=0.11.0`, it
was necessary to update the `gix-pack` and `gix-odb` crate versions to `0.51`
and `0.61`, respectively.

This in turn updated `fastrand` which changed its algorithm[^1], which meant
that some test outputs were reordered or matched new values.

[0]: https://rustsec.org/advisories/RUSTSEC-2024-0350
[1]: https://github.com/smol-rs/fastrand/blob/master/CHANGELOG.md#version-210
2024-08-09 12:41:01 +02:00
Fintan Halpenny 52554af4f1 cli: add rad unblock command
Add the `rad unblock` command for reversing the effect of blocking an NID or RID.

In both cases, the entry is removed from the DB, if the policy was set to block
-- otherwise it is a no-op.
2024-08-06 14:12:44 +02:00
Fintan Halpenny 81a3bc3d1b cli: rad id allow missing default branch
If a peer had created any references but was also missing the
canonical branch, the `rad id update` command will fail when adding
that peer as a delegate.

Instead, missing delegates and missing default branches are tracked
during verification. If the number of missing delegates means the
`threshold` cannot be met, then the errors are returned.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-24 16:43:36 +02:00
Fintan Halpenny dc34eafdf4 remote-helper: allow patch creation from detached HEAD
The patch creation flow would be partially interrupted when creating a
patch via a detached HEAD state. The creation of an upstream branch
would fail since there is no branch to set an upstream for. It would
result in the following error:

    ✓ Patch 6035d2f582afbe01ff23ea87528ae523d76875b6 opened
    To rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji/z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
     ! [remote rejected] HEAD -> refs/patches (git: reference 'HEAD' is neither a local nor a remote branch.; class=Invalid (3))
    error: failed to push some refs to 'rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji/z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi'

To fix this, the reference that's resolved is checked to see if it a
local branch. If not, it will return early.

The behaviour is tested in the `rad-patch-detached-head` example.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-23 17:13:30 +02:00
Fintan Halpenny 29e681210b node: test multiple offline inits
Ensure that when a node initialises multiple repositories, those
repositories can successfully be fetched by another peer once they
connect.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-22 10:48:10 +02:00
Fintan Halpenny fa19105f2f cli: add un/resolve for patch review comments
Patch review comments can be resolved and unresolved as part of the Patch COB
API. This functionality was missing from the `rad patch command`.

Add a subcommand `rad patch resolve` which can resolve a review comment, and
unresolve given the `--undo` flag.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-03 20:21:46 +02:00
Fintan Halpenny 84aaf9efd0 remote-helper: relax the quorum checks
The rules for checking the quorum of the remote helper, when pushing to the
canonical branch, can prevent the operator from even attempting to try converge
to a state of agreeance.

Instead, the checking of the quorum rules are relaxed. It is still the case that
if the peer is pushing a diverging commit while in a working state they are
warned and the push does not go through. However, there are two cases where it
is fine to force push a change:

1. When pushing a new tip, as a delegate, there is still a canonical tip and the
   new tip still converges with the history, i.e. it is equal to, behind, or
   ahead of another delegate tip.
2. If there is no quorum in the current state, for example there were multiple
   concurrent pushes, then the peer is free to continue to push to their default
   branch in attempt to converge with the other peers.

The example tests ensure that these cases are covered, while also preserving the
old behaviour.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-03 15:14:58 +01:00
Fintan Halpenny 54d23545ff radicle: port quorum code to Canonical
Remove the old `quorum` function in favour of using the newly introduced
`Canonical` code. Port over the tests to use the `Canonical` type. The change is
minimised by introducing a helper function called `quorum` in the test module
that acts like the old function.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-03 15:10:43 +01:00
Fintan Halpenny 7fd9c3be8f radicle: introduce Canonical type
Introduces a type, `Canonical`, to make manipulating and calculating canonical
reference quorum's easier.

The idea is that the type can be constructed by providing a set of delegates and
the reference name for which the calculations are referencing. It can construct
the set of tips and can output the quourum.

It provides a shortcut for constructing the `default_branch` set of tips.

It also provides some extra methods for helping in quorum calculations that will
be used within the remote helper code.

Note that this does not yet replace the `quorum` function in the `git` module --
this is to help minimise the review of this commit.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-03 15:09:38 +01:00
Fintan Halpenny b731dd1429 cli: test delegate convergence
If a delegate is in a diverged state from the canonical head, ensure
that they can easily reset to another delegate's head to converge
again.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-07-03 15:08:26 +01:00
Fintan Halpenny 80f1e6516c cli: idempotent rad id update
Instead of producing an error when there is no update during `rad id
update`, a message that the document is up to date is printed. The
command is considered is considered successful.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-06-21 13:01:59 +02:00
Fintan Halpenny f26be552c2 cli: improve rad id allow list UX
The previous implementation of using the `--allow` option in `rad id
update` would be an absolute list, rather than additive. That is, if
there were existing DIDs not passed in via `--allow` then they would
be remove.

Change this behaviour to check the existing value of the document's
allow list, and add to it if `--allow` is specified.

To enable the user to also remove a DID, the `--disallow` option is
also added.

The correct usage of these options is improved by checking the
validity of the `--visibility` option being used, the current state of
the repository's privacy, and the use of the `--allow` and
`--disallow` options.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-06-20 17:22:38 +02:00
Fintan Halpenny 3260046c67 radicle: fix diverging quorum
When calculating a quorum there was a case where two heads that were equal would
result in double-counting, allowing the quorum to pass a threshold higher than
the expected votes.

To prevent this, each head is immediately counted as a direct vote. Then, when
comparing the head to the rest of the set, if they are equal that iteration
would be skipped. This is because the merge base of two equal commits is that
commit, resulting in the double-counting.

Note that the `skip` can also skip the current head, so `i + 1` is used.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-06-20 14:57:00 +02:00
d46b0b6d11 radicle: Fix file limit setting for FreeBSD
On FreeBSD systems, the file limits are of type `i64` in comparison to
`u64` on Linux systems.

Also fix `rad web` command name on Windows and FreeBSD.

Co-authored-by: cloudhead <cloudhead@radicle.xyz>
Co-authored-by: Shawn Webb <shawn.webb@hardenedbsd.org>
2024-05-21 13:48:44 +02:00
Fintan Halpenny 064ece32ac node: reduce default upload-pack timeout
From empirical testing, the `FETCH_TIMEOUT` can be set to a lower value of 3s.
For a repository that is about 300MiB, the process will successfully send all
packfile bytes and the references will be written.

The `--timeout` option is also used in the `git-upload-pack` process for good
measure.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-17 21:05:50 +02:00
Fintan Halpenny de434bb437 node: simplify upload-pack reader joining
The `join` method is equivalent to the custom loop that was written for joining
the `reader` thread in upload-pack.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-17 21:05:50 +02:00
Fintan Halpenny 740106d759 cli: upload-pack events
Introduce an `UploadPack` type for keeping track of relevant data from
upload-pack events and displaying progress to the terminal.

It keeps track of the number of remotes that are being uploaded to as well as
the throughput of transmitted data.

The `Progress` events are logged rather than being displayed since they don't
provide any useful details to the user.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-17 21:05:43 +02:00
Fintan Halpenny d56d619f56 fetch: remove .keep files
A `.keep` file in the `objects/pack` directory acts as a lock to ensure that
operations like `git gc` do not clean up `.idx` files while a fetch is being
performed. It is safe to remove the `.keep` file once a fetch has completed and
a reference points to a given object in the packfile.

To ensure this happens in `radicle-fetch`, a newtype `Keepfile` is introduced.
It uses special `Drop` semantics by attempting to remove the file when the
object is dropped -- logging a warning if it fails to do so.

`FetchState` is augmented to keep a set of the `Keepfile`s. This ensures that
when the fetch completes, either successfully or with a failure, when
`FetchState` is dropped then so are the `Keepfile`s.

We ensure that the expected behaviour of removing the `.keep`files occurs by
adding a check in `e2e::test_replication` to check Alice's repository's packs.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-16 15:47:14 +02:00
Fintan Halpenny cb2cbf014a cli: reject rad id proposals that do not verify
To ensure that the `rad id` command does not modify the project payload without
it being able to be deserialized into the `Project` type, the command performs
this check throught the `project` method, and returns an error if this fails.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-16 10:11:48 +01:00
Fintan Halpenny 72c715016f fetch: add trace logging to fetch protocol
Log the following at the trace level:
- Receiving the pack along with the amount of time it took
- The closing of the connection

Also organising the imports, while in the neighbourhood.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny f038b7ee5c cli: watch for upload-pack events in rad init
To improve the user experience of `rad init`, the events listener waits for
upload-pack messages indefinitely. The spinner reports back the upload progress
over time.

This process can be cancelled via Ctrl+C without affecting the announcement,
since that is happening in the background.

To also improve the experience, the remote NIDs that have successfully synced
are reported to the terminal.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny 344fe3a901 node: increase FETCH_TIMEOUT
After empirically testing HardenedBSD[[0]], which has a total blob size of
42GiB, and its largest checkout is 1.26GiB (using `git-sizer`[[1]]), it was
found that letting the background process fetch the repository with a timeout of
1 hour is enough (a very generous estimate).

This deemed sufficient, since cases smaller than this will report back that they
are done before this time. For cases larger than this, more research will be
needed.

[0]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD
[1]: https://github.com/github/git-sizer

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan HalpennyandAlexis Sellier 027bfbef67 node: stream timeout semantics for Node::call
The timeout for a `UnixStream` will interrupt a `read` even when more data is
available. The preference would be that the timeout applies to each read, where
the timeout is reset again once a read is successful. This better supports
streaming semantics expected for the `Handle::subscribe` method.

To achieve this, the `LineIter` type holds the `UnixStream` and timeout
`Duration`. The `Iterator` implementation can then use `read_line` to progress
the stream one line at a time, and the timeout only applies to each read.

Co-authored-by: Alexis Sellier <alexis@radicle.xyz>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny c8a24d5584 node: report upload-pack progress
Emit the `UploadPack` events from the `upload_pack` method.

A wrapper type, `Reporter`, is introduced for implementing a `Write` instance
that will parse the given bytes as `gix_protocol::RemoteProgress`, if possible,
and emit it via the `Emitter`, and then write the bytes to the underlying
writer. This allows any subscribers to see the progress of an upload-pack, i.e.
`Compressing objects`, `Counting object`, `Enumerating objects`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny 89627a8b6a radicle: add upload-pack events
Add a variant for upload-pack events to allow reporting from the upload-pack
process.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny f0648c2a18 radicle: move Emitter to events module
Move the events Emitter to the `radicle::node::events` module to allow it to be
used outside of `radicle-node`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny fd4c4cd0af node: dynamic fetch timeout via channels
Instead of using `DEFAULT_CHANNEL_TIMEOUT`, the timeout is dynamically set
per-fetch by the caller.

This passed via the `Io::Fetch` variant -- when initiating a fetch, and the
`Control::Open` variant when responding to a fetch.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:55 +01:00
Fintan Halpenny e8f2f88bf6 cli: Add test for unknown field in identity doc
Add a test to confirm that the identity can still be deserialized when an
unknown field is added.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-07 10:21:07 +02:00
Fintan Halpenny 9b719d0cca radicle: allow for unknown fields in Project deserialization
Allow unknown fields to pass through when deserializing the `Project` payload.
This allows the deserialization to be more resilient when unknown fields are
added, e.g. from the CLI, and improves backwards-compatability when new fields
may be added in the future.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-07 10:19:59 +02:00
Fintan Halpenny 3556758386 cli: fix rad inbox clear output
If multiple ids are specified for `rad inbox clear`, it would report
that only 1 item was cleared, when in fact multiple were.

This was found to be due to the SQL code returning a change count of
`1`. This is likely due to the way the statement is being reset each
time. The `sqlite` library does not seem to easily support the usage
of `IN` in a `WHERE` clause. So instead, the `count` is aggregated in
the loop and returned instead.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-04-09 16:09:44 +02:00
Fintan Halpenny 8bf871903a node: handle removal of cob from cache
When a `Patch` or `Issue` is entirely removed from the Git repository,
then there is no update to be made -- instead it should also be
removed from the cache.

This covers the case where the original author of the COB deletes the
COB.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-04-09 16:06:51 +02:00
Fintan Halpenny ea69168f44 cli: explain public to private changes
Add an example of changing a public repository to a private
repository, noting that any replicated data will be public, but any
new changes will not be replicated.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-04-03 15:41:48 +02:00
Fintan Halpenny cd9b46fe51 radicle: fault tolerant thresholds
There are two areas where we can be more tolerant of delegate namespaces
being missing or not validating:
1. Calculating the canonical HEAD
2. Fetching from a remote

In 1. the protocol is tolerant in that if the local node does not have
the default branch for a delegate, it will still attempt to use any of
the delegates it does have to reach the threshold.

This is made safe by ensuring that if the threshold is being updated
then the node performing the update must have a threshold of delegates
locally in their storage. It also made safe by 2.

In 2. the protocol is tolerant by allowing delegates to be missing
from the serving side, as long as they can still meet a threshold of
delegates. This is further tolerant, when validating the received
data, a threshold of delegates are valid to consider the fetch
successful -- otherwise it will fail.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-04-03 14:30:05 +02:00
Fintan Halpenny ea31040ec6 cli: do not require RID for sync inventory
The inventory operation of the `rad sync` command does not require an
RID. Localise resolving the RID in the match arms that require it.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-21 20:29:11 +01:00
Fintan Halpenny ca9d2b44df node: fix clippy suggestion on borrowing
Remove the borrow of `rid` to fix the clippy warning.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-21 20:29:11 +01:00
Fintan Halpenny 3ad2b4431f fetch: only send changed wants and haves
This change limits the amount of `wants` and `haves` data that is sent
to the serving side of a fetch.

When a `RefsAt` announcement is sent, the fetching peer can calculate
the difference between the `rad/sigrefs` they're aware of -- if it
exists -- and the newly advertised `rad/sigrefs`. This means they can
efficiently ask for the `wants` and `haves` of references that have
changed -- saving some data sent over to the serving side.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-12 13:01:03 +01:00
Fintan Halpenny 14ce5fe7b1 cli: show reviews on rad patch list
When running `rad patch list`, show the set of delegate reviews for
each patch, with the following meaning for the symbols:

- ✔: delegate has accepted the patch
- ✗: delegate has rejected the patch
- -: delegate has not made a verdict on the patch

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 13:22:14 +01:00
Fintan Halpenny 18fc41c53f cli: error on no id update
If `rad id` is called with no arguments, or just `--title` and
`--description`, then it will fail because there were no changes made
to the identity.

Check if the `proposal` document is the same the `current` document,
and if so supply a hint and a better error.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 13:05:01 +01:00
Fintan Halpenny 864ca656c0 cli: add Unknown variant for NotificationKind
When the `rad inbox` command would come across a reference name that
it did not recognise as a `Cob` or `Branch`, then it would fail with
an `Unknown` error.

Instead, add an `Unknown` variant to `NotificationKind` which can be
displayed in the `inbox` CLI, by using the `--show-unknown`
flag. Otherwise, it will be skipped and the inbox will work as usual.

While refactoring, the `TypedId` type is used for the `Cob` variant to
simplify some of its uses.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 12:57:53 +01:00
Fintan Halpenny 71d4cb0128 radicle: TypedId improvements
- Add `Serialize` implementation
- Add `is_identity` method

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 12:57:53 +01:00
Fintan Halpenny 2fe0cca6d9 cli: ref_update should take a ref
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 12:57:53 +01:00
Fintan Halpenny 7ddb1119a5 cli: show revision on rad patch checkout
Help the user know which revision they currently have checked out when
running `rad patch checkout`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:56:11 +01:00
Fintan Halpenny b1435629f9 fetch: re-work transport::done signal call
The protocol requires that the fetcher signals to the remote that it
is done with the fetching process, so that the remote can exit out of
its upload-pack process.

The first change is that the `done` signal can happen before the
validation, since nothing else needs to be fetched.

The second change is that the fetcher does not care if the remote
responds to this signal since it has the data required for validation
and applying the changes to its refdb. Thus, if an error is
encountered then it is logged, but the rest of the process proceeds as
normal.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:42:15 +01:00
Fintan Halpenny c02ff1e47e fetch: log handshake error
When performing a handshake, the error source is hidden in the error
variant and never shown.

To get insight as to what the error was, it is logged when it is
encounterd.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:34:49 +01:00
Fintan Halpenny 6efd0d97a5 fetch: add logging for stages
Log when different stages of the fetch have completed, while also
providing the elapsed time, in seconds.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:34:17 +01:00
Fintan Halpenny 4b268adf49 node: downgrade log statements
Announcement are logged at the debug level and are very frequent. This
causes a lot of noisy logging in a node.

Downgrade the log levels for these messages from debug to trace.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:28:38 +01:00
Fintan Halpenny 68339d86b9 node: increase timeouts
For larger fetches, a timeout for the channels and socket control are
required. Bump the timeout from 9s to 30s.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:28:38 +01:00
Fintan Halpenny 4d50f79f1c node: remove unused timeout
The timeout was never being used, so it is being removed for the time
being.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:28:38 +01:00
Fintan Halpenny 87d1cb50a0 fetch: allow fetch of annotated tags
If an ancestry check of an annotated tag was triggered then the fetch
from that peer would fail. This is due to `graph_ahead_behind` only
working for commit-ish objects.

To allow for annotated tags, and other object that peel to commits,
the check now peels the two objects to commits before getting their
`Oid`s. The call to `graph_ahead_behind` can take these two `Oid`s
instead.

A test for annotated tags is included as part of the `examples/git`
suite in the CLI tests.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:13:26 +01:00
Fintan Halpenny cac6c0aad7 cli: add name to rad seed output
If the name is available to show for the repository -- it may not be
if it hasn't been replicated yet -- then add it to the output of the
`rad seed` command.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-09 13:45:35 +01:00
Fintan Halpenny bb4ed84b95 node: duplicate fetching logic respects RefsAt
Two concurrent fetches can occur from the same peer, while also being
for a different set of references. This was seen in the
`test_outdated_sigrefs` test. Alice would have an ongoing fetch from
Eve, and concurrently, Eve will create a new issue -- thus creating a
new `RefsAt` message -- and Alice would attempt to fetch this.

The previous logic would only check that Alice is fetching from Eve
and ignore the new updates -- meaning the test could fail. However, if
the `RefsAt` are checked then the other fetch can be recognised as
new, and queue it.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-09 13:43:08 +01:00
Fintan Halpenny 50833d88d9 cli: document rad inbox further
The documentation for `rad inbox show` and `rad inbox clear` were
missing.

Add documentation into the help message for both.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-09 00:01:26 +01:00
Fintan Halpenny b6e1385e39 httpd: improve error message for missing browser
If `open` or `xdg-open` fail the error is:

    No such file or directory (os error 2)

This generally means that the command could not open the browser on
the OS. To provide better help, improve the error message by informing
the user what went wrong and that they can run `rad web --no-open`
instead, while still printing the URL to visit.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:58:34 +01:00
Fintan Halpenny 3bba7d1634 cli: test rad patch delete
No tests for `rad patch delete` existed.

Add a test where two peers interact on a patch and one decides to
delete the patch. It ensures that the patch is still visible to the
peer that did not delete the patch.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:43:19 +01:00
Fintan Halpenny 73c120d8d4 node: do not delete cob in cache after fetch
If a peer deletes a COB, it does not mean that the COB should be
deleted. This was incorrectly assumed in the `radicle-node` fetch
code.

To fix this, when a `RefUpdate::Deleted` is seen after a fetch, the
cache is updated with the new state of the COB.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:43:19 +01:00
Fintan Halpenny 96c65eb2b3 cli: add replies to patch show output
If any replies to patches were made, it was not possible to see them
in the output of `rad patch show`.

Add the replies to show output, but only if the `--verbose` flag is
set.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:43:19 +01:00
Fintan Halpenny 3a2e06e498 cli: document --reply-to option
The option to reply to a comment was undocumented in the `rad patch
comment` command.

Add the option to the "Comment options" section of the help message.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:43:19 +01:00
Fintan Halpenny 64fd41961e nix: add radicle-full
To allow Nix users to install radicle with all tools and helpers.
2024-03-08 23:19:15 +01:00
Fintan Halpenny e1affc2da6 nix: use rust-overlay for toolchain pinning
To ensure the Nix derivations are using the correct toolchain, defined
in the `rust-toolchain` file, use the `rust-overlay` library.

Also remove the `fenix` input, since this was actually never used and
was an artefact of some copy-paste shenanigans.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:19:15 +01:00
Fintan Halpenny 9e745b68d1 cli: add caching commands for patch and issue
In order to bootstrap the COB cache, and also in case of issues,
supply a command for caching a set of patches/issues, or a single
patch/issue.

This is necessary since reads are not read-thru, it may appear that
COBs are missing when in fact they haven't been cached when first
moving over to the cache.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 16:03:26 +01:00