Commit Graph
40 Commits
Author SHA1 Message Date
cloudhead 09a284f09c node: Make fetch timeout configurable
When a fetch is user-requested, a timeout can be supplied that is passed
down to the worker.

When a fetch is service-requested, a default timeout is used.
2023-09-01 12:13:45 +02:00
cloudhead 8cb8398be9 Introduce cargo-deny
Add `deny.toml` with a few exceptions, and update certain dependencies
to get rid of duplicates.

Running `cargo deny check` will yield some warnings still, but no
errors.
2023-08-22 14:33:00 +02:00
Alexis Sellier f639192dc6 cob: Remove CRDTs from COB state
It turns out that the CRDT formed by the union of Git DAGs
is enough to guarantee everything we need for COBs.

This changes the following things:
* COB operations no longer need to be commutative
* COB histories are traversed in the same deterministic order on all
  replicas
* It's now possible to implement RSMs on top of COBs, eg. scripting
* Lamport clocks have been removed
* `radicle-crdt` is no longer a dependency of `radicle`
* COBs are no longer instances of `Semilattice`
* The `Ops` type was removed in favor of having `Op` contain multiple
  actions
2023-07-25 15:29:27 +02:00
Alexis Sellier 1a1e63d998 node: Implement sessions command
Returns session information through the control socket.

We also update `rad node status` to display sessions.
2023-07-04 00:26:31 +02:00
Alexis Sellier 1d9ee81b6d node: Rename reactor module to io
The name was always confusing and didn't mean much in the contetx of the
service.
2023-05-08 19:31:20 +02:00
Alexis Sellier ca0abc4356 node: Keep track of connection attempts
When attempting to connect to a peer, and when succeeding, write the timestamp
to the database.

This will allow us to better select addresses and not retry peers to
soon.
2023-05-05 17:23:52 +02:00
Alexis Sellier 94bef61944 node: Return only remotes that were fetched from
Instead of returning the namespaces that we fetched with,
we return the remotes that were actually fetched from.

This can differ if some trusted peers were not available
on the remote node.

Previously, if that was the case, it would cause errors
since the remotes were looked up and that lookup failed.
2023-04-17 18:58:56 +02:00
Alexis Sellier 0e7ffcc916 node: Properly validate signed refs on fetch
With this change, we not only verify the signed refs before
the repository "transfer", but also validate that all refs in the
repo are signed and there is no discrepancy with the signed refs
file.
2023-04-15 00:17:17 +02:00
Alexis Sellier 89b9eb53b7 node: Implement protocol multiplexing
To improve the reliability and flexibility of the protocol, we introduce
multiplexing over peer connections. This involves a new `Frame` type
that carries a stream-id and payload.

Three stream types are made available:

1. Control
2. Gossip
3. Git

This change brings the following improvements:

* Removed need to queue fetch requests
* Removed need to queue gossip messages
* Removed need for `Fetch` and `FetchOk` messages
* Service doesn't need to know about inbound fetches
* Removed one round-trip for fetch negotiation
* Removed special `done` git packet
* Removed session logic and state around Git/Fetch protocols
* Removed code around upgrading/downgrading transport
* Worker in responder mode is able to process any number of fetches
* Connections support any number of concurrent fetches

We had to introduce a few extra things however to make it all work:

* A `VarInt` type for variable-length integers, since we want the frames
  to be lightweight.
* A custom "tunnel" implementation, since we couldn't use the existing
  one anymore.

Overall the change removes more complexity than it adds, while improving
the protocol along the way.
2023-04-10 14:41:55 +02:00
Fintan Halpenny c618c3e443 node: remove Namespaces::One
The One variant caused many paint points for fetching logic, where it
was not necessary. It was only constructed in one place, which could
be replaced by using the variant that holds a set of keys.

Remove the Namespaces::One variant and rename Namespaces::Many to
Namespaces::Trusted.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-03-31 12:23:32 +01:00
Fintan Halpenny adf6312a55 node: fetch trusted peers on clone
When a repository does not yet exist during a fetch, i.e. a clone,
then only delegates are being fetched.

Augment Namespaces::Many to hold the trusted and optional delegate peers
separately. Doing so allows the construction of the variant without
the repository existing, but tracking relationships existing.

This variant can then be used to build refspecs for both the trusted
peers and delegates when doing a cloning fetch.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-03-31 12:23:31 +01:00
Alexis Sellier d3f4189324 node: Fix reconnection logic
First, we make sure to always attempt reconnection to persistent peers,
no matter the reason.

Second, we don't re-attempt connections when an inbound peer
disconnects. This minimizes the chance of connections crossing.

Third, we clean up `session::Error`, and don't use it to signal a
missing session, since it doesn't really belong there.
2023-03-24 13:20:34 +01:00
xphoniex a49eec9892 node: Add eventing system to Handler and Service
Signed-off-by: xphoniex <dj.2dixx@gmail.com>
2023-03-16 22:26:57 +01:00
Fintan Halpenny e937cdd79a node: improve fetch
The fetch implementation will potentially fetch the local operator's
remote, which is unnecessary and could result in deleted data.

Fix this by threading the local operator's public key and add an
ignore refspec to the fetch arguments, i.e.

    ^refs/namespaces/<local>/*

The implementation also strictly fetches all or one namespace. This is
improved upon by introducing a new variant for specifying a set of
namespaces.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-03-08 11:29:34 +00:00
Fintan Halpenny e408b0d62c node: use enum for fetch direction
A fetch is a client-server interaction and so there are two sides to
this interaction. These sides are generally referred to as receive --
the client side -- and upload -- the server side.

The `Fetch` type indicated whether it was doing a receive or upload by
using a field `initiated: bool`. However, this can be confusing when
you pair it with the `namespaces: Namespaces` field, since only the
receive side should be sending what refspecs it wants and the upload
side responds with those matching refspecs.

A better way to represent this is using an enum for which direction
the fetch is being considered, where the receive side contains the
Namespaces. To borrow from Noise, the variant names are 'Initiator'
for the client and 'Responder' for the server.

The resulting code then makes decisions based on what variant of the
enum was passed.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-03-08 10:12:54 +00:00
Alexis Sellier 950fae209d Make ReadStorage more usable
Previously, we were not able to access a `ReadRepository` instance
through `ReadStorage`. With this change, we are able to.
2023-02-15 22:08:20 +01:00
Alexis Sellier 3e7761740f node: Rely on inventory announcement in rad init 2023-02-13 09:59:44 +01:00
Alexis Sellier 2f1a18470e node: Fix filter construction and policy code 2023-02-12 17:07:57 +01:00
Alexis Sellier 5f4c6edee4 node: Simulator logging improvements 2023-02-12 16:48:46 +01:00
Alexis Sellier f66ecd4ece node: Replication fixes
* Read git request line before starting daemon, so that we don't leave
  git data in the socket read buffer in case of an error connecting to
  the daemon.
* Send a response on the fetch channel even if we disconnect the remote.
* Don't send `done` packet if there's an error; it won't be expected.
2023-02-07 21:04:13 +01:00
Alexis Sellier 45e31a5192 Communicate fetch results through control socket
This allows a CLI to report to the user what fetch requests were
fulfilled, and which ones failed.
2023-02-07 10:46:12 +01:00
Fintan Halpenny 3f48c2c516 Rust 1.67
Rust 1.67 was announced[0]. Update the necessary files for running
1.67 and fix the clippy suggestions for string interpolation.

[0]: https://blog.rust-lang.org/2023/01/26/Rust-1.67.0.html

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-02-04 19:30:29 +01:00
Alexis Sellier 498a5c4e4a node: Handle fetched differently if initiated 2023-01-30 13:22:33 +01:00
Alexis Sellier a0910f7593 Cleanup old fetch code
We only use the old code in the simulator now.
2023-01-30 12:26:29 +01:00
Alexis Sellier fea14ff7b3 Update clone functionality for new replication 2023-01-25 10:21:57 +01:00
Alexis Sellier ef950d515a node: Fully integrate worker fetch into service 2023-01-25 10:21:57 +01:00
Alexis Sellier 3bc07195f5 Migrate to localtime from nakamoto
The only thing we needed nakamoto for was `LocalTime`.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-18 18:03:11 +01:00
Alexis SellierandDr. Maxim Orlovsky 63e7248847 node: Complete Noise handshake integration
Co-authored-by: Dr. Maxim Orlovsky <dr.orlovsky@gmail.com>
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Dr. Maxim Orlovsky 08334f790b node: Get rid of nakamoto::DisconnectReason
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-10 16:35:40 +01:00
Dr. Maxim Orlovsky a7b1c19746 node: Implement worker thread handover
Also implements listener behavior.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2023-01-05 15:16:38 +01:00
Dr Maxim Orlovsky aa02c52a2f node: Integrate new Noise_XK transport
This is a large change that introduces transport encryption via
a Noise_XK handshake. The inner `Service` now works directly
with `NodeId` and doesn't need to interact with socket addresses.

The transport logic takes place in `wire::transport`, and the
established sessions are passed down to `Service`.

Note that the i/o implementation in the `poll-reactor` crate still
needs testing, and there is still work to be done to integrate the
Git transport as part of this architecture.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-13 14:59:06 +01:00
Dr. Maxim Orlovsky 983b2dc534 Two-stage connectivity with handshake 2022-11-21 16:05:53 +01:00
Alexis Sellier f05a040be6 Make track functions return correctly
Previously we always returned `true`. Now we return
what the node actually returns.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-09 13:29:54 +01:00
Alexis Sellier 45c7f3049b Update to Rust 1.65
* Use new let-else syntax
* Fix new clippy lints

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-05 12:47:26 +01:00
Alexis Sellier 09c438f8fb Remove Envelope type
This type that was wrapping `Message` isn't very useful here.
If we decide that we do need something like that, it's best handled one
layer below.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 16:45:10 +02:00
Alexis Sellier ae95b229f5 Activate lint for dead-code checking
For test code, we need it in a couple of places.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:03:34 +02:00
Alexis Sellier d9517cc966 node: add more tests for refs relay
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-14 18:51:53 +02:00
Alexis Sellier 832c6ad53f Improve error handling in a few places
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-10 11:29:22 +02:00
Alexis Sellier fcb1007f9d Refactor service
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-09-23 15:43:55 +02:00
Alexis Sellier d1ccf1b807 node: Rename folder to radicle-node
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-09-21 11:38:09 +02:00