Commit Graph
100 Commits
Author SHA1 Message Date
cloudhead 5f1657484f build: Add "upload" build step 2024-04-29 10:47:03 +02:00
cloudhead a3ffe51ddf Rename RAD_SEED to RAD_KEYGEN_SEED
More explicit.
2024-04-26 10:38:54 +02:00
cloudhead fe5757d122 Tidy up environment variables
* Try to use constants instead of strings
* Move crypto seed override out of radicle-crypto
* Use GIT_COMMITTER_DATE for commit overrides
* Use RAD_LOCAL_TIME as general time override
* Allow variables to be used in release mode
2024-04-26 10:38:54 +02:00
cloudhead deeb39c558 node: Support "transparent" Tor
This allows for configuring your node such that `.onion` addresses
encountered are treated as regular DNS names.
2024-04-26 10:37:41 +02:00
cloudhead 70d2e1a0db node: Improve Timestamp conversion safety
Remove `Timestamp::from(u64)` instance, since not all `u64`s are valid
Timestamps, and add `try_from` instead.
2024-04-26 10:33:34 +02:00
cloudhead cd2884a609 build: Adjust folder structure
This allows for simply extracting into $PATH.
2024-04-25 17:15:58 +02:00
cloudhead 85b321f943 cli: Use correct color for private repos 2024-04-24 21:11:26 +02:00
cloudhead c607619683 node: Don't relay old messages
Avoids spamming peers on startup.
2024-04-24 13:25:58 +02:00
cloudhead 2b771921d5 node: Type-safe timestamps
Use a struct to represent timestamps, to improve type safety.
2024-04-24 12:19:40 +02:00
cloudhead 0fcfbb3ecf node: Don't re-use timestamps
There are rare cases where ref announcements may be sent in close
succession, and the current clock time is re-used. This will cause the
second (newer) announcement to be ignored by peers.

We add a `timestamp` method that checks if the timestamp is re-used, and
increments it if so, ensuring that we always use a fresh one.
2024-04-24 12:19:40 +02:00
cloudhead c07522365a cargo: Run cargo update on workspace 2024-04-23 17:43:21 +02:00
cloudhead 9fe5a8922f build: Update to zig 0.12.0, add rad-web 2024-04-23 10:59:33 +02:00
cloudhead cdebbe5bb7 node: Improved message logging 2024-04-23 10:41:43 +02:00
cloudhead d77b5d5abe node: Add 'disconnect' command to handle 2024-04-23 09:52:25 +02:00
cloudhead 65aeebb2d1 docs: Update manual pages to 1.0.0 2024-04-22 14:37:19 +02:00
cloudhead 7a1ba7d4b1 node: Add support for .onion addresses in wire
Adds support for encoding and decoding onion addresses in the gossip
protocol.
2024-04-22 12:16:17 +02:00
cloudhead ae709b00ed cli: Enable HTML syntax highlighting in diffs 2024-04-20 15:59:20 +02:00
cloudhead 95b5191533 Reproducible cross-compiled builds
Implement a new build pipeline using `podman` and `zig` that is
reproducible and can be run entirely on linux.

We also simplify the versioning system, defaulting to the output of `git
describe` when there are no exact tag matches.
2024-04-20 11:11:00 +02:00
cloudhead 0b5fa51a92 helper: Allow pushing a raw SHA-1 2024-04-18 17:09:45 +02:00
cloudhead 4276a70eb1 node: Event on local refs announcement
This allows us to listen on the socket for when the user announces new
refs.
2024-04-18 10:57:55 +02:00
cloudhead e5d8563cca node: Update routing table on fetch
Since a user can force a fetch of a private repository without any
announcement being involved, we update the routing table if the fetch
completes successfully, with the remote node.
2024-04-18 10:51:23 +02:00
cloudhead f17df88229 term: Update inquire crate 2024-04-15 17:12:36 +02:00
cloudhead 53ecc88437 node: Always notify the service of a fetch result
Even if the peer is no longer connected.
2024-04-09 16:00:49 +02:00
cloudhead 395d317954 node: Keep track of data sent/received in fetch
Keep track and log the Git data sent/received.
2024-04-09 15:59:55 +02:00
cloudhead e3ecf4d7a0 cli: Add partial clone fail test
Ensure `rad clone` is successful as long as one fetch succeeds.
2024-04-09 15:54:08 +02:00
cloudhead 2eb665678e cli: rad sync UX improvements
* Add `--force` flag to force a fetch even if the node is not a seed.
* Don't fail if we can't connect to a node during a fetch.
* More accurately exit when we hit our replica count.
2024-04-09 10:43:35 +02:00
cloudhead e9b79fb6d0 cli: Improve UX for cloning/fetching private repos
When `--private` is passed to `clone`, we force a fetch from nodes that
aren't known seeds.

When it isn't specified, we skip nodes that aren't known seeds, even if
they are in the preferred seeds list. Previously, we would still fetch
from them, which would cause errors.
2024-04-08 22:15:56 +02:00
cloudhead ef1ed621d7 node: Don't re-queue fetch that is already queued 2024-04-08 17:15:57 +02:00
cloudhead e19c773a86 fetch: Improve logging of clone 2024-04-08 17:06:38 +02:00
cloudhead 976b58ba7a node: Simplify and fix fetch dequeueing
There was redundant code and a missing `break`.
2024-04-08 12:48:49 +02:00
cloudhead bd8e0ebcda cli: Add tool in case sqlite3 is unavailable
Adds the `rad node db exec <query>` command.

Easy way to run sqlite queries if the sqlite3 CLI is not available or
too old.
2024-04-06 17:55:59 +02:00
cloudhead ad7ba82e6a node: Populate refs db on startup
If the refs database is empty, populate it on startup.
Also includes a script to clear it.
2024-04-05 21:59:58 +02:00
cloudhead 6be77ca9c3 radicle: Implement remote_refs_at
Implement this function on `RemoteRepository`, and move `repositories`
to the trait level.

This function will be used to quickly get the `RefsAt` of all repository
remotes.
2024-04-05 21:59:28 +02:00
cloudhead 7834dccd2c node: Account for skipped refs when announcing 2024-04-05 17:40:53 +02:00
cloudhead bc247dff24 node: Skip fetch early if no refs in announcement 2024-04-05 12:17:17 +02:00
cloudhead e55c86f1ce node: Improve fetch logging 2024-04-04 18:41:53 +02:00
cloudhead 6ff4efebed node: Remove namespaces from fetch request
It was unused..
2024-04-04 18:22:19 +02:00
cloudhead f2fe0242e1 node: Always check refs status before fetching
We were only checking it on dequeue.

Also simplifies some of the logic by centralizing the check for fetching
our own refs.
2024-04-04 18:10:49 +02:00
cloudhead 36808234a6 cli: Don't allow announcing without a fork 2024-04-04 14:13:34 +02:00
cloudhead 51e64cfa8b node: Fix logging for skipped refs
We were using the wrong condition.
2024-04-04 12:24:49 +02:00
cloudhead a5212a7a88 term: Fix editor command with args
We add support for `EDITOR` being set to a command with arguments, eg.
`code --wait`.

Fixes dd6861f.
2024-04-04 10:52:36 +02:00
cloudhead 7e13e0759f node: Simplify handling of fetch completion
* Don't branch based on whether it was a user-requested fetch or not.
* Only announce inventory on clones of public repos.
2024-04-03 14:25:02 +02:00
cloudhead fd4f5fff7d node: Don't fetch if only our own remote
Skip the fetch from announcement if the announcement only contains our
own remote.
2024-04-03 14:25:02 +02:00
cloudhead 6744ffc325 node: Improvements to test reliability 2024-04-03 14:25:02 +02:00
cloudhead e90c6a49bc node: Include identity doc in fetch result
This avoids having to reload the document in the service, which can be
an expensive operation.
2024-04-03 14:25:02 +02:00
cloudhead 991505ec9d radicle: Switch error type for repository methods
This simplifies error handling for the next commit.
2024-04-03 14:25:02 +02:00
cloudhead 2b9a014243 node: Small changes and note around initial state 2024-04-01 16:32:30 +02:00
cloudhead 9a60d4ad7f node: Put new refs table in new migration 2024-04-01 15:00:13 +02:00
cloudhead 256c620d31 node: Use refs cache to decide on fetching
When deciding whether or not to fetch a repo based on a refs
announcement, we need to know whether our signed refs are stale or not,
compared to the ones in the announcement. This can be quite expensive as
it requires multiple reads from Git's ODB, which often results in
packfile loading.

To remedy this, we introduce a refs cache in our node database that is
consulted when an announcement is received.

We also remove the logic that generates `info` messages, since they are
rarely needed and introduced more complexity.

Additionally, we revert commit 3ad2b4431f,
which introduced logic to minimze wants and haves with additional logic,
since it potentially also increases the load on the ODB.
2024-03-29 16:00:33 +01:00
cloudhead 345fa57b64 ci: Fix release workflow
By default, the `checkout@v4` action does a shallow clone, so we weren't
getting the history.
2024-03-27 13:51:38 +01:00
cloudhead 081af03362 Release 1.0.0-rc.1 2024-03-26 16:54:47 +01:00
cloudhead aaea06dd98 term: Fix spinner output on certain platforms 2024-03-26 13:13:54 +01:00
cloudhead f01aac8828 cli: Add inbox to help output 2024-03-26 13:13:14 +01:00
cloudhead 9cdf0aa1fd Update radicle crates to 0.9.0 2024-03-26 11:56:27 +01:00
cloudhead 77076af23c radicle: Add new ash.radicle.garden seed node
Add the new `ash` node as a preferred seed, to load balance.
2024-03-26 10:36:31 +01:00
cloudhead 24bf2826cc Update build script for 1.77
Remove the `rerun-if-changed` since it was causing problems with test
speed. By default, it will re-run if any file changes, which should be
fine.
2024-03-26 10:35:27 +01:00
cloudhead 5b5a6842c1 Update toolchain to Rust 1.77 2024-03-26 10:03:11 +01:00
cloudhead e40e642e76 scripts: Add 'tag' script for releases 2024-03-26 09:50:32 +01:00
cloudhead 7b1847d8ad Make sure user-supplied timeout takes precedence 2024-03-25 14:03:08 +01:00
cloudhead f7737b73c9 Update CONTRIBUTING.md 2024-03-25 13:39:56 +01:00
cloudhead 0bcec941ee node: Change behavior on conflicting connections
Instead of dropping the existing connection(s), we drop the same
connection on both sides, by using a simple rule that yields the same
result whether the node sees the connection as inbound or outbound.

The trick is to use something that both nodes agree on, for instance
whos public key is greater than the other, and use that information
to close the same connection on both sides.
2024-03-25 12:48:27 +01:00
cloudhead e695d06bbb node: Logging improvements & other small things 2024-03-25 12:05:05 +01:00
cloudhead dbf47fe4e9 Implement new versioning system
See `VERSIONING.md` for details.
2024-03-25 11:54:08 +01:00
cloudhead ada492f699 node: Improve logging more 2024-03-22 18:00:48 +01:00
cloudhead a4928f50b6 node: Combine event with sync status change
Use the same logic to check whether the sync status was updated, to emit
the `RefsSynced` event. That way they are at least consistent.
2024-03-22 17:07:56 +01:00
cloudhead 474df7fc3c node: Update logging config
Promote message logging to `debug` level, and change default level to
`info`.

This is because `trace` level often covers details at a different level
of detail than for eg. messages sent and received. This means we don't
have to use `trace` level just to see the gossip.
2024-03-22 16:42:35 +01:00
cloudhead 33be2a62da cli: Improve error when radicle-node isn't found 2024-03-22 16:41:51 +01:00
cloudhead 25c7162760 man: Update rad-patch with new sync options 2024-03-22 16:35:06 +01:00
cloudhead 98b0d55fde cli: Include all nodes we can announce to on sync
Previously, we only kept tracked of unsynced nodes we were connected to.
Now we track all unsynced nodes, since announcements are relayed.

We also cap the replicas to exit earlier in case of very low replica
count.
2024-03-22 16:35:06 +01:00
cloudhead 3535170f2d cli: Improve errors and code for syncing
We distinguish between the cases:

a) There are no seeds at all.
b) There are no seeds to announce to despite the user wanting a higher
   replication factor, for example.
2024-03-22 16:35:06 +01:00
cloudhead ff79d15bb8 cli: Change syncing rule for configured seeds
Instead of waiting for *all* configured/preferred seeds to be synced,
we wait for at least one.

Note that the previous behavior can still be used by setting
`--replicas` appropriately.
2024-03-22 16:35:06 +01:00
cloudhead e6728db15a cli: Unify all syncing code
We were using similar but slightly different syncing code in the
remote-helper, issues/patches and `rad sync`. This commit unifies them.
2024-03-22 16:35:06 +01:00
cloudhead 8f89f088c6 cli: Correctly honor sync timeout 2024-03-22 16:35:06 +01:00
cloudhead 38a76a42de radicle: Don't update sync status redundantly
We were updating it even when the `head` was the same.
2024-03-22 16:35:06 +01:00
cloudhead 9f4227d378 term: Improve errors when editor can't be spawned 2024-03-22 14:09:27 +01:00
cloudhead 6288dd0a24 radicle: Load inventory lazily
Instead of loading the inventory cache on storage open, load it
when the `inventory` function is called and the cache is `None`.
2024-03-21 18:03:18 +01:00
cloudhead 057af40ea3 radicle: Fix config loading with bad alias
Previously, a config would load fine with an invalid alias,
due to how serde is configured for the `Alias` type. This fixes it.
2024-03-21 14:44:04 +01:00
cloudhead 04a4aa06c9 httpd: Fix broken test 2024-03-21 12:43:35 +01:00
cloudhead f1f1313b6d node: Update netservices to 0.8.0
The previous version was relying on a broken method from `socket2` that
switched non-blocking sockets to blocking.
2024-03-21 12:40:58 +01:00
cloudhead d771ce8522 node: Make Storage::refresh private 2024-03-21 12:40:58 +01:00
cloudhead 2bcb03b021 node: Re-think inventory update code
Instead of re-loading the full inventory on `rad init`, we simply add
the new repository to the in-memory cache.

We also keep track of whether a fetch was a full clone, to know when
to update our routing table.
2024-03-21 12:40:58 +01:00
cloudhead 0e880e12e6 node: Cache node inventory
For large seeds with many repos, it's expensive to traverse the
inventory everytime the node connects to a new peer, for example.

To improve the situation, we cache the inventory per `Storage` instance,
and add a `refresh` method to refresh the inventory.

This method is called when the node is told to synchronize its
inventory.
2024-03-21 12:40:58 +01:00
cloudhead 9cd1b0ec46 cli: Improve missing man page handling
Instead of saying "no manual entry", we output regular help.
2024-03-17 21:14:58 +01:00
cloudhead cf8f262cb2 man: Add rad-id.1 man page 2024-03-17 21:14:58 +01:00
cloudhead 74abd789c7 httpd: Fix private repo visibility
Make private repos invisible on all routes, including /raw endpoints
and Git clones.
2024-03-17 21:13:25 +01:00
cloudhead 62fa8c25c8 scripts: Delete old install script 2024-03-15 16:53:09 +01:00
cloudhead 15efae2a74 docs: Use POSIX-compliant install command 2024-03-15 10:36:22 +01:00
cloudhead 752656fbfa cli: Fix output of rad auth
The `rad help` command was not the right one.
2024-03-15 10:20:28 +01:00
cloudhead 54aacc9619 ci: Add version file to binary release
This commit also upgrades deprecated actions.
2024-03-13 12:36:03 +01:00
cloudhead d7c88f12ab radicle: Fix some doc comment issues 2024-03-12 16:13:08 +01:00
cloudhead de0b9ae44d cli: Don't highlight unknown languages
Previously, we'd show "Empty file".
2024-03-12 16:12:16 +01:00
cloudhead eb1ae13e68 cli: Use correct styling for revision id 2024-03-12 15:46:55 +01:00
cloudhead 0f0f9ff9b4 helper: Check for missing canonical head
Without the canonical head in the working copy, the next check will fail
with a cryptic error.
2024-03-12 12:56:45 +01:00
cloudhead 93ff59ae25 cli: Add rad version --json flag
This will be used to automatically populated metadata around releases
for the homepage.
2024-03-11 17:36:31 +01:00
cloudhead fa4b929a06 ci: Remove old deploy script, cleanup build 2024-03-11 16:54:52 +01:00
cloudhead 3d7ae89a83 httpd: Fix tests 2024-03-11 12:24:30 +01:00
cloudhead b917c1b411 cli: Rename rad ls --seeds flag
Rename to `--seeded` as per our discussion in Zulip.
2024-03-11 10:46:09 +01:00
cloudhead d1e2e3b61e cli: Fix some tests 2024-03-09 14:01:30 +01:00
cloudhead a48081f271 cli: Move node addresses command
To make things a little more consistent, move the `rad self --address`
command to `rad node config --addresses`, since it's more about
externally-facing configuration of the node, rather than your local user
or profile.
2024-02-26 14:23:13 +01:00
cloudhead 091f7b7e98 node: Don't limit LAN addresses
These are defacto trusted.
2024-02-26 13:42:06 +01:00