mirror of
https://seed.radicle.dev/rad:z3trNYnLWS11cJWC6BbxDs5niGo82.git
synced 2026-10-18 19:10:39 +08:00
treewide: Convert existing RIPs to AsciiDoc
Convert the existing RIPs from markdown to AsciiDoc. Post conversion steps taken were: - Re-order headers, reducing each level by 1 - Remove the hand crafted Table of Contents in favour of `:toc:` - Change each from `Draft` to `Final` - Add the type `Standard` to each
This commit is contained in:
Fintan Halpenny
committed by
Lorenz Leutgeb
Lorenz Leutgeb
parent
67c39d4c11
commit
06915c595e
@@ -0,0 +1,658 @@
|
||||
= RIP-0001: Heartwood
|
||||
@cloudhead <cloudhead@radicle.xyz>
|
||||
:rip: 0001
|
||||
:status: Final
|
||||
:type: Standard
|
||||
:created-at: 2022-09-06
|
||||
:updated-at: 2022-09-06
|
||||
:copyright: CC0-1.0
|
||||
|
||||
include::_metadata.adoc[]
|
||||
|
||||
:toc:
|
||||
|
||||
In this RIP, we define a major iteration of the Radicle network protocol
|
||||
and the various related sub-systems. We call it "`Heartwood`".
|
||||
|
||||
The intent of this proposal is not to define a complete specification of
|
||||
the protocol, but to be a foundation for subsequent RIPs. Various
|
||||
aspects of the protocol, in particular around the issues of privacy,
|
||||
censorship resistance, peer misbehavior and DoS are left for future RIPs
|
||||
to expand on, and won’t be tackled here. Additionally, details and
|
||||
specifics on the wire protocol, message formats, hash functions and
|
||||
encodings may be left out, to focus this proposal on the big ideas.
|
||||
|
||||
== Overview
|
||||
|
||||
The Radicle network protocol can be defined through the intended
|
||||
use-case for a peer-to-peer code hosting network:
|
||||
|
||||
_Alice publishes a repository on the network under a unique identifier,
|
||||
and Bob, using that identifier is able to retrieve it from the network
|
||||
whether Alice is online or offline, and verify the authenticity of the
|
||||
data._
|
||||
|
||||
The above must hold true independent of the network topology and number
|
||||
of nodes hosting the project, as long as at least one node is hosting
|
||||
it. We can therefore say that the primary function of the protocol is to
|
||||
locate repositories on the network, and serve them to users, all in a
|
||||
timely, and resource-efficient manner.
|
||||
|
||||
This functionality can be broken down into three components:
|
||||
|
||||
[arabic]
|
||||
. Locating repositories, ie. finding which nodes host a given repository
|
||||
. Replicating a given repository between two nodes, such that they both
|
||||
hold a copy
|
||||
. Verifying the authenticity of all the data retrieved from the network,
|
||||
such that any node can serve any data without the need for trust
|
||||
|
||||
To achieve (1), nodes need to exchange information about which
|
||||
repositories they host, so that they can point users to the right
|
||||
locations, as well as notify each other when there are updates to the
|
||||
repositories. This in turn requires _peer_ discovery: nodes need a way
|
||||
to find each other on the network.
|
||||
|
||||
To achieve (2), git is used for its excellence as a replication
|
||||
protocol.
|
||||
|
||||
To achieve (3), given the nature of peer-to-peer networks, ie. that any
|
||||
node can join the network, git alone is not enough. Replication through
|
||||
git needs to be paired with a way of verifying the authenticity of the
|
||||
data being replicated. While git checks for data _integrity_, our
|
||||
protocol will have to make sure that the data Bob downloads is the data
|
||||
Alice published on the network. Without such verification, an
|
||||
intermediary node on the network could easily tamper with the data
|
||||
before serving it to Bob. We also can’t require that Alice serve her
|
||||
data directly to Bob, as that would require them to be online at the
|
||||
same time, and would introduce a single point of failure.
|
||||
|
||||
== Repository Identity
|
||||
|
||||
To locate, or even "`talk`" about repositories on a peer-to-peer
|
||||
network, we require a stable, unique identifier that can be verifiably
|
||||
associated with a repository. Without this, there is no way for a user
|
||||
to request a specific repository and verify its authenticity. Unlike
|
||||
centralized forges such as GitHub, where repositories are deemed
|
||||
authentic based on their _location_, eg.
|
||||
`https://github.com/bitcoin/bitcoin`; in an _untrusted_ network,
|
||||
location is not enough and we need a way to automatically verify the
|
||||
data we get from any given location. Therefore, before we talk about
|
||||
networking, we must make a little detour into repository identity.
|
||||
|
||||
It’s important to understand that although git repositories use content
|
||||
addressing for their objects, repositories are _mutable_
|
||||
data-structures. Therefore, the identity of a repository _cannot_ be
|
||||
derived solely from its contents. Instead, the identity must be
|
||||
determined by some other authority. In Radicle, this is no other than
|
||||
the _maintainers_ of the repository, since it is their mandate to decide
|
||||
what gets merged into a codebase.
|
||||
|
||||
We can then define a repository’s identity as the set of all branches
|
||||
and tags that the maintainers of the repository agreed upon at a given
|
||||
point in time, along with a unique identifier.
|
||||
|
||||
For anyone to be able to verify an identity, we require maintainers to
|
||||
provide a cryptographic signature over the repository’s heads, tags, and
|
||||
other relevant git references, along with repository metadata such as
|
||||
name and description. We call this the _signed refs_. Signed refs can be
|
||||
updated whenever there are changes to a repository that are accepted by
|
||||
maintainers. They represent a repository’s _canonical state_.
|
||||
|
||||
As for the identifier, it must be provably associated with the above
|
||||
state. In other words, it must be possible, given an identifier and a
|
||||
set of signed refs, to prove association.
|
||||
|
||||
== The Identity Document
|
||||
|
||||
Before a repository can be published to the network, it needs to be
|
||||
initialized into a Radicle _project_. A project is simply a repository
|
||||
with an associated _identity document_. In this document, the public
|
||||
keys of the repository’s current maintainers are stored. When a project
|
||||
is initialized from an existing git repository for the first time, the
|
||||
user initializing becomes the de-facto initial maintainer of the
|
||||
project, and his key is included in the new identity document’s key
|
||||
ring. We call this set of trusted keys the project’s _delegation_, and
|
||||
each key is called a _delegate_. Though these will often map one to one
|
||||
with maintainers, this is not a requirement. The only requirement is
|
||||
that they be trusted to represent a given project, as they will be used
|
||||
to determine the canonical state of the project repository.
|
||||
|
||||
From this initial identity document we can then derive a unique, stable
|
||||
identifier for the project, by hashing the document’s contents. In
|
||||
addition to the _delegation_, we include in the document a user-chosen
|
||||
_alias_ for the project, as well as a _description_. The process for
|
||||
hashing the document shall be discussed in a subsequent RIP.
|
||||
|
||||
It is by including the identity document in the _signed refs_ that we
|
||||
establish a relationship between the source code and the identity, and
|
||||
thus associate the project identifier with the project source code. Note
|
||||
that this permits identical source codes to have more than one identity.
|
||||
This is useful when a user wishes to a _fork_ a repository. In that
|
||||
case, a new project would be initialized with a brand new identifier,
|
||||
but a mostly identical source code history.
|
||||
|
||||
The storage, update and verification mechanism for the identity document
|
||||
will be discussed in more detail in a future RIP. For the purposes of
|
||||
this document, we can assume a verification mechanism that takes as
|
||||
input the project identifier, signed refs, and identity document, and
|
||||
outputs whether the project is valid or not.
|
||||
|
||||
At the networking level, all we need is a way to derive stable
|
||||
identifiers for repositories, and a verification process that asserts
|
||||
that a given repository corresponds to some project identifier.
|
||||
|
||||
== Repository Discovery
|
||||
|
||||
The Radicle network protocol has to serve one core purpose: given a
|
||||
project identifier, a user should be able to retrieve the full project
|
||||
source code associated with that identifier, and verify its
|
||||
authenticity. This function should be independent of where the project
|
||||
is located, and how many replicas exists, provided at least one replica
|
||||
exists.
|
||||
|
||||
== Topology
|
||||
|
||||
Given that there is no natural incentive for nodes to host _arbitrary_
|
||||
projects, nodes on the network should be given the choice of which
|
||||
projects to host.
|
||||
|
||||
For example,
|
||||
|
||||
* A company that uses or provides open source software may want to host
|
||||
it on their node, to ensure its continued availability on the network.
|
||||
* A business that hosts projects for a fee would need to be able to
|
||||
choose which projects it hosts.
|
||||
* A developer contributing to a project may want to self-host it on his
|
||||
node.
|
||||
|
||||
For this reason, we cannot deterministically compute on which node(s) a
|
||||
given project should be hosted, as is the case with DHTs. Nodes are able
|
||||
to choose what they host, and therefore the network is fundamentally
|
||||
_unstructured_. Some nodes may host thousands of projects, while others
|
||||
may host only one or two. Though there is a benefit to arranging the
|
||||
peer topology in a certain fashion (eg. to reduce communication
|
||||
overhead), this cannot be relied on in an untrusted network, and
|
||||
therefore we don’t make these assumptions in the basic protocol either.
|
||||
|
||||
== Routing
|
||||
|
||||
The general problem of reaching a specific node on the network is
|
||||
usually known as "`routing`". Where IP routing tries to route traffic to
|
||||
a certain IP address, in the Radicle network, we attempt to route
|
||||
requests to one or more nodes that host a given project; these are
|
||||
called _seeds_ in the context of that project. A seed is a node that
|
||||
hosts and serves one or more projects on the network.
|
||||
|
||||
Routing information is usually stored in a _routing table_ that is keyed
|
||||
by the "`target`", in our case this is the project identifier:
|
||||
|
||||
....
|
||||
RoutingTable = BTreeMap<RepositoryId, Vec<NodeId>>
|
||||
....
|
||||
|
||||
For each project, we keep track of the nodes that are known to host this
|
||||
project. Using hashes for project identifiers and IP addresses as node
|
||||
identifiers, the table might look something like:
|
||||
|
||||
....
|
||||
80a2970… 54.122.99.1, 89.2.23.67
|
||||
c5e079e… 66.12.193.8, 89.2.23.67, 12.43.212.9
|
||||
…
|
||||
....
|
||||
|
||||
To build the table, nodes gossip information about other nodes, namely
|
||||
_what_ projects are hosted _where_.
|
||||
|
||||
Assuming `32 byte` project and node identifiers, an average of `3` nodes
|
||||
hosting each project, and a million projects, all stored in a binary
|
||||
tree, we would need only about `244 MB` to store the entire routing
|
||||
table in memory with no compression:
|
||||
|
||||
....
|
||||
project count = 1'000'000
|
||||
leaf size = 32 + 32 * 3 = 128 B
|
||||
leaves size = leaf size * project count = 128'000'000 B = ~122 MB
|
||||
index size = leaf size * (project count - 1) = ~122 MB
|
||||
total = 122 MB + 122 MB = ~244 MB
|
||||
....
|
||||
|
||||
Since this amount of memory is available on commodity hardware, we see
|
||||
no need to partition the routing table for the time being, and propose
|
||||
that each node store the entire routing table on disk or in memory.
|
||||
|
||||
== Node Identity
|
||||
|
||||
The identity of a node on the network is simply the identity of the user
|
||||
operating the node. To be able to securely verify data authorship, we
|
||||
use public key cryptography, with the public key being used as the node
|
||||
identifier. In the case of nodes run by end-users; which is likely most
|
||||
nodes; the node’s secret key is used to create the _signed refs_ and
|
||||
optionally to sign git commits.
|
||||
|
||||
The use of the same identity for both network communications and code
|
||||
signing makes the network more transparent, while allowing nodes to
|
||||
trust each other based on the code they publish.
|
||||
|
||||
For nodes that are run as always-on "`servers`", the node identity may
|
||||
not be used for signing code. These _seed_ nodes only use their secret
|
||||
keys to sign gossip messages and establish secure connections.
|
||||
|
||||
== Gossip
|
||||
|
||||
We design the Radicle networking layer as a _gossip_ protocol. In this
|
||||
proposal, we go over some of the fundamental types of messages that are
|
||||
sent between peers over the network. We contend that the core
|
||||
functionality can be achieved with three message types: _inventory_
|
||||
announcements, _reference_ announcements and _node_ announcements. Each
|
||||
fulfilling a distinct role. The exact wire protocol will be described in
|
||||
a future proposal; this section should serve as a short introduction to
|
||||
the topic.
|
||||
|
||||
== Inventory Announcements
|
||||
|
||||
To build their routing table, nodes connecting to the network announce
|
||||
to their peers what inventory they have, ie. what projects they are
|
||||
seeding. These announcements are relayed to other connected peers, and
|
||||
so on until they reach the majority of nodes on the network. Messages
|
||||
that have already been seen are dropped, to prevent messages from
|
||||
propagating forever. Gossip messages may be retained by nodes for a
|
||||
certain amount of time, so that they can be served to new nodes joining
|
||||
the network. The _inventory announcement_ message has the following
|
||||
shape:
|
||||
|
||||
....
|
||||
InventoryAnnouncement = (
|
||||
NodeId,
|
||||
Vec<RepositoryId>,
|
||||
Timestamp,
|
||||
Signature,
|
||||
)
|
||||
....
|
||||
|
||||
It contains the identifier of the node making the announcement, the
|
||||
inventory of projects, a timestamp, and the signature of the node over
|
||||
the projects and timestamp. By using a public key as the `NodeId`, we
|
||||
can then both identify and verify the provenance of the message, using
|
||||
the signature.
|
||||
|
||||
In this manner, every node in the network will eventually converge
|
||||
towards a single routing table, provided the network is well connected.
|
||||
|
||||
For larger networks, where nodes cannot be fully meshed, it’s desirable
|
||||
for seeds that have projects in common to be connected to each other.
|
||||
Hence, nodes should prioritize connecting to peers that seed the same
|
||||
projects as them. This is simply because relevant messages can reach
|
||||
interested nodes more quickly and efficiently if nodes with shared
|
||||
interests are directly connected to each other; but also because nodes
|
||||
can use already-established connections to fetch data of interest.
|
||||
|
||||
As is often the case with large, unstructured networks, gossip messages
|
||||
can be received out of order. For this reason, the inventory message
|
||||
includes a _timestamp_, which is used for ordering messages. Since the
|
||||
inventory message is meant to communicate a node’s complete inventory,
|
||||
nodes can simply ignore inventory messages with timestamps lower than
|
||||
the latest received, and not relay them. To mitigate issues with
|
||||
timestamps far in the future, we reject messages with timestamps too far
|
||||
in the future.
|
||||
|
||||
=== Pruning
|
||||
|
||||
One worry with routing tables on permissionless networks is that nodes
|
||||
come and go all the time. While a project may be available on a node one
|
||||
day, the node may go offline the next day and never come back online.
|
||||
Additionally, nodes may choose to stop hosting a certain project, making
|
||||
it no longer available.
|
||||
|
||||
Hence, the routing table needs to be constantly pruned, with out-of-date
|
||||
entries evicted. To achieve this, we set an expiry on routing table
|
||||
entries, and require live nodes to "`refresh`" their entries on other
|
||||
nodes by sending `inventory` messages periodically.
|
||||
|
||||
Entries that have been in the table for more than a day without updates
|
||||
or refreshes can then be automatically evicted.
|
||||
|
||||
== Reference Announcements
|
||||
|
||||
When an update to a project is made by a user, the user’s node sends a
|
||||
message to the network, announcing the update. Nodes that are tracking
|
||||
this project are then able to fetch the updates via the _git_ protocol,
|
||||
either directly from the user’s node, or from an intermediary node. This
|
||||
_refs announcement_ message looks like this:
|
||||
|
||||
....
|
||||
RefsAnnouncement = (
|
||||
NodeId,
|
||||
RepositoryId,
|
||||
Map<RefName, CommitId>,
|
||||
Signature,
|
||||
)
|
||||
....
|
||||
|
||||
It contains the identifier of the node announcing the updated
|
||||
references, the repository under which these refs reside, the map of
|
||||
reference names (`RefName`) with their new commit hashes (`CommitId`)
|
||||
and a signature from the publishing node (`NodeId`), over the refs and
|
||||
project identifier.
|
||||
|
||||
This allows any receiving node tracking the project to verify the
|
||||
legitimacy of the message using `NodeId` and `Signature`. For new
|
||||
projects, published on the network for the first time, the same type of
|
||||
message can be used.
|
||||
|
||||
Reference announcements, unlike inventory announcement should only be
|
||||
relayed to interested nodes, ie. nodes that are hosting the given
|
||||
project, as they will usually be followed by a `git-fetch`.
|
||||
|
||||
We should also note that the `NodeId` in this case is not only the
|
||||
announcer of these updated references, but may be the _author_. When
|
||||
Alice pushes changes to a project, she announces these changes over the
|
||||
network using a reference announcement, via her own node.
|
||||
|
||||
== Node Announcements
|
||||
|
||||
We’ve touched upon inventory gossip, and how project metadata and data
|
||||
is exchanged, but not how node metadata is exchanged; or in other words,
|
||||
how _peer discovery_ is carried out. For this purpose, we devise a _node
|
||||
announcement_ message:
|
||||
|
||||
....
|
||||
NodeAnnouncement = (
|
||||
NodeId,
|
||||
NodeFeatures,
|
||||
Vec<Address>,
|
||||
Timestamp,
|
||||
Signature,
|
||||
)
|
||||
....
|
||||
|
||||
This message is designed to be authored by a node announcing _itself_ on
|
||||
the network, and therefore contains a signature and timestamp, and is
|
||||
meant to be relayed by other nodes on the network. The key "`payload`"
|
||||
of this message is the vector of addresses sent by the node. This should
|
||||
contain all addresses on which the node is publicly reachable. At
|
||||
minimum, this should contain one IP address, but in the future could
|
||||
contain `.onion` addresses or DNS names.
|
||||
|
||||
As with the inventory message, nodes should buffer these announcements
|
||||
to serve them to new nodes connecting to the network. In addition to the
|
||||
list of addresses, we propose to also include a list of features
|
||||
supported by the announcing node, to allow for future protocol upgrades.
|
||||
|
||||
=== Bootstrap Nodes
|
||||
|
||||
A node joining the network for the first time will not know of any
|
||||
peers. Hence, it’s advised that network client software be
|
||||
pre-configured with DNS "`seeds`". These are registered DNS names, eg.
|
||||
`seeds.radicle.xyz` that resolve to node addresses on the network. In
|
||||
the bootstrapping process, nodes can resolve these names to have a set
|
||||
of addresses to initially connect to, and once they find a peer, use the
|
||||
regular peer discovery process to find more nodes.
|
||||
|
||||
== Replication
|
||||
|
||||
While gossip is used to exchange _metadata_, the actual repository
|
||||
_data_, ie. Git objects are transferred via the process of
|
||||
_replication_.
|
||||
|
||||
Nodes are configured with a list of projects that they are meant to
|
||||
host. These are called _tracked_ projects, and this configuration is
|
||||
called the _tracking policy_.
|
||||
|
||||
When a new node joins the network, the first thing it will attempt to do
|
||||
is to retrieve these tracked projects from the network. This is called
|
||||
_bootstrapping_. To do this, the node consults its routing table,
|
||||
locates the project’s _seeds_, and initiates a `git-fetch` via the _git_
|
||||
protocol, with one or more seed. This fetch operation downloads the
|
||||
relevant git objects into the node’s _storage_, making them available to
|
||||
other interested nodes.
|
||||
|
||||
To notify its peers that its inventory has changed, it sends an
|
||||
_inventory_ message to each of its peers. Replication is only possible
|
||||
because of the exchange of information on the gossip layer. Without it,
|
||||
nodes wouldn’t know where to replicate projects from, and would quickly
|
||||
fall behind.
|
||||
|
||||
== Project Tracking and Branches
|
||||
|
||||
While it’s possible to always replicate and track at the _repository_
|
||||
level, it is highly impractical: such an _open_ tracking policy is
|
||||
easily abused by malicious nodes. Given limited disk space and
|
||||
bandwidth, nodes need a way to replicate only a _subset_ of repository
|
||||
data, authored by users they can trust.
|
||||
|
||||
If we want to allow contributors to publish code that is intended to be
|
||||
merged into another project, we need to think about a node’s tracking
|
||||
policy with respect to individual contributors and their git reference
|
||||
_trees_ (the set of all branches published under a project by a given
|
||||
contributor). The safest tracking policy is to only track trees
|
||||
published by the delegates of a tracked project. But this means that
|
||||
contributors (non-delegates) won’t have their changes replicated on the
|
||||
network. This becomes a problem when a contributor wants to propose a
|
||||
new patch to a project: unless the contributor is online to _seed_ that
|
||||
branch, there is no way for the maintainer to retrieve it.
|
||||
|
||||
Lacking a good answer to this problem at the protocol level, we defer to
|
||||
node operators to implement policies at the individual seed level. For
|
||||
example, a seed node may require of users to link their social profiles
|
||||
with their Radicle key before enabling tracking for their branches.
|
||||
|
||||
In the past, we’ve explored the idea of a "`social`" tracking graph. We
|
||||
leave this door open for potential future iterations of the protocol.
|
||||
|
||||
== Unintentional Forks and Conflicts
|
||||
|
||||
It is possible, through a software bug or user error to unintentionally
|
||||
fork one’s history. For example, a user publishes changes to a branch
|
||||
that land on a node in the network, but later re-writes that branch’s
|
||||
history and re-publishes it. Nodes that received the initial changes
|
||||
will not be able to merge these new changes via a simple history
|
||||
_fast-forward_, while nodes that never saw the initial changes will have
|
||||
no problem fetching the new ones.
|
||||
|
||||
....
|
||||
User Seed #1 Seed #2
|
||||
|
||||
B B
|
||||
| A A |
|
||||
| / / |
|
||||
|/ / |
|
||||
| | |
|
||||
....
|
||||
|
||||
In the above diagram, a user publishes history `A` under branch
|
||||
`master`, which lands on `Seed #1` and then publishes history `B` under
|
||||
the same branch. `Seed #2`, which didn’t have the prior history is able
|
||||
to fetch that branch without problems. However, `Seed #1` isn’t, since
|
||||
the histories are divergent.
|
||||
|
||||
Now, depending on which seed is used to fetch, a user would get a
|
||||
completely different history for that branch.
|
||||
|
||||
To solve this problem, we have to realize one simple thing: every time a
|
||||
user publishes new code to the network, a commit in the _signed refs_
|
||||
history is created. This means that after publishing `B`, the user’s
|
||||
signed refs history will look like this:
|
||||
|
||||
....
|
||||
refs/…/heads/master B
|
||||
refs/…/heads/master A
|
||||
…
|
||||
....
|
||||
|
||||
Since `Seed #1` will have this history if it fetches `B`, it will be
|
||||
able to simply set the head of the branch to the latest signed ref for
|
||||
that branch, which is `B`.
|
||||
|
||||
Forks in the signed refs history itself, though much more problematic,
|
||||
could be recovered by picking the history with the latest timestamp.
|
||||
|
||||
== Storage
|
||||
|
||||
Since storage and replication are tightly coupled, and replication makes
|
||||
use of Git, so does storage. Unlike previous versions of Radicle, each
|
||||
project is stored in its own _bare_ Git repository, under a common base
|
||||
directory.
|
||||
|
||||
Storage is accessed directly by the node to report inventory to other
|
||||
nodes, and accessed by the end user through either specialized tooling
|
||||
or `git`.
|
||||
|
||||
It’s important to know that a user working on a project will typically
|
||||
have _two_ copies of the repository: one in storage, called the _stored_
|
||||
copy, and one _working copy_. The working copy will be setup in such a
|
||||
way that it is linked to storage via a _remote_ named `rad`. Publishing
|
||||
code is then a matter of running for eg. `git push rad`. Code in storage
|
||||
can be considered _public_, since it is shared with connected peers,
|
||||
while code in the working copy is considered _private_ until pushed.
|
||||
|
||||
This allows for code to be staged for publishing even while offline,
|
||||
provided the user’s storage is accessible. In most cases, it makes sense
|
||||
to keep storage and working copies on the same machine.
|
||||
|
||||
....
|
||||
┌╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┐ ┌╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┐
|
||||
┆ ┌───────────────────┐ ┌──────┐ ┆ ┆ ┌──────┐ ┌─────────────────┐ ┆
|
||||
┆ │ Storage │ │ │ ┆ Git ┆ │ │ │ Storage │ ┆
|
||||
┆ │ ├╸┆╸╸╸╸╸╸┆╸╸╸╸╸╸╸╸╸╸╸╸╸╸┆╸╸╸╸╸╸┆╸┤ │ ┆
|
||||
┆ │ ┌──────┐ ┌─────┐ ┌│ │ │ ┆ protocol ┆ │ │ │ ┌─────┐ ┌─────┐ │ ┆
|
||||
┆ │ │repo │ │repo │ ││ │ │ ┆ ┆ │ │ │ │repo │ │repo │ │ ┆
|
||||
┆ │ ├──────┤ ├─────┤ ├│ │ │ ┆ ┆ │ │ │ ├─────┤ ├─────┤ │ ┆
|
||||
┆ └─┴───╿──┴─┴───┬─┴─┴┘ │ │ ┆ ┆ │ │ └─┴───┬─┴─┴───╿─┴─┘ ┆
|
||||
┆ │ │ │ │ ┆ gossip ┆ │ │ │ │ ┆
|
||||
┆ │ │ │ Node ├╸╸╸╸╸╸╸╸╸╸╸╸╸╸┤ Node │ │ │ ┆
|
||||
┆ │ │ │ │ ┆ protocol ┆ │ │ │ │ ┆
|
||||
┆ push pull │ │ ┆ ┆ │ │ pull push ┆
|
||||
┆ │ │ │ │ ┆ ┆ │ │ │ │ ┆
|
||||
┆ │ │ │ │ ┆ ┆ │ │ │ │ ┆
|
||||
┆ │ │ │ │ ┆ ┆ │ │ │ │ ┆
|
||||
┆ ┌────┴───┐ ┌──╽─────┐│ │ ┆ ┆ │ │ ┌─────╽──┐ ┌──┴────┐┆
|
||||
┆ │working │ │working ││ │ ┆ ┆ │ │ │working │ │working│┆
|
||||
┆ │copy │ │copy ││ │ ┆ ┆ │ │ │copy │ │copy │┆
|
||||
┆ └────────┘ └────────┘└──────┘ ┆ ┆ └──────┘ └────────┘ └───────┘┆
|
||||
└╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┘ └╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┘
|
||||
....
|
||||
|
||||
Splitting the storage into per-project repositories has numerous
|
||||
advantages over previous designs that used a "`monorepo`":
|
||||
|
||||
* Private repositories become possible to offer
|
||||
* Concurrency is simpler, since we can have locks at the project level
|
||||
* The Git object database is used more efficiently, since there is more
|
||||
sharing
|
||||
* Repository settings can be tuned on a per-project level
|
||||
|
||||
== Layout
|
||||
|
||||
Each project under radicle is stored in a bare Git repository containing
|
||||
the local user’s refs, as well as the refs of all of the peers that the
|
||||
user’s node is configured to track. A simple "`namespacing`" scheme can
|
||||
be used, similar to the one used by git for remote tracking branches.
|
||||
|
||||
Since nodes replicate Git data from other nodes, a partitioning scheme
|
||||
is needed to separate references belonging to each node within each
|
||||
project. This can be achieved by using a node’s unique identifier (its
|
||||
public key) to namespace Git references, since references are by their
|
||||
nature hierarchical.
|
||||
|
||||
=== Special References
|
||||
|
||||
To store project metadata, special Git references are used. These are
|
||||
references that are written to a known location that doesn’t vary
|
||||
between projects, and in some cases is meant to be hidden from the user,
|
||||
and accessed only through purpose-built tooling.
|
||||
|
||||
The first one is the head of the project identity branch:
|
||||
|
||||
....
|
||||
refs/rad/id
|
||||
....
|
||||
|
||||
This reference points to the latest version of the identity document.
|
||||
Users who want to make changes to their project’s identity document can
|
||||
checkout this branch.
|
||||
|
||||
The second one is the _signed refs_:
|
||||
|
||||
....
|
||||
refs/rad/sigrefs
|
||||
....
|
||||
|
||||
You’ll notice these references are not under the `heads/` hierarchy and
|
||||
therefore aren’t regular branches. These references point to commit
|
||||
histories, though these histories are disjoint from the source code’s
|
||||
history. Since they determine the outcome of project verification, they
|
||||
need to be handled with care and are not intended to be accessed
|
||||
directly by an end user.
|
||||
|
||||
Storage layout will be specified in more detail in future work.
|
||||
|
||||
== Canonicity
|
||||
|
||||
In the above examples, we limited ourselves to projects with a single
|
||||
delegate. For the majority of open source projects, this is fitting: a
|
||||
single maintainer is in charge of the code. However, larger projects
|
||||
often have more than one user with push access or commit rights. In the
|
||||
Radicle model, there are no shared branches: each branch or set of
|
||||
branches under a tree is owned by _one_ key. This is why they are
|
||||
partitioned by a public key in the repository hierarchy.
|
||||
|
||||
In a project with multiple delegates, for example, Alice, Bob and Eve,
|
||||
each would have their _own_ `master` branch which only they could write
|
||||
to, eg.:
|
||||
|
||||
....
|
||||
<alice>/refs/heads/master
|
||||
<bob>/refs/heads/master
|
||||
<eve>/refs/heads/master
|
||||
....
|
||||
|
||||
So how does a contributor know which of those branches is the canonical
|
||||
one? The protocol itself does not have a notion of canonicity. It is
|
||||
left up to social consensus: perhaps the three delegates have agreed
|
||||
that Eve’s `master` branch will be the canonical one that everyone
|
||||
should pull from. This agreement can be encoded in the project’s
|
||||
identity document, and leveraged by tooling, but it is entirely
|
||||
optional, as it may not be desirable for all projects to "`elect`" a
|
||||
delegate that way.
|
||||
|
||||
For projects that do not have a canonical `master`, there is another
|
||||
option to establish consensus on the state of a repository: _quorum_.
|
||||
Simply put, we can examine the commit histories of all three `master`
|
||||
branches, and pick the _latest_ commit included in a _majority_ of
|
||||
histories as our canonical state. For example, in the example below, the
|
||||
commit referred to by `B`, that is Bob’s latest would be used.
|
||||
|
||||
....
|
||||
Alice Bob Eve
|
||||
|
||||
C o D o
|
||||
| /
|
||||
B o B o B o <- quorum
|
||||
| | |
|
||||
A o A o A o
|
||||
| | |
|
||||
....
|
||||
|
||||
== Closing Thoughts
|
||||
|
||||
The protocols described above are part of a larger system. It’s our hope
|
||||
that subsequent RIPs will answer some of the questions that arise from
|
||||
this proposal, as well as flesh out the functionality on top of the base
|
||||
protocol. How to represent rich social interactions and collaboration
|
||||
such as code review, comments, issues and patches on top of this
|
||||
protocol, is left for future discussion.
|
||||
|
||||
== Credits
|
||||
|
||||
* Kim Altintop, for coming up with the original design this protocol is
|
||||
based on
|
||||
* Alex Good, for helping iterate on some of the ideas found in this
|
||||
proposal
|
||||
* Fintan Halpenny, for helping iterate on some of the ideas found in
|
||||
this proposal
|
||||
|
||||
== Copyright
|
||||
|
||||
This document is licensed under the Creative Commons CC0 1.0 Universal
|
||||
license.
|
||||
@@ -0,0 +1,488 @@
|
||||
= RIP-0002: Identity
|
||||
@cloudhead <cloudhead@radicle.xyz>
|
||||
:rip: 0002
|
||||
:status: Final
|
||||
:type: Standard
|
||||
:created-at: 2022-12-06
|
||||
:updated-at: 2022-12-06
|
||||
:copyright: CC0-1.0
|
||||
|
||||
include::_metadata.adoc[]
|
||||
|
||||
:toc:
|
||||
|
||||
In RIP #1, we discussed _repository identity_, and the _identity
|
||||
document_. We said that to make it possible for repositories to be
|
||||
hosted in a peer-to-peer network, Git repositories on their own are not
|
||||
enough: we need a secure way to identify repositories that goes beyond
|
||||
source code. We need a stable identifier and a mechanism for
|
||||
self-certifying repositories against this identifier, so that changes to
|
||||
source code can be verified locally, by users.
|
||||
|
||||
In this RIP, we discuss the method through which we can achieve the
|
||||
above in a secure, decentralized way.
|
||||
|
||||
== Overview
|
||||
|
||||
To introduce the topic of identity, we point the reader to the opening
|
||||
paragraphs of the original specification of identities on the Radicle
|
||||
network, which is still very much applicable to the Heartwood protocol:
|
||||
|
||||
____
|
||||
In order to collaborate on repositories within a consensus-free network,
|
||||
we must be able to refer to them using a stable identifier. Note that
|
||||
this identifier is a statement of intent: a repository can be described
|
||||
as a collection of ever-moving leaves of a DAG whose root element is the
|
||||
empty object. Therefore, the content of a repository is not enough to
|
||||
describe it – while two views on the repository may share objects, they
|
||||
may diverge substantially otherwise. Both views may however state their
|
||||
intent to eventually converge to the same state.
|
||||
|
||||
While in principle a random identifier with sufficient entropy would
|
||||
suffice for the purpose, this would put the burden of deciding which
|
||||
repository views are legitimate entirely on the user. Instead, our
|
||||
approach is to establish an ownership proof, tied to the network
|
||||
identity of a peer, or set of peers, such that repository views can be
|
||||
replicated according to the trust relationships between peers
|
||||
(“tracking”).
|
||||
|
||||
Our model is loosely based on The Update Framework
|
||||
(TUF)footnote:[https://theupdateframework.github.io/specification/latest/],
|
||||
conceived as a means of securely distributing software packages.
|
||||
____
|
||||
|
||||
With this in mind, there are three core components to the Radicle
|
||||
identity system, for any given repository:
|
||||
|
||||
[arabic]
|
||||
. A set of peers on the network, each holding a signing key.
|
||||
. A document which establishes the identity of this repository, using
|
||||
these signing keys to self-certify.
|
||||
. A stable identifier that can be used to refer to the repository,
|
||||
derived from this document.
|
||||
|
||||
== Peer Identity
|
||||
|
||||
Since Radicle repositories on the network are created by peers, we must
|
||||
first establish the concept of a _peer identity_. In Heartwood, peers
|
||||
are simply identified by their public key. This key is an
|
||||
Ed25519footnote:[https://ed25519.cr.yp.to/] key that is encoded as a DID
|
||||
using the `did:key`
|
||||
methodfootnote:[https://w3c-ccg.github.io/did-method-key/]. DIDs are
|
||||
used for interoperability with other systems as well as allowing for
|
||||
other types of identifiers in the future.
|
||||
|
||||
....
|
||||
did:key:z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
|
||||
....
|
||||
|
||||
_Example of a peer identifier in DID format._
|
||||
|
||||
We’ll also note that peers on the network – also called _nodes_ are
|
||||
indistinguishable from _users_ at the protocol level. The terms "`Node
|
||||
ID`", "`Peer ID`", "`Public Key`" are thus all used interchangeably.
|
||||
|
||||
== Repository Identity
|
||||
|
||||
With the establishment of peer identities, we can now move on to
|
||||
repository identities. A repository identity consists of an identity
|
||||
document and an associated unique identifier.
|
||||
|
||||
The identity document is a JSON document associated with a repository on
|
||||
Radicle. The _hypothetical_ minimal identity document looks like this:
|
||||
|
||||
....
|
||||
{ "delegates": ["did:key:z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi"],
|
||||
"threshold": 1 }
|
||||
....
|
||||
|
||||
It describes a repository with a single _delegate_. Delegates are
|
||||
trusted entities that can cryptographically sign data within the scope
|
||||
of a given repository. In the identity document, they are represented by
|
||||
a DID. As of this RIP, only the `did:key` method is supported.
|
||||
|
||||
Using the `threshold` property, the document specifies that only _one_
|
||||
delegate is required to sign updates to the repository. In this case,
|
||||
since we only have one delegate, this is the only possible value for
|
||||
`threshold`.
|
||||
|
||||
____
|
||||
Repository delegates are responsible for signing all updates to a
|
||||
repository, whether it be source code commits or updates to the identity
|
||||
document itself. They can be thought of as repository "`maintainers`",
|
||||
though the applicability is broader. We will see how delegates sign
|
||||
repository updates in one of the following sections.
|
||||
____
|
||||
|
||||
Though the above document could constitute a valid identity, it does not
|
||||
contain any identifiable data that may be used to describe a particular
|
||||
repository. This is what the `payload` section is for. Heartwood defines
|
||||
a single payload type, `xyz.radicle.project`, which can be used to
|
||||
describe a project stored in a repository:
|
||||
|
||||
....
|
||||
{ "delegates": ["did:key:z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi"],
|
||||
"threshold": 1,
|
||||
"payload": {
|
||||
"xyz.radicle.project": {
|
||||
"name": "heartwood",
|
||||
"description": "Radicle Heartwood Protocol & Stack",
|
||||
"defaultBranch": "master"
|
||||
}
|
||||
}
|
||||
}
|
||||
....
|
||||
|
||||
The string `xyz.radicle.project` is called a _payload ID_, and the
|
||||
`project` payload is the default payload type for Radicle repositories.
|
||||
Using this payload, type, a repository may be given a name, a
|
||||
description, and a default branch.
|
||||
|
||||
Identity documents are designed to be extensible, and developers may
|
||||
create their own payload types and applications can choose which payload
|
||||
types to support.
|
||||
|
||||
....
|
||||
{ ...
|
||||
"payload": {
|
||||
"xyz.radicle.project": { ... },
|
||||
"xyz.radicle.funding": { ... },
|
||||
"com.atproto.account": {
|
||||
"email": "eve@atproto.com",
|
||||
"handle": "eve"
|
||||
}
|
||||
}
|
||||
}
|
||||
....
|
||||
|
||||
Figure 1. Fictional example of an identity with multiple payloads
|
||||
|
||||
Payload IDs use reverse domain-name
|
||||
notationfootnote:[https://en.wikipedia.org/wiki/Reverse++_++domain++_++name++_++notation]
|
||||
and are comprised of two parts: an _authority_, eg. `radicle.xyz`, and a
|
||||
_name_, eg. `project`. To keep payload types globally unique, developers
|
||||
wishing to create new payload types must control the authority (domain)
|
||||
under which these live.
|
||||
|
||||
As of this RIP, there is only one recognized payload type:
|
||||
`xyz.radicle.project`. Repositories which include this type of payload
|
||||
are sometimes called _projects_.
|
||||
|
||||
____
|
||||
When specifying new payload types, it’s worth thinking about how the
|
||||
payload schema might evolve over time. For example, it might be worth
|
||||
versioning the payload types, either via the identifier (eg.
|
||||
`com.atproto.account.v1`) or via a field inside the payload (eg.
|
||||
`++{++"version": 1}`). This will ensure that changes to the payload
|
||||
schema are able to be made in a backwards compatible way.
|
||||
____
|
||||
|
||||
== Validation
|
||||
|
||||
An identity document is valid if the following conditions are met:
|
||||
|
||||
* There is at least _one_ `delegate`, but no more than `255`.
|
||||
* Strings are not empty, and at most `255` characters long.
|
||||
* The `threshold` is not zero and not greater than the number of
|
||||
`delegates`.
|
||||
* The items in `delegates` are valid DIDs.
|
||||
* There is a `payload` property with at least one payload object and a
|
||||
valid payload ID.
|
||||
* Each payload under `payload` is valid according to the rules of that
|
||||
payload.
|
||||
|
||||
These rules can be partly described in the following JSON
|
||||
Schemafootnote:[https://json-schema.org/] document:
|
||||
|
||||
....
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"delegates": {
|
||||
"type": "array",
|
||||
"items": [{ "type": "string" }],
|
||||
"minItems": 1,
|
||||
"maxItems": 255,
|
||||
"uniqueItems": true
|
||||
},
|
||||
"threshold": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 255
|
||||
},
|
||||
"payload": {
|
||||
"type": "object",
|
||||
"additionalProperties": { "type": "object" },
|
||||
"minProperties": 1
|
||||
}
|
||||
},
|
||||
"required": ["delegates", "threshold", "payload"]
|
||||
}
|
||||
....
|
||||
|
||||
Finally, the schema and validation rules for the `xyz.radicle.project`
|
||||
payload are described as:
|
||||
|
||||
....
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 255
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"maxLength": 255
|
||||
},
|
||||
"defaultBranch": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 255
|
||||
}
|
||||
},
|
||||
"required": ["name", "description", "defaultBranch"]
|
||||
}
|
||||
....
|
||||
|
||||
== The Repository Identifier
|
||||
|
||||
Now that we have a document describing our repository, we can derive
|
||||
from it a unique identifier that can be used to refer to the repository
|
||||
on the peer-to-peer network. This identifier must meet certain criteria:
|
||||
|
||||
[arabic]
|
||||
. It must be _stable_, in other words it must not change throughout the
|
||||
lifetime of the repository.
|
||||
. It must be deterministically derivable from the identity document
|
||||
alone, for the purpose of verification.
|
||||
. It must contain enough entropy to be globally unique.
|
||||
. It must allow for easy retrieval of the document from storage.
|
||||
|
||||
To fulfill the above, and given that Radicle uses Git for storage of
|
||||
repository data, we choose to use the _Git Object
|
||||
ID_footnote:[https://git-scm.com/book/en/v2/Git-Internals-Git-Objects]
|
||||
of the identity document, as identifier. Git object IDs, or _OIDs_ are
|
||||
"`hardened`" SHA-1 checksums of their content, prefixed with a short
|
||||
header. We can compute this OID using the `git hash-object` command. But
|
||||
before doing so, we must take care of one last thing: to make the
|
||||
process of hashing our identity document fully deterministic, we must
|
||||
first ensure our document is in canonical JSON
|
||||
formfootnote:[https://datatracker.ietf.org/doc/html/rfc8785]. This
|
||||
prevents things like whitespace or key ordering from influencing the
|
||||
document hash and therefore the identifier. In turn, this makes the
|
||||
identifier easier to compute correctly.
|
||||
|
||||
The above document in canonical form looks like this:
|
||||
|
||||
....
|
||||
{"delegates":["did:key:z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi"],
|
||||
"payload":{"xyz.radicle.project":{"defaultBranch":"master","description":
|
||||
"Radicle Heartwood Protocol & Stack","name":"heartwood"}},"threshold":1}
|
||||
....
|
||||
|
||||
We can now compute the Git object ID by placing the above JSON in a
|
||||
file, eg. `radicle.json`, taking care to strip all newline characters
|
||||
from it, and running the following command:
|
||||
|
||||
....
|
||||
$ git hash-object -t blob radicle.json
|
||||
....
|
||||
|
||||
The output should be:
|
||||
|
||||
....
|
||||
d96f425412c9f8ad5d9a9a05c9831d0728e2338d
|
||||
....
|
||||
|
||||
This SHA-1 hash is the document’s OID. To turn it into a Radicle
|
||||
repository identifer, we encode the underlying 20-byte hash value using
|
||||
`multibase` encodingfootnote:[https://w3c-ccg.github.io/multibase/] with
|
||||
the `base-58-btc` alphabet; the same encoding used for the `did:key`
|
||||
method, and prefix `rad:` to it, making it a valid URN:
|
||||
|
||||
....
|
||||
"rad" ":" multibase(base58-btc, raw-oid-bytes)
|
||||
....
|
||||
|
||||
This results in the repository identifier, or RID:
|
||||
|
||||
....
|
||||
rad:z42hL2jL4XNk6K8oHQaSWfMgCL7ji
|
||||
....
|
||||
|
||||
This RID is theoretically unique thanks to the entropy provided by the
|
||||
delegate key and payload.
|
||||
|
||||
== Identity Storage
|
||||
|
||||
A storage system suitable for storing identity documents must have two
|
||||
properties:
|
||||
|
||||
[arabic]
|
||||
. It must guarantee data integrity.
|
||||
. It must preserve the history of changes to the documents.
|
||||
|
||||
Radicle repositories are stored in Git, and criteria (1) is guaranteed
|
||||
by Git natively, so long as we store our identity documents in the Git
|
||||
object database. This is because Git hashes all objects under it, and
|
||||
structures its data such that a change in hash of one object means a
|
||||
change in hash of all dependent objects.
|
||||
|
||||
Criteria (2) can be guaranteed by encoding changes to the documents as a
|
||||
Git commit history. Not only does a commit history allow us to preserve
|
||||
all changes, it also proves, via hash-linking that no change was omitted
|
||||
from the history.
|
||||
|
||||
....
|
||||
Commit Commit Commit
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ fb8e40a │◄─┐ │ c581f25 │◄─┐ │ 43eb12d │
|
||||
│ │ └─┤ fb8e40a │ └─┤ c581f25 │
|
||||
│ ┌──────────┐│ │ ┌──────────┐│ │ ┌──────────┐│
|
||||
│ │ Document ││ │ │ Document ││ │ │ Document ││
|
||||
│ ├──────────┤│ │ ├──────────┤│ │ ├──────────┤│
|
||||
└─┴──────────┴┘ └─┴──────────┴┘ └─┴──────────┴┘
|
||||
....
|
||||
|
||||
In the above diagram, we see three commits, with the left-most being the
|
||||
_root_ commit of the document, ie. the initial state, and the right-most
|
||||
being the _head_, ie. the latest state.
|
||||
|
||||
Each commit includes within its Git tree, a blob named `radicle.json`
|
||||
which contains a version of the identity document.
|
||||
|
||||
....
|
||||
┌───────┬─────────┐ ┌─────┬───────────┐
|
||||
│commit │ fb8e40a │ ┌─────┬────────────────────┐ ┌►│blob │ d96f425 │
|
||||
├───────┴─────────┤ ┌►│tree │ 82bc09a │ │ ├─────┴───────────┤
|
||||
│tree 82bc09a ├─┘ ├─────┴────────────────────┤ │ │{"delegates":...,│
|
||||
│author ... │ │blob d96f425 radicle.json ├──┘ │ "payload":..., │
|
||||
│ │ └──────────────────────────┘ │ "threshold":...}│
|
||||
└─────────────────┘ └─────────────────┘
|
||||
....
|
||||
|
||||
Using this representation, we can hence track all changes to a given
|
||||
identity. To keep track of the head of this history, we use a special
|
||||
Git reference named `rad/id` which points to the latest version of the
|
||||
identity document:
|
||||
|
||||
....
|
||||
fb8e40a ◄─ c581f25 ◄─ 43eb12d ◄─ refs/rad/id
|
||||
....
|
||||
|
||||
Just like regular Git branches, when the identity is updated,
|
||||
`refs/rad/id` is reset to point to the latest commit. Note that this
|
||||
commit history is completely separate from the source code history
|
||||
pointed to by eg. `refs/heads/master` and other branches. The identity
|
||||
history is kept in the repository’s _stored copy_, which is a bare
|
||||
repository, and is not included in working copies.
|
||||
|
||||
== Verification
|
||||
|
||||
Verifying the latest state, ie. commit `43eb12d` is a matter of
|
||||
verifying all preceding states, starting from the root (`fb8e40a`).
|
||||
|
||||
The root commit is verifiable intrinsically, since it contains as part
|
||||
of its Git tree, the document from which we derived the RID. We can see
|
||||
above that the original blob from which we computed the repository
|
||||
identifier is contained in the tree pointed to by the root commit of the
|
||||
document history. The root commit is hence valid for a given RID if and
|
||||
only if it contains a blob under the name `radicle.json` containing a
|
||||
valid identity document which hashes to the given RID, _and_ the commit
|
||||
is signed by all delegates in the initial `delegates` list.
|
||||
|
||||
Once the root commit is verified, we can proceed to the next commit.
|
||||
Since the document may have changed in this commit, the RID is no longer
|
||||
useful for verifying this commit. Instead, we make sure that two
|
||||
conditions are fulfilled:
|
||||
|
||||
[arabic]
|
||||
. The commit containing the updated document is signed by a number of
|
||||
keys greater than or equal to the `threshold` property of the
|
||||
_previous_, valid version of the document.
|
||||
. Each of the aforementioned signatures belongs to a key that is part of
|
||||
the `delegates` set of the previous document version.
|
||||
|
||||
____
|
||||
Git commit signatures can be verified with the `git verify-commit` tool.
|
||||
____
|
||||
|
||||
These delegate signatures are expected to be included in the commit
|
||||
header under the `gpgsig` key, and be encoded in the SSH signature
|
||||
format.
|
||||
|
||||
....
|
||||
tree c66cc435f83ed0fba90ed4500e9b4b96e9bd001b
|
||||
parent af06ad645133f580a87895353508053c5de60716
|
||||
author Buck Mulligan <buck@mulligan.xyz> 1664467633 +0200
|
||||
committer Buck Mulligan <buck@mulligan.xyz> 1664786099 -0200
|
||||
gpgsig -----BEGIN SSH SIGNATURE-----
|
||||
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgvjrQogRxxLjzzWns8+mKJAGzEX
|
||||
4fm2ALoN7pyvD2ttQAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
|
||||
AAAAQIQvhIewOgGfnXLgR5Qe1ZEr2vjekYXTdOfNWICi6ZiosgfZnIqV0enCPC4arVqQg+
|
||||
GPp0HqxaB911OnSAr6bwU=
|
||||
-----END SSH SIGNATURE-----
|
||||
gpgsig -----BEGIN SSH SIGNATURE-----
|
||||
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgDb3ulFKnHALG8AnuuFPY9prvVZ
|
||||
kyLc73tcQ+HG3sCzQAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
|
||||
AAAAQM9rxErTt7AtcLypSyVM/jmd9/syO4D5hjMjL/9lbGzIkXXDL6+QlUsLipeLuYHV92
|
||||
F/6nm/lEaPUTeiZQ5o9AI=
|
||||
-----END SSH SIGNATURE-----
|
||||
....
|
||||
|
||||
A Git commit header with two SSH signatures.
|
||||
|
||||
We proceed in this manner until the last commit in the history. If all
|
||||
commits pass this verification process, we consider the identity valid.
|
||||
Note that every version of the document must be validated according to
|
||||
the rules stated under the link:#validation[Validation] section. This
|
||||
includes the document payload, and implies that application developers
|
||||
supporting payload extensions will have to provide their own validation
|
||||
for these payloads, that will have to run for each commit in the
|
||||
document history.
|
||||
|
||||
It’s important to restate that for any commit `C`, other than the root
|
||||
commit, verification is done by using the `delegates` and `threshold`
|
||||
values of the _parent_ commit to `C`, which has already been verified.
|
||||
|
||||
== Security
|
||||
|
||||
The combination of Git storage and cryptographic verification provides
|
||||
very strong security and integrity guarantees around Radicle
|
||||
repositories and identities:
|
||||
|
||||
* Omitted data up to the latest commit is detected by Git itself
|
||||
* Tampering with the identity root will result in a different RID
|
||||
* Adding a delegate key without the sign-off of the existing delegate
|
||||
set will fail verification
|
||||
|
||||
There is one possible attack that can be carried out by network
|
||||
participants: serving old data. Since it isn’t possible to know whether
|
||||
a document history has a more recent update than the latest known
|
||||
update, a dishonest peer may choose to hide the last _N_ identity
|
||||
updates from its peers. This means it will serve a stale document to its
|
||||
peers.
|
||||
|
||||
However, this attack is only effective if _all_ of a victim’s connected
|
||||
peers perform this censorship. It takes only one honest peer to serve
|
||||
the full document history for the censorship to fail.
|
||||
|
||||
== Closing Thoughts
|
||||
|
||||
In this RIP we described an identity system for Git repositories that
|
||||
can be used to securely distribute code on a peer-to-peer network. The
|
||||
system is self-certifying and requires only basic Git primitives to
|
||||
implement.
|
||||
|
||||
== Credits
|
||||
|
||||
* Kim Altintop, for the original design this system is based on
|
||||
|
||||
== Copyright
|
||||
|
||||
This document is licensed under the Creative Commons CC0 1.0 Universal
|
||||
license.
|
||||
@@ -0,0 +1,590 @@
|
||||
= RIP-0003: Storage Layout
|
||||
@fintohaps <fintan.halpenny@gmail.com>
|
||||
:rip: 0003
|
||||
:status: Final
|
||||
:type: Standard
|
||||
:created-at: 2022-10-27
|
||||
:updated-at: 2022-10-27
|
||||
:copyright: CC0-1.0
|
||||
|
||||
include::_metadata.adoc[]
|
||||
|
||||
:toc:
|
||||
|
||||
The storage layer is a crucial component of the Radicle network, and it
|
||||
is designed with a local-first approach. This means that it can
|
||||
accommodate not only the local operator’s view of a repository, but also
|
||||
the views of peers in whom the operator is interested. These views, also
|
||||
known as _forks_ or _source trees_, play a key role in enabling
|
||||
collaboration and version control within the network.
|
||||
|
||||
== Overview
|
||||
|
||||
In a peer-to-peer network, there is no centralized server or repository
|
||||
for users to submit their changes. Additionally, the absence of a
|
||||
consensus mechanism at the protocol level means that the sequence of
|
||||
operations cannot be guaranteed. To tackle these issues, Radicle
|
||||
implements a partitioned approach in which each user maintains their own
|
||||
local "`fork`" of a repository, as well as any other forks they have an
|
||||
interest in. These forks are then shared among users across the network.
|
||||
This method not only enhances the user experience by allowing offline
|
||||
work but also eliminates the need for a server to process data. Each
|
||||
repository fork has a single owner and writer, and users are only
|
||||
permitted to make changes to their respective forks.
|
||||
|
||||
The storage layer must also be designed for efficient replication of
|
||||
data between peers. For this reason, Git is used as the underlying
|
||||
protocol and database, as it maps nicely to the type of data exchanged
|
||||
on the Radicle network, and is flexible enough for our use case. In
|
||||
addition, Git has been optimized for speed and disk space, and will
|
||||
automatically de-duplicate repository data and fetch missing objects
|
||||
from
|
||||
peersfootnote:[https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols].
|
||||
|
||||
With the above in mind, this document proposes a storage layer that
|
||||
meets the following requirements:
|
||||
|
||||
[arabic]
|
||||
. The storage layer is capable of maintaining a local copy of the
|
||||
working dataset.
|
||||
. The storage layer can store any number of repositories.
|
||||
. For each repository, it can represent multiple views, or _forks_, of
|
||||
the repository.
|
||||
. The storage layer can natively interoperate with Git.
|
||||
|
||||
There are two aspects to consider for Git interoperability:
|
||||
|
||||
[arabic]
|
||||
. Repository replication between peers.
|
||||
. Associating a _working_ repository or "`copy`" with a _stored_
|
||||
repository.
|
||||
|
||||
In the next sections we will cover how the above works with the storage
|
||||
layout.
|
||||
|
||||
== Layout
|
||||
|
||||
The storage layout must support multiple repositories and multiple peers
|
||||
per repository. Each stored repository is a _bare_ Git
|
||||
repositoryfootnote:[https://git-scm.com/docs/git-init#Documentation/git-init.txt—bare].
|
||||
To ensure uniqueness and easy identification of repositories, a stable
|
||||
and globally unique identifier, known as the Repository ID (RID), is
|
||||
assigned to each stored repository. The RID for each repository is
|
||||
established according to the guidelines provided in RIP#2’s section _The
|
||||
Repository Identifier_, and is represented as `++<++rid++>++` in
|
||||
diagrams found in this document.
|
||||
|
||||
Since our underlying storage uses Git, we represent the storage layout
|
||||
as a file tree on the file-system, with `++<++storage++>++` representing
|
||||
the storage root, or top-level directory under which all repositories
|
||||
are stored on a user’s device. Though this storage tree is browsable by
|
||||
the user with standard file system commands, it is not meant to be
|
||||
interacted with directly by users, for risk of corrupting the data.
|
||||
Additionally, Git is free to pack the objects, which means they may not
|
||||
always appear as individual files.
|
||||
|
||||
....
|
||||
<storage> # Storage root containing all local repositories
|
||||
├── <rid> # Some repository, e.g. a project, as a bare git repository
|
||||
│ └── refs # All Git references under this project
|
||||
├── <rid>
|
||||
│ └── refs
|
||||
├── <rid>
|
||||
│ └── refs
|
||||
└── ...
|
||||
....
|
||||
|
||||
Basic overview of the storage layout with multiple repositories
|
||||
|
||||
For every repository, each peer associated with that repository must
|
||||
have a separate, logical Git source tree – which contains all the usual
|
||||
reference namespaces, i.e. `heads`, `tags`, and `notes`. This _logical
|
||||
repository_ is what we call _fork_ or _view_, and allows peers to
|
||||
maintain different sets of changes for the same physical repository.
|
||||
|
||||
....
|
||||
<storage>
|
||||
└─ <rid> # The "physical" Git repository
|
||||
└─ refs
|
||||
└─ namespaces # All forks are stored under this namespace
|
||||
├─ <nid> # One peer's fork is stored here
|
||||
│ └─ refs
|
||||
├─ <nid> # Another peer's fork is stored here
|
||||
│ └─ refs
|
||||
└─ <nid> # Etc.
|
||||
└─ refs
|
||||
....
|
||||
|
||||
Storage partitioning by Node ID or `++<++nid++>++`
|
||||
|
||||
To have this separation, instead of having each peer stored in a
|
||||
separate Git repository with a separate object database (ODB), the
|
||||
`gitnamespaces`footnote:[https://git-scm.com/docs/gitnamespaces] feature
|
||||
is used. For each peer, including the local peer, their unique
|
||||
identifier is used as the namespace within each repository to separate
|
||||
Git objects. The identifier used is described in _Peer Identity_ in
|
||||
RIP#2, and is usually known as the _Node Identifier_ (NID):
|
||||
|
||||
____
|
||||
In Heartwood, peers are simply identified by their public key. This key
|
||||
is an Ed25519 key that is encoded as a DID using the `did:key` method.
|
||||
DIDs are used for interoperability with other systems as well as
|
||||
allowing for other types of identifiers in the future.
|
||||
____
|
||||
|
||||
Thus, each peer can have its own namespace for references, while sharing
|
||||
the objects with other peers via a shared ODB. This ensures only one
|
||||
copy of each object is stored across all repository forks.
|
||||
|
||||
The storage uses the encoded public key portion of the `did:key` string
|
||||
as the namespace path, denoted as `++<++nid++>++` or _Node ID_ going
|
||||
forward. This means that a peer’s references will be scoped by their
|
||||
Node ID via the path prefix `refs/namespaces/++<++nid++>++`. We
|
||||
demonstrate this organisation below in more detail:
|
||||
|
||||
....
|
||||
<storage> # Storage root containing all local repositories
|
||||
├─ <rid> # Storage for first repository
|
||||
│ └─ refs # All Git references locally stored
|
||||
│ └─ namespaces # All peer source trees or "forks"
|
||||
│ ├─ <nid> # First node's source tree
|
||||
│ │ └─ refs # First node's Git references
|
||||
│ │ ├─ heads # First node's branches
|
||||
│ │ │ └─ master # First node's master branch
|
||||
│ │ ├─ tags # First node's tags
|
||||
│ │ │ ...
|
||||
│ │ └─ rad
|
||||
│ │ └─ id # First node's version of the repository identity document
|
||||
│ │
|
||||
│ └─ <nid> # Second node's source tree
|
||||
│ ├─ refs # Second node's references
|
||||
│ └─ ...
|
||||
├─ <rid> # Storage for second repository
|
||||
│ ...
|
||||
└─ <rid> # etc.
|
||||
...
|
||||
....
|
||||
|
||||
Note that top-level references may still exist,
|
||||
i.e. `++<++rid++>++/refs/++{++heads,tags}`. The top-level namespace must
|
||||
be reserved for canonical references – references that are agreed upon
|
||||
collaboratively, as published and stable. They do not belong to any one
|
||||
peer and thus may be different on each device. How canonical references
|
||||
are decided and written is left for a future RIP.
|
||||
|
||||
....
|
||||
<storage>
|
||||
└─ <rid>
|
||||
└─ refs
|
||||
├─ HEAD # Canonical head reference
|
||||
├─ heads # Canonical branches
|
||||
│ └─ master # Canonical master branch
|
||||
├─ tags
|
||||
│ └─ v1.0.0 # Canonical v1.0.0 release tag
|
||||
├─ rad
|
||||
│ └─ id # Canonical identity reference
|
||||
└─ namespaces # All peer source trees
|
||||
├─ <nid> # First node's source tree
|
||||
└─ <nid> # Second node's source tree
|
||||
...
|
||||
....
|
||||
|
||||
Example of canonical references under a repository
|
||||
|
||||
== Replication
|
||||
|
||||
Repository replication involves retrieving data from a remote peer. As
|
||||
the storage consists of Git repositories, data can be transferred
|
||||
remotely using the Git
|
||||
protocolsfootnote:[https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols]
|
||||
and appropriate
|
||||
refspecsfootnote:[https://git-scm.com/book/en/v2/Git-Internals-The-Refspec].
|
||||
However, this document does not cover the protocol used or how to verify
|
||||
fetched data, as those topics are beyond its scope. They may be
|
||||
discussed in a separate document.
|
||||
|
||||
That being said, we designed the storage layout such that it’s easy to
|
||||
transfer data between repositories over the network, using an unmodified
|
||||
Git protocol. Using refspecs, it’s possible to transfer only the objects
|
||||
we’re interested in, for example we can fetch only a certain peer’s fork
|
||||
and not another.
|
||||
|
||||
== Working Copy
|
||||
|
||||
A working copy is a local copy of a repository, which corresponds to a
|
||||
repository in storage. The operator can make changes to the source code
|
||||
in the working copy. This is similar to how one would use `git clone` to
|
||||
obtain a copy of an upstream repository, such as one hosted on GitHub or
|
||||
GitLab. Once the changes have been made in the working copy, they can be
|
||||
pushed upstream. With Radicle, changes are fetched and pushed between
|
||||
the _working_ copy and the _stored_ copy within the local storage.
|
||||
|
||||
The connection between the working copy and the storage is maintained
|
||||
through a set of Git
|
||||
remotesfootnote:[https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes],
|
||||
where each remote represents a single remote peer or _namespace_ for
|
||||
that repository and is associated with a Node ID.
|
||||
|
||||
The name of each remote, defined by the operator or application, can be
|
||||
customized to suit their preferences. For instance, the operator may use
|
||||
the Node ID of the peer, `origin`, `rad`, a nickname, or any other
|
||||
desired name. By convention, we use the `rad` remote for the local
|
||||
peer’s remote, such that a user may push changes to his or her own fork
|
||||
with `git push rad`.
|
||||
|
||||
The URL of each Git remote must resolve the local storage’s repository
|
||||
corresponding to the working copy. As such, the URL serves as a mapping
|
||||
between the working copy and the stored copy.
|
||||
|
||||
== URL
|
||||
|
||||
The URL scheme for a given Radicle remote is of the form:
|
||||
|
||||
....
|
||||
rad://<rid>[/<nid>]
|
||||
....
|
||||
|
||||
* The `rad://` scheme is used for Radicle repositories, and identifies a
|
||||
project on the network. By using this scheme with Git, the user
|
||||
instructs Git to invoke the `git-remote-rad` executable during
|
||||
`git push` or `git fetch`, which allows the user to interact with the
|
||||
network through the storage layer. This will be covered in more detail
|
||||
in the _Remote Helper_ section.
|
||||
* The `++<++rid++>++` component is the repository identifier to be found
|
||||
in storage.
|
||||
* The `++<++nid++>++` component is the Node ID which the `--namespace`
|
||||
option will be set to. If `++<++nid++>++` is not specified, Git will
|
||||
interact with the repository’s _canonical references_.
|
||||
|
||||
Here’s an example URL for repository `z42hL2jL4XNk6K8oHQaSWfMgCL7ji` and
|
||||
peer `z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi`:
|
||||
|
||||
....
|
||||
rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji/z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
|
||||
....
|
||||
|
||||
Here’s a URL for the same repository’s canonical references:
|
||||
|
||||
....
|
||||
rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji
|
||||
....
|
||||
|
||||
== Refspecs
|
||||
|
||||
Since Git namespaces are used, the `fetch`
|
||||
refspecfootnote:[https://git-scm.com/book/en/v2/Git-Internals-The-Refspec]
|
||||
may be:
|
||||
|
||||
....
|
||||
+refs/heads/*:refs/remotes/<name>/*
|
||||
....
|
||||
|
||||
The operator may also want to scope tags to particular remotes. This can
|
||||
be achieved by using the `tagOpt` of a remote and adding another fetch
|
||||
refspec.
|
||||
|
||||
....
|
||||
fetch = +refs/tags/*:refs/remotes/<name>/tags/*
|
||||
tagOpt = --no-tags
|
||||
....
|
||||
|
||||
When using these refspecs with `git fetch` or `git push`, it is
|
||||
necessary to specify the namespace that is being used for the operation.
|
||||
This can be achieved using `git --namespace=++<++nid++>++` or
|
||||
`GIT++_++NAMESPACE=++<++nid++>++ git`. Unfortunately, this is somewhat
|
||||
cumbersome for the user and does not prevent pushing to namespaces
|
||||
belonging to a non-local peer. This is remedied in
|
||||
link:#remote-helper[Remote Helper].
|
||||
|
||||
== Example
|
||||
|
||||
Here’s an example remote configuration based on the above
|
||||
specifications:
|
||||
|
||||
....
|
||||
[remote "rad"]
|
||||
url = rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji/z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
|
||||
fetch = +refs/heads/*:refs/remotes/rad/*
|
||||
....
|
||||
|
||||
To support fetching canonical references while pushing to the local
|
||||
peer’s namespace, a configuration like the following can be used:
|
||||
|
||||
....
|
||||
[remote "rad"]
|
||||
url = rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji
|
||||
pushurl = rad://z42hL2jL4XNk6K8oHQaSWfMgCL7ji/z6MknSLrJoTcukLrE435hVNQT4JUhbvWLX4kUzqkEStBU8Vi
|
||||
fetch = +refs/heads/*:refs/remotes/rad/*
|
||||
....
|
||||
|
||||
In the above configuration, `git pull rad` would pull the canonical
|
||||
references while `git push rad` would push to the local user’s
|
||||
namespace.
|
||||
|
||||
For a more thorough example, see the link:#appendix[Appendix].
|
||||
|
||||
== Remote Helper
|
||||
|
||||
The remote helper is what allows Git to interpret URLs with the `rad://`
|
||||
scheme.
|
||||
|
||||
As mentioned in the link:#working-copy[Working Copy] section, there is
|
||||
currently no way to configure a Git remote to be aware of additional
|
||||
logic, such as the appropriate `refs/namespaces` to use (to avoid having
|
||||
to use `--namespace`) or to prevent pushing to other peers’ namespaces.
|
||||
|
||||
To address these requirements, a `git-remote-rad` helper binary can be
|
||||
introduced to supply the necessary namespace and enforce the correct use
|
||||
of peer namespaces.
|
||||
|
||||
`git-remote-rad` is a
|
||||
gitremote-helperfootnote:[https://git-scm.com/docs/gitremote-helpers]
|
||||
binary. When Git encounters a URL that uses the `rad` transport
|
||||
protocol, it delegates the call to `git-remote-rad`, which should be
|
||||
found in the operator’s path, during a `fetch` or `push` operation.
|
||||
|
||||
=== Authorization
|
||||
|
||||
With the remote helper installed, `git push` can automatically set
|
||||
`GIT++_++NAMESPACE` to the Node ID of the current user after verifying
|
||||
that it matches the one specified in the URL, and reject pushes to other
|
||||
Node IDs.
|
||||
|
||||
When fetching, the remote helper can set `GIT++_++NAMESPACE` to whatever
|
||||
Node ID is specified in the URL, as no authorization is required to
|
||||
fetch.
|
||||
|
||||
== Future Work
|
||||
|
||||
You may have noticed that in this link:#layout[layout] the top-level
|
||||
namespace is left for canonical references. The definition and
|
||||
verification of canonicity is left for a future RIP.
|
||||
|
||||
== Appendix
|
||||
|
||||
== Alternative Designs
|
||||
|
||||
An alternative design for organizing peer source trees is to use the
|
||||
`remotes` namespaces, i.e. `refs/remotes/++<++nid++>++`. This particular
|
||||
namespace is deemed special by `git` and its tooling. A "`remote`"
|
||||
reference is one that corresponds to a remote location. The remote
|
||||
location and how to fetch/push from/to is configured using
|
||||
`git remote`footnote:[https://git-scm.com/docs/git-remote]. When
|
||||
`git fetch` is used for that remote, it will place the references under
|
||||
`refs/remotes`footnote:[https://git-scm.com/book/en/v2/Git-Internals-Git-References].
|
||||
|
||||
=== Associating a Working Copy
|
||||
|
||||
Continuing along this line of enquiry, we look at how this storage will
|
||||
link to a working copy – our personal directory for editing the code. As
|
||||
we previously said, we will want to setup a remote in the working copy.
|
||||
This will look like the following:
|
||||
|
||||
....
|
||||
[remote "alice"]
|
||||
url = file:///path/to/storage
|
||||
fetch = +refs/remotes/alice/heads/*:refs/remotes/alice/*
|
||||
....
|
||||
|
||||
This will do what you expect when running:
|
||||
|
||||
....
|
||||
$ git fetch alice
|
||||
....
|
||||
|
||||
However, you may be surprised that when running:
|
||||
|
||||
....
|
||||
$ git fetch alice master
|
||||
fatal: couldn't find remote ref master
|
||||
....
|
||||
|
||||
It will not result in fetching the latest changes from `master`. In
|
||||
fact, it will say no reference exists. To get the exact `master` we are
|
||||
looking for we must run:
|
||||
|
||||
....
|
||||
$ git fetch alice refs/remotes/alice/heads/master
|
||||
....
|
||||
|
||||
To explain, `git` tends to work under a DWIM (Do What I Mean) principle.
|
||||
The `master` in `git fetch alice master` is ambiguous, in general. It
|
||||
could be `refs/heads/master`, `refs/remotes/origin/master`,
|
||||
`refs/remotes/alice/heads/master`, etc. `git` will assume that what you
|
||||
meant was `refs/heads/master` and will look for this on the remote end,
|
||||
but of course it does not exist.
|
||||
|
||||
This problem is only compounded with
|
||||
`refs/tags`footnote:[https://git-scm.com/book/en/v2/Git-Internals-Git-References],
|
||||
where pushing a tag to a remote will always DWIM and target the
|
||||
`refs/tags` namespace – unless otherwise specified.
|
||||
|
||||
Thus, we see that this design is not adequate.
|
||||
|
||||
== Worked Example
|
||||
|
||||
To begin we want to set up three git repositories: `storage`, `project`,
|
||||
and `fork`. The `storage` repository will act like the Radicle storage,
|
||||
while `project` and `fork` are working copies that will be linked to
|
||||
`storage` via their remote entries.
|
||||
|
||||
....
|
||||
# Storage setup
|
||||
$ mkdir storage
|
||||
$ cd storage
|
||||
$ git init --bare
|
||||
|
||||
# Project setup
|
||||
$ mkdir project
|
||||
$ cd project
|
||||
$ git init
|
||||
|
||||
# Fork setup
|
||||
$ mkdir fork
|
||||
$ cd fork
|
||||
$ git init
|
||||
....
|
||||
|
||||
=== Pushing Changes
|
||||
|
||||
Our first action will be to make changes in `project` and push them to
|
||||
`storage`. In order for us to do that we need to create a remote in
|
||||
`project`, create a commit, and push it to `storage`.
|
||||
|
||||
....
|
||||
# Add remote: "alice" will be used instead of a Node ID
|
||||
$ cd project
|
||||
$ git remote add alice file:///home/user/radicle/storage
|
||||
|
||||
# Add a commit
|
||||
$ touch README.md && git add README.md && git commit -am "Add README"
|
||||
$ git --namespace=alice push alice master
|
||||
....
|
||||
|
||||
`git` will then print out that it pushed a new branch and we can confirm
|
||||
by inspecting the `refs` in `storage`.
|
||||
|
||||
....
|
||||
# Inspect refs
|
||||
$ cd storage
|
||||
$ tree refs
|
||||
refs
|
||||
├── heads
|
||||
├── namespaces
|
||||
│ └── alice
|
||||
│ └── refs
|
||||
│ └── heads
|
||||
│ └── master
|
||||
└── tags
|
||||
....
|
||||
|
||||
=== Fetching Changes
|
||||
|
||||
Our next action will be to fetch the changes from `alice` in the `fork`
|
||||
repository. To do this, we must add a remote – like before – and run a
|
||||
`git fetch`.
|
||||
|
||||
....
|
||||
# Add remote; alice will mimic the public key hash
|
||||
$ cd fork
|
||||
$ git remote add alice file:///home/user/radicle/storage
|
||||
|
||||
# Fetch the changes
|
||||
$ git --namespace=alice fetch alice
|
||||
....
|
||||
|
||||
This will fetch the `heads` from `alice` and put them under the remote
|
||||
`alice`. We can confirm this by inspecting the `refs` in `fork`.
|
||||
|
||||
....
|
||||
# Inspect refs
|
||||
$ tree .git/refs
|
||||
.git/refs
|
||||
├── heads
|
||||
├── remotes
|
||||
│ └── alice
|
||||
│ └── master
|
||||
└── tags
|
||||
....
|
||||
|
||||
=== Different Peers
|
||||
|
||||
To imitate the reality that there will be a namespace per peer, we add a
|
||||
new remote for `fork`. We can then make changes to `alice/master` and
|
||||
publish it under the `bob` namespace.
|
||||
|
||||
....
|
||||
# Add bob remote
|
||||
$ git remote add bob file:///home/user/radicle/storage
|
||||
|
||||
$ git merge bob/master
|
||||
$ echo "Hello, Radicle" >> README.md
|
||||
$ git commit -am "Hello, Radicle"
|
||||
$ git --namespace=bob push bob master
|
||||
....
|
||||
|
||||
Again, we can confirm this did what we wanted in `storage`.
|
||||
|
||||
....
|
||||
# Inspect storage refs
|
||||
cd storage
|
||||
tree refs
|
||||
refs
|
||||
├── heads
|
||||
├── namespaces
|
||||
│ ├── alice
|
||||
│ │ └── refs
|
||||
│ │ └── heads
|
||||
│ │ └── master
|
||||
│ └── bob
|
||||
│ └── refs
|
||||
│ └── heads
|
||||
│ └── master
|
||||
└── tags
|
||||
....
|
||||
|
||||
=== Non-global Tags
|
||||
|
||||
Often we find that pushing tags pollutes the `refs/tags` namespace since
|
||||
they do not get placed under `remotes` when fetching. With the use of
|
||||
the `gitnamespaces` feature we avoid this.
|
||||
|
||||
....
|
||||
$ cd fork
|
||||
$ git tag v1.0.0
|
||||
$ git push v1.0.0
|
||||
|
||||
# Inspect storage refs
|
||||
refs
|
||||
├── heads
|
||||
├── namespaces
|
||||
│ ├── alice
|
||||
│ │ └── refs
|
||||
│ │ └── heads
|
||||
│ │ └── master
|
||||
│ └── bob
|
||||
│ └── refs
|
||||
│ ├── heads
|
||||
│ │ └── master
|
||||
│ └── tags
|
||||
│ └── v1.0.0
|
||||
└── tags
|
||||
....
|
||||
|
||||
This shows that namespaces are superior in organising references
|
||||
correctly for each given peer.
|
||||
|
||||
== Credits
|
||||
|
||||
* Kim Altintop, for shining the light on the lesser known
|
||||
`gitnamespaces`footnote:[https://git-scm.com/docs/gitnamespaces] feature
|
||||
while developing `radicle-link`.
|
||||
* Alex Good, for attempting to implement a feature dubbed "`ref
|
||||
rewriting`" to solve the remotes problem, before realising that using
|
||||
`gitnamespaces`footnote:[https://git-scm.com/docs/gitnamespaces] could
|
||||
be a better option.
|
||||
|
||||
== Copyright
|
||||
|
||||
This document is licensed under the Creative Commons CC0 1.0 Universal
|
||||
license.
|
||||
@@ -0,0 +1,11 @@
|
||||
[horizontal]
|
||||
RIP:: {rip}
|
||||
ifdef::discussion[Discussion:: {discussion}]
|
||||
Status:: {status}
|
||||
Type:: {type}
|
||||
Created:: {created-at}
|
||||
Updated:: {updated-at}
|
||||
ifdef::requires[Requires:: {requires}]
|
||||
ifdef::replaces[Replaces:: {replaces}]
|
||||
ifdef::superseded-by[Superseded by:: {superseded-by}]
|
||||
Copyright:: {copyright}
|
||||
Reference in New Issue
Block a user