3 posts tagged with "Draft"

Status​

Draft

Context​

When we will run Mithril on mainnet there will be thousands of signers running. Upgrading the version of the nodes has an impact as different versions of API, messages, signature may lead to loss of a significant part of the signers population over one epoch or more. In any case we must prevent a gap in the certificate chain while upgrading critical parts.

We need to be able to keep enough of signer nodes and the aggregator able to work together in order to produce at least one certificate per epoch.

Examples of such changes:

• change in the message structure
• change in the cryptographic algorithm
• change in communication channels

Decision​

In order to synchronize all nodes behavior transition, the Release Team will define Eras that start at a given Cardano Epoch and lasts until the next Era begins. When nodes detect an Era change, they switch from old to new behavior hence all transitioning at almost the same time.

Consequences​

Release Team​

The release team is the team responsible of releasing new versions of Mithril software. The Release Team will be responsible to set the Epoch at which Eras change using an Era Activation Marker. In order to be able to determine when the new Era will begin, the Release Team has to know what is the share of the total Mithril stake that can run the new behavior. Signer node software versions has to be monitored.

Version monitoring​

The Release Team must be aware of the software version run by the Signer nodes and their associated stake. The version is going to be added to all HTTP headers in inter-node communication. In a first step, the Aggregator nodes will record this information, and provide the mapping of stakes to Signer nodes.

This configuration works in the case where there is a centralized Aggregator Node (as it is today). In the future, there may be several Aggregator nodes working in a decentralized manner. This would mean having a separate monitoring service, and also monitor the aggregators node versions.

Era Activation Marker​

An Era Activation Marker is an information shared among all the nodes. For every upgrade, there are two phases:

• a first marker is set on the blockchain that just indicates a new Era will start soon and softwares shall be updated.
• a second marker is set that specifies the Epoch when they must switch from old to new behavior.

Every Era Activation Marker will be a transaction in the Cardano blockchain. This implies the nodes must be able to read transactions of the blockchain. Era Activation Markers can be of the same type, the first maker does not hold any Epoch information whereas the second does.

Node will check the blockchain for Markers at startup and for every new Epoch. When a node detects a Marker, it will warn the user if it does not support the incoming Era that he must upgrade his node. If the node detects it does not support the current Era, it will stop working with an explicit error message. To ease that operation, Era Activation Marker will be made sortable.

Behavior Switch​

The nodes must be able to switch from one behavior to another when the Era Epoch is reached. This means the software must embed both behaviors. The switch is developed as a one time operation, there is no rollback mechanism available. Once the Epoch is transitioned and the switch has occurred, a new software release can remove the old behavior from the codebase.

Status​

draft

Context​

In order to deliver regularly the software to our users, we should implement a release process based on a predictable versioning scheme.

Versioning​

A Release Version determines a distribution of determined node versions and underlying libraries.

• Our softwares must be able to interact seamlessly with other Mithril software.
• Our softwares must be able to be hosted on crates.io.
• Our softwares must clearly indicate compatibility with other Mithril components to end users.

Release process​

A Release is a software package that is built once and then promoted from the testing environment to the production environment. It can be signed.

• Keep it simple.
• Automated as much as possible: all points not requiring human decision shall be automated.
• Minimize the mean time to release.

Decision​

There are 3 versioned layers in the Mithril stack:

• HTTP API protocol to ensure compatibility in the communication between nodes (use Semver).
• Crate version: each node & library has its own version (use Semver). The commit digest is automatically added to the version by the CI pipeline.
• Release Version: the distribution version (use version scheme YYWW.patch | YYWW.patch-name). The VERSION file is computed by the pipeline from the tag release.

The documentation is tied to a Release Version.

Release Process​

Starting just after a new release has been made:

1. Develop on a dedicated development branch.
2. When merging PR on main: update the Cargo.toml files with version of the updated nodes.
3. Once merged, the CI creates an unstable tag & release which is deployed on testing environment.
4. Push a tag using the distribution version format on this commit with a -prerelease suffix.
5. The CI gets the built artifacts associated with this commit and generates a named pre-release which is deployed on pre-release for testing.
6. Push a tag using the distribution version format on this commit without the -prerelease suffix.
7. The CI gets the built artifacts associated with this commit and generates a named release which is deployed on pre-release for testing.
8. In the release GitHub interface, edit the newly generated release, uncheck the This is a pre-release checkbox.
9. The CI gets the built artifacts associated with this commit and generates a named release which is deployed on release.
10. Create a commit:
    1. to promote the documentation website from future to current.
    2. to update the SQL schema with alterations from the previous release.

Hotfix Release​

​ In case of a blocking issue (following a distribution release) on the release environment that requires an immediate fix: ​

1. Create a branch on the last release tag with the following scheme: hotfix/{last_distribution-version}.{last_patch_number + 1}.
2. Development of the fix is done on this branch.
3. After each commit on this branch, the CI creates an unstable tag & release which is not deployed on testing environment (testing must be done on an ad hoc environment manually created).
4. Push a tag on the branch last commit using the branch distribution version with a -hotfix suffix.
5. The CI gets the built artifacts associated with this commit and generates a named pre-release which is deployed on pre-release for testing.
6. In the release GitHub interface, edit the newly generated release, uncheck the This is a pre-release checkbox.
7. The CI gets the built artifacts associated with this commit and generates a named release which is deployed on release.
8. Merge the hotfix branch on main branch (and adapt the changes if they are not compatible with the current main branch).

Status​

Draft

Context​

• Logs are a critical tool for operating any software system, enabling observability of the system.
• Following 12 Factor Apps principles, providing the needed components and tools to be able to configure logging and monitoring should not be the responsibility of the software components

Decision​

Therefore

• Each component of the system use Structured logging using documented and standardised JSON format for its logs
• Logs are always emitted to stdout of the process the component is part of

Consequences​

• The schema of the logged items should be properly documented in a JSON schema
• It is the responsibility of the node operator to consume the logs and process them
• We use existing libraries to provide needed log infrastructure, like slog for Rust