Update 2022/12/19: The signer registration with declarative PoolId has been decommissioned.

Update 2022/11/30: The signer registration with declarative PoolId has been deprecated and the certified PoolId is now the stable mode.

The way the Mithril nodes handle the Certification of the SPOs is evolving​

PR: New STM registration procedure #433

Issues: Implement Certification of the Mithril Verification Keys in Signer/Aggregator #455

We have released a new Mithril Signer Verification Keys Certification mechanism:

• Mithril Signer nodes running the previous version are still able to interact with the network without any further intervention
• Mithril Signer nodes that are updated from a previous version must migrate some of their stores
• This mechanism is experimental and can be activated on demand by the SPOs

Upgrade a Mithril Signer running a previous version​

The SPOs need to recompile their Signer node (as in this guide).

The data stores of the node need to be updated by running the following command:

# The path to your data stores directory, which defaults to:
DATA_STORES_DIRECTORY=/opt/mithril/stores

# Run this command to upgrade your stores:
sqlite3 ${DATA_STORES_DIRECTORY}/signer.sqlite3 "UPDATE protocol_initializer SET value = json_object('stm_initializer', json(value), 'kes_signature', null) WHERE json_extract(value, '$.stm_initializer') IS NULL;"

⚠️ If you don't update your data stores with this procedure, your node will not be able to register to the Aggregator temporarily. It should then take up to 3 epochs before it is able to successfully register its individual signatures with the Aggregator.

Hybrid Certification mode in the Mithril network​

From now, SPOs can either run their node by:

• Declaring their Cardano PoolId:

  • This is the mode that all nodes were running prior to this release
  • This mode is still the stable mode
  • We intend to deprecate this mode in the near future

• Certifying their Cardano PoolId:

  • The certification is done by providing the Mithril Signer node with KES Secret Key Path and Operational Certificate Path
  • This is an experimental mode
  • We intend to make this mode the only way of providing a PoolId in the near future
  • These PoolIds will be marked with a Verified Signer green badge in the Mithril Explorer (2 epochs after activating the Certification mode)

The setup of a Mithril Signer node with these two modes is available in this guide.

Here is an example of the Verified Signer badge displayed in the Certificate details popin:

How Keys Certification works​

We rely on the Cardano KES Keys and Operational Certificate to be able to:

• Compute automatically the PoolId from a valid Operational Certificate
• Sign the Mithril Signer Verification Key with the KES Secret Key
• Verify that the Mithril Signer Verification Key is associated to the owner of the pool

Feel free to reach out to us on the Discord channel for questions and/or help.

What is that?​

Since almost the beginning of the Mithril project, the software used to rely on a store mechanism to save its different states allowing Signers and Aggregators to resume on correct state when switched on and off. This internal store mechanism used to be a bunch of JSON files saved in a given directory. Even though this does the job it still presents flaws: data are hard to query when debugging especially when crossing data (which signers have participated in this multi-signature?). Also, data are stored in different places which can be a problem when moving these files from one place to another. We also had to imagine what would be a migration scenario in case of a structure change. Switching to a file based SQL database solves these issues.

The new release now uses SQLite stores in place of JSON file storage. This means that to continue running a Signer or an Aggregator node it is necessary to migrate from the old storage system to SQLite. This release comes with a tool to perform the migration which should be as straightforward as launching a command line (read below). The migration tool will be available only for a limited time in order to make Mithril beta testers able to migrate their existing data.

How to migrate data from old storage system to SQLite stores?​

There are 2 ways of getting the new version and the associated migration tool. Either downloading binaries from GitHub or compiling them yourself.

Downloading​

Download the new mithril-signer and mithril-signer-migrate files from the nightly builds page. Make them executable:

$> chmod +x mithril-signer*
$> ls -1F mithril-signer*
mithril-signer*
mithril-signer-migrate*

note: the suffix * appended to the the entries output above indicates the file is executable. If it is not present, ensure the chmod command does not produce any error.

Compiling​

If you used to compile your node as stated in the guide, you have to compile the migration tool as well:

$> cd mithril-signer
$> cargo build --all-targets --release
  Compiling mithril-signer v0.1.0 (/home/somebody/shared/mithril/mithril-signer)
    Finished release [optimized] target(s) in 4.56s
$> ls -1F ../target/release/mithril-signer*
../target/release/mithril-signer*
../target/release/mithril-signer.d
../target/release/mithril-signer-migrate*
../target/release/mithril-signer-migrate.d

Running the migration​

The first step is to stop the running Mithril node if any. The mithril-signer-migrate executable can perform the migration automatically once you know where your actual JSON files are located. Have a look in your configuration file (default /opt/mithril/mithril-signer.env), check the value associated with the DATA_STORES_DIRECTORY key (default to /opt/mithril/stores) and copy the path indicated here. Copy this path after the --db-dir option on the following command line:

$> ./mithril-signer-migrate automatic --db-dir /paste/the/data/stores/directory/here
Mithril Aggregator JSON → SQLite migration tool.
Migrating protocol_initializer_store data…
OK ✓
Migrating stake_store data…
OK ✓

At the end of this command, a file signer.sqlite3 (or aggregator.sqlite3 if you run an Aggregator) should be present in the specified base directory.

That should be enough, launch your upgraded mithril node.

Note: The migration executable does not remove the old JSON files from the disk.

Manual migration process​

The executable also provides a manual switch for migrating Mithril JSON store directories placed in custom directories. This is mainly intended for developers who work on tweaked environments. Each internal store has its own data structure. In order to correctly migrate and process data, the type of the store has to be given on the command line.

$> ./mithril-signer-migrate manual --help

The command above should give you all informations needed to run a custom store migration.

Feel free to reach out to us on the Discord channel for questions and/or help.

Update: The Stake Distribution computation is evolving with the release of Cardano node 8.0.0: the computation now relies on the new cardano-cli query stake-snapshot --all-stake-pools command that retrieves the Stake Distribution all at once and that is way faster. Prior versions of the Cardano node 1.35+ are backward compatible and will keep implementing the algorithm detailed below.

The way the Mithril nodes retrieve the Stake Distribution is changing​

PR: Fix Stake Distribution retrieval #499

Issue: Stake distribution discrepancy #497

We have noticed that the way the Mithril nodes computed the Stake Distribution was erroneous: the epoch that was used to make the computation was the current epoch instead of the previous epoch. This has lead to some de-synchronization between the Signers and the hosted GCP Aggregator for a few epochs.

Indeed, the Stake Distribution retrieved from the Cardano node depended on the time at which it was done: the nodes where having differents values that prevented them from being able to work together to produce valid multi-signatures. The problem is related to the epoch that is used (current epoch) to make the computation of the Stake Distribution when the cardano-cli query stake-distribution command is ran, whereas the Mithril protocol needs to work with the previous epoch.

A workaround is being implemented in this fix that will compute differently the Stake Distribution to target the previous epoch. To do so, the Stake value that is now retrieved sequentially for each pool available in the cardano-cli query stake-distribution by using the command cardano-cli query stake-snapshot --stake-pool-id **pool-id*. This guarantees that the Stake Distribution is computed deterministically on all nodes of the Mithril Network.

We will continue our efforts to enhance the way the Stake Distribution is retrieved in the future, and so that it works smoothly on the mainnet (where the numbers of pools is bigger ~3,000 vs ~100 on the preview network).

The SPOs need to recompile their Signer node in order to compute correctly the Stake Distributions on their node (as in this guide). It should then take up to 2 epochs before they are able to successfully register their individual signatures with the Aggregator.

More information about the Certificate Chain and the epochs retrieval requirements is available here.

Feel free to reach out to us on the Discord channel for questions and/or help.

The way the Signers list is computed inside a Certificate on the Mithril Aggregator is changing​

PR: Implement filtered Signers in Certificate #494

Issue: Record 'contributing' Signers only in Certificate #495

Before this change, the list of Signers displayed in the Certificate detail of the Mithril Explorer was the list of all eligible Signers of the epoch used for signing (those who have successfully registered with the Mithril Aggregator 2 epochs earlier).

Now that this change has been merged, the list of Signers displayed will only include the contributing Signers, which means only those who have successfully sent individual signatures.

Note that the already existing Certificates will not be updated as this would break the Certificate Chain and therefore would involve the bootstraping of a new Genesis Certificate.

This change is transparent to the Signer nodes runned by the SPOs and does not require any specific action from them.

Feel free to reach out to us on the Discord channel for questions and/or help.

Update: The PR has been merged and the feature is being deployed on the GCP Mithril Aggregator.

This afternoon, we plan to merge the PR that activates the Genesis Certificate feature on the GCP Mithril Aggregator​

PR: Implement Real Genesis Certificate #438

Issue: Bootstrap Certificate Chain w/ Genesis Certificate #364

This will involve some manual operations that will prevent temporarily the service to be running:

• We will have to reset the stores of the Snapshots and Certificates. This means that the Mithril Explorer will display a No snapshot available message.

• The Mithril Signers will have to wait until the next epoch #30 to be able to sign. This means that we should see the first available Snapshot 1 hour after the epoch transition.

The SPOs that are currently running a Mithril Signer will have to recompile their node in order ot take advantage of the latest improvements (such as the registration of the nodes that will take few minutes instead of few hours). However, the previously compiled node will be able to contribute to signatures.

In order to restore a Mithril Snapshot, a Mithril Client will now need access to the Genesis Verification Key by adding an environment variable when running: GENESIS_VERIFICATION_KEY=$(wget -q -O - https://raw.githubusercontent.com/input-output-hk/mithril/main/TEST_ONLY_genesis.vkey).

Feel free to reach out to us on the Discord channel for questions and/or help.