Published using Google Docs
Solana Mainnet-Beta Cluster Restart 25 February 2023
Updated automatically every 5 minutes

This document has been deprecated – please refer to new instructions at https://docs.google.com/document/d/e/2PACX-1vQVMKVqh1o-qF4Z8ATM-Qxa_FhT96qoCmqGHFroq4e1EbrHkfg-H8Q8Z_FRnX3mYOxRgBbK6yzDols_/pub 

Solana Mainnet-Beta Cluster Restart 25 February 2023

This document has been deprecated – please refer to new instructions at https://docs.google.com/document/d/e/2PACX-1vQVMKVqh1o-qF4Z8ATM-Qxa_FhT96qoCmqGHFroq4e1EbrHkfg-H8Q8Z_FRnX3mYOxRgBbK6yzDols_/pub 


Restart details:

version

v1.13.6

slot

179526408

Shred version

12743

Expected bank hash

JCfD7vqjQWKV8Vusx7degWczh55WkQ2cEifGTzCkXdKk


Step 0: Check for nonvote transactions

If you’d like to verify there are only votes past 179526408, you can run the following

solana-ledger-tool print --starting-slot 179526408 -v

Results should return Vote counts only

Summary of Programs:

Vote111111111111111111111111111111111111111 : 10273021 [your number of votes may differ]

If you see anything besides Votes, please report it on Discord and do not continue


Step 1: Create a snapshot at slot 179526408

You need to stop your validator process if it is still running before creating your snapshot

Use the ledger tool to create a new snapshot at slot 179526408, replacing the two instances of <ledger path> to your actual ledger path:

$ solana-ledger-tool --ledger <ledger path> create-snapshot 179526408 <ledger path> --hard-fork 179526408

Add --snapshot-archive-path <PATH_TO_SNAPSHOTS> before “create-snapshot” if you have a separate snapshot dir, and --accounts <PATH_TO_ACCOUNTS> before –hard-fork if you have a separate accounts dir

 

The final line of output should be “Shred version: 12743”, and this snapshot file should now exist:

snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst

Check your ledger directory to ensure that you have no snapshot newer than snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst Newer snapshots should be removed. Snapshots older than snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst should NOT be removed.

NOTE: You may need to move the created snapshot from your ledger directory to your snapshots directory if you have a custom snapshot directory

NOTE: If you receive “Error: Slot 179526408 is not available”, please see appendix


Step 2: Adjust your validator command-line arguments, temporarily for this restart to include:

(--known-validators aren’t needed if you have your own local snapshot and have set –no-genesis-fetch as your validator won’t be downloading anything, you can omit those arguments in this case)

--wait-for-supermajority 179526408 \

--no-snapshot-fetch \

--no-genesis-fetch \

--expected-bank-hash JCfD7vqjQWKV8Vusx7degWczh55WkQ2cEifGTzCkXdKk \

--expected-shred-version 12743 \

--known-validator CMPSSdrTnRQBiBGTyFpdCc3VMNuLWYWaSkE8Zh5z6gbd \

--known-validator 6WgdYhhGE53WrZ7ywJA15hBVkw7CRbQ8yDBBTwmBtAHN \

--known-validator 7Np41oeYqPefeNQEHSv1UDhYrehxin3NStELsSKCT4K2 \

--known-validator GwHH8ciFhR8vejWCqmg8FWZUCNtubPY2esALvy5tBvji \

--known-validator Ninja1spj6n9t5hVYgF3PdnYz2PLnkt7rvaw3firmjs \

--known-validator PUmpKiNnSVAZ3w4KaFX6jKSjXUNHFShGkXbERo54xjb \

--known-validator DE1bawNcRJB9rVm3buyMVfr8mBEoyyu73NBovf2oXJsJ \

(Remove the previous value of “--expected-shred-version“ if present). 

Once the cluster restarts and normal operation resumes, remember to remove --wait-for-supermajority and --expected-bank-hash before the next update or restart. They are only required for the restart. You can also go back to your old known-validators at that point.

(see next page for step 4)


Step 3: Install version 1.13.6

solana-install init 1.13.6

Step 4: Start your validator

As it boots, it will load the snapshot for slot 179526408 and wait for 80% of the stake to come online before producing/validating new blocks.

To confirm your restarted validator is correctly waiting for 80% stake, look for this periodic log message to confirm it is waiting:

INFO  solana_core::validator] Waiting for 80% of activated stake at slot 179526408 to be in gossip...

And if you have RPC enabled, ask it repeated for the current slot:

$ solana --url http://127.0.0.1:8899 slot

Any number other than 179526408 means you did not complete the steps correctly.

Once started you should see log entries for “Activate stake” visible in gossip and “waiting for 80% of stake” to be visible. You can track these to see how stake progresses.

If you couldn’t produce your snapshot locally follow appendix on next page below


Appendix: Resolution if you did not preserve your ledger or does not contain a reachable slot 179526408

NOT RECOMMENDED - this resolution should only be attempted if your ledger/ directory is unavailable or you are unable to produce a snapshot for 179526408.

ONLY IF your ledger history is corrupt, otherwise unavailable or does not contain a reachable slot 179526408, follow these instructions to get a new snapshot:

Your validator will need to download a new snapshot from one of the known validators. Alternative snapshot download methods are also provided further below. A snapshot will be verified as valid by the bank hash in the arguments below.

For this you need to remove –no-snapshot-fetch if present.

Add these arguments to restart:
        
--wait-for-supermajority 179526408 \

--expected-shred-version 12743 \

--expected-bank-hash JCfD7vqjQWKV8Vusx7degWczh55WkQ2cEifGTzCkXdKk \

--known-validator CMPSSdrTnRQBiBGTyFpdCc3VMNuLWYWaSkE8Zh5z6gbd \

--known-validator 6WgdYhhGE53WrZ7ywJA15hBVkw7CRbQ8yDBBTwmBtAHN \

--known-validator 7Np41oeYqPefeNQEHSv1UDhYrehxin3NStELsSKCT4K2 \

--known-validator GwHH8ciFhR8vejWCqmg8FWZUCNtubPY2esALvy5tBvji \

--known-validator Ninja1spj6n9t5hVYgF3PdnYz2PLnkt7rvaw3firmjs \

--known-validator PUmpKiNnSVAZ3w4KaFX6jKSjXUNHFShGkXbERo54xjb \

--known-validator DE1bawNcRJB9rVm3buyMVfr8mBEoyyu73NBovf2oXJsJ \

Additional snapshot sources:

If you follow this add back --no-snapshot-fetch, you can remove old snapshots, none should have a higher slot than the one you download.

By solanacompass.com

wget https://dn4xk96mcxun5.cloudfront.net/snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst

wget http://20230225soloutage.s3-website.eu-central-1.amazonaws.com/snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst

wget http://5.199.161.213/snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst

By Mint's Validator

NYC:
wget http://64.185.227.122:36969/snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst

HKG:
wget http://58.153.19.142:36969/snapshot-179526408-9kUTC2pYALrYD8rb93E1mY2vG4QG8jCX132Z4CEhBHgJ.tar.zst


Appendix: For Jito-Solana Validators

If you are running a Jito-Solana client and need to rebuild / downgrade your validator to a different version of Jito-Solana, please follow the instructions below to rebuild:

```

export TAG=v1.13.6-jito

cd jito-solana

git pull

git checkout tags/$TAG

git submodule update --init --recursive

CI_COMMIT=$(git rev-parse HEAD) scripts/cargo-install-all.sh ~/.local/share/solana/install/releases/"$TAG"

```

then proceed with the instructions in the rest of this document