Setup BSP Node

Prerequisites for Operator nodes

1. CXT Stake

   Before you get started, make sure you have:

   • the minimum self-stake CXT available, currently 175K CXT
   • your operator address is added and enabled on the operator dashboard

   
2. GLMR for gas

   Glimmer (GLMR) is needed to pay for gas on Moonbeam in order to send proofs of block specimens and block results. This process currently costs 5 GLMR per day. The balance should be held at the same address as the operator address.

Install Dependencies

MacOS 12.x (M1/Intel) Installation requirements

Install XCode

• You must also Install brew (for mac m1/intel)

This installs all the minimum necessary terminal/command-line tools to get started with software development on a mac.

Complete Installation Time: 1-1.5 hrs depending on your machine and network.

Install Git, Golang, Redis.

• Git is used as the source code version control manager across all our repositories.
• Go is the programming language that is used to develop on go-ethereum and bsp patches, the agent given below is also entirely written in Go.
• Redis is our in-memory database, cache and streaming service provider.

Installation steps are given for various types of OS follow the one suits best for you.

Current Go Version: Go 1.22 or later

MacOS 12.x (M1/Intel)

 brew install git go redis 

Debian/Ubuntu

wget https://golang.org/dl/go1.22.5.linux-amd64.tar.gz
tar -xvf go1.22.5.linux-amd64.tar.gz 
sudo mv go /usr/local 

echo "" >> ~/.bashrc 
echo 'export GOPATH=$HOME/go' >> ~/.bashrc 
echo 'export GOROOT=/usr/local/go' >> ~/.bashrc 
echo 'export GOBIN=$GOPATH/bin' >> ~/.bashrc 
echo 'export PATH=$PATH:/usr/local/go/bin:$GOBIN' >> ~/.bashrc 
echo 'export GO111MODULE=on' >> ~/.bashrc 
source .bashrc

# install redis and git
apt install git redis-server

Fedora

dnf install git golang redis

RHEL/CentOS

Install Git, Golang, Redis

yum install git go-toolset redis

OpenSUSE/SLES

Install Git, Golang, Redis

zypper addrepo https://download.opensuse.org/repositories/devel:languages:go/openSUSE_Leap_15.3/devel:languages:go.repo
zypper refresh
zypper install git go redis

After Installing the dependencies mentioned above according to your OS needs, Follow these steps to setup the environment.

Let’s now start with Environment Setup before running Node, Here, you need to setup the following;

1. BSP-Geth & Lighthouse
2. DAS-Pinner Setup
3. BSP-Agent Setup

1. BSP-Geth & Lighthouse Setup :

1. Clone Repo

   git clone https://github.com/covalenthq/bsp-geth.git
   cd bsp-geth
   

2. Build geth

   Build geth (install go if you don’t have it) and other geth developer tools from the root repo with (if you need all the geth related development tools run make all )

   make geth
   

3. Start redis (our streaming service) with the following

   Build geth (install go if you don’t have it) and other geth developer tools from the root repo with (if you need all the geth related development tools run make all )

   brew services start redis
   ==Successfully started `redis` (label: homebrew.mxcl.redis)
   
   On Linux:
   systemctl start redis
   

4. Start redis-cli

   Start redis-cli in a separate terminal so you can see the encoded bsps as they are fed into redis streams.
   We are now ready to start accepting stream message into redis local

   redis-cli
   127.0.0.1:6379>
   

5. • Go back to ~/bsp-geth and start geth with the given configuration, here we specify the replication targets (block specimen targets) with Redis stream topic key replication, in snap syncmode.
   • Prior to executing, please replace $PATH_TO_GETH_MAINNET_CHAINDATA with the location of the mainnet snapshot that you will be downloading in the next step via lighthouse. Everything else remains the same as given below.
   • Please review the flags and use them according to your system need
     Each of the bsp flags and their functions is described below
     --mainnet - lets geth know which network to synchronize with, this can be --ropsten, --goerli etc
     --discv5=true- Enables the node to participate in the Ethereum network’s Discovery v5 protocol, which is used for finding peers.
     --txlookuplimit - Disables the limitation on how far back in the chain transactions are indexed, allowing for retrieval of transaction data from any block in the node’s history.
     --cache - Sets the memory allocation for the node’s cache in megabytes, which is used to improve performance by holding recently accessed data.
     --syncmode - this flag is used to enable different syncing strategies for geth and a full sync allows us to execute every block from block 0; while snap allows us to execute from live blocks
     --light.ingress - Limits the bandwidth in kilobytes per second that the node dedicates to serving light clients on Ingress (incoming traffic).
     --light.egress - Limits the bandwidth in kilobytes per second that the node dedicates to serving light clients on Egress (outgoing traffic).
     --light.maxpeers - Sets the maximum number of light client peers that the node can connect to.
     --http - Enable the HTTP-RPC server
     --http.addr - HTTP-RPC server listening interface (default: localhost)
     --http.api - API’s offered over the HTTP-RPC interface(default: eth,net,web3)
     --http.vhosts - Allows all virtual hostnames to access the HTTP-RPC server, effectively disabling the host-based security check.
     --ws - Enable the WS-RPC server
     --ws.addr - WS-RPC server listening interface (default: localhost)
     --ws.api - API’s offered over the WS-RPC interface (default: eth,net,web3)
     --ws.origins - Origins from which to accept WebSocket requests
     --datadir - specifies a local datadir path for geth (note we use “bsp” as the directory name with the Ethereum directory), this way we don’t overwrite or touch other previously synced geth libraries across other chains
     --authrpc.jwtsecret - Specifies the path to the JWT secret file used for authentication in the RPC API, providing an additional layer of security by requiring tokens for access.
     --replication.targets - this flag points to redis,  and lets the bsp know where and how to send the bsp messages (this flag will not function without the usage of either one or both of the flags below if both are selected a full block-replica is exported)
     --replica.result - this flag lets the bsp know if all fields related to the block-result specification need to be exported (if only this flag is selected the exported object is a block-result)
     --replica.specimen - this flag lets the bsp know if all fields related to the block-specimen specification need to be exported (if only this flag is selected the exported object is a block-specimen)
     —replica.blob - Enables the extraction of blob specimens as part of the complete block specimen export, which includes state specimens and transaction receipts, collectively referred to as a block replica. This flag is used to include additional block content, particularly useful for nodes participating in more extensive data analysis or archiving, ensuring comprehensive block data is available for these purposes.
     --log.file - specifies the file location where the log files have to be placed. In case of error (like permission errors), the logs are not recorded in files.

   Sample Service systemmd file for BSP-Geth can be found here.

   ./build/bin/geth 
   	--mainnet \
   	--log.debug \
       --v5disc=true \
       --txlookuplimit=0 \
       --cache=8192 \
       --syncmode=snap \
       --light.ingress=200 \
       --light.egress=200 \
       --light.maxpeers=3 \
       --http=true \
       --http.addr=localhost \
       --http.api=admin,debug,eth,net,web3,txpool \
       --http.vhosts=* \
       --ws=true \
       --ws.addr=localhost \
       --ws.api=admin,debug,eth,net,web3,txpool \
       --ws.origins=* \
       --datadir=$PATH_TO_GETH_MAINNET_CHAINDATA \
       --authrpc.jwtsecret= $PATH_TO_GETH_MAINNET_CHAINDATA/geth/jwtsecret \
       --replication.targets=redis://127.0.0.1:6379/0?topic=replication \
       --replica.result \
       --replica.specimen \
   	--replica.blob \
       --log.file=$PATH_TO_GETH_MAINNET_CHAINDATA/geth/log.log
   

6. Install Lighthouse

   bsp-geth also needs a consensus client. We’ve tested and found the lighthouse to be stable and works well. Goto the installation instructions to install it. There are multiple versions of lighhouse available Recommended is to use Pre-built binaries -> releases on github -> Then find a version where the Breaking Changes are none (i.e currently Summer Smith Prime v7.0.1) or you can ask us on the discord for assistance with the version. Then run the lighthouse using:

   Sample Service systemmd file for lighthouse can be found here.

   lighthouse bn \
       --network mainnet \
       --execution-endpoint http://localhost:8551 \
       --execution-jwt $PATH_TO_GETH_MAINNET_CHAINDATA/geth/jwtsecret \
       --checkpoint-sync-url https://mainnet.checkpoint.sigp.io \
       --disable-deposit-contract-sync
   

7. Wait for the eth blockchain to sync to the tip. Connect to the node’s ipc instance to check how far the node is synced

   ./build/bin/geth attach $PATH_TO_GETH_MAINNET_CHAINDATA/geth.ipc
   

8. Once connected, wait for the node to reach the highest known block to start creating live block specimens

   Welcome to the Geth JavaScript console!
   
   instance: Geth/v1.15.11-stable-94f6ae31/linux-amd64/go1.24.0
   at block: 22433567 (Wed May 07 2025 18:38:59 GMT+0000 (UTC))
   datadir: /scratch/bsp
   modules: admin:1.0 debug:1.0 engine:1.0 eth:1.0 miner:1.0 net:1.0 rpc:1.0 txpool:1.0 web3:1.0
   
   To exit, press ctrl-d or type exit
   > eth.syncing
   {
       currentBlock: 22433567,
       healedBytecodeBytes: 0,
       healedBytecodes: 0,
       healedTrienodeBytes: 0,
       healedTrienodes: 0,
       healingBytecode: 0,
       healingTrienodes: 0,
       highestBlock: 22433567,
       startingBlock: 22432567,
       syncedAccountBytes: 0,
       syncedAccounts: 0,
       syncedBytecodeBytes: 0,
       syncedBytecodes: 0,
       syncedStorage: 0,
       syncedStorageBytes: 0
   }
   

9. Now wait till you see a log from the terminal here with something like this:

   INFO [04-11|16:35:48.554|core/chain_replication.go:317]             Replication progress                     sessID=1 queued=1 sent=10960 last=0xffc46213ccd3c55b75f73a0bc29c25780eb37f04c9f2b88179e9d0fb889a4151
   INFO [04-11|16:36:04.183|core/blockchain_insert.go:75]              Imported new chain segment               blocks=1       txs=63         mgas=13.147  elapsed=252.747ms    mgasps=52.015   number=10,486,732 hash=8b57c8..bd5c79 dirty=255.49MiB
   INFO [04-11|16:36:04.189|core/block_replica.go:41]                  Creating Block Specimen                  Exported block=10,486,732 hash=0x8b57c8606d74972c59c56f7fe672a30ed6546fc8169b6a2504abb633aebd5c79
   INFO [04-11|16:36:04.189|core/rawdb/chain_iterator.go:338]          Unindexed transactions                   blocks=1       txs=9          tail=8,136,733 elapsed="369.12µs"
   

This can take a few days or a few hours depending on if the source chaindata is already available at the datadir location or live sync is being attempted from scratch for a new copy of blockchain data obtained from syncing with peers. In the case of the latter the strength of the network and other factors that affect the Ethereum network devp2p protocol performance can further cause delays. Once blockchain data state sync is complete and eth.syncing returns false. You can expect to see block-specimens in the redis stream. The following logs are captured from bsp-geth service as the node begins to export live Block Specimens.

The last two logs show that new block replicas containing the block specimens are being produced and streamed to the redis topic replication. After this you can check that redis is stacking up the bsp messages through the redis-cli with the command below (this should give you the number of messages from the stream)

$ redis-cli
127.0.0.1:6379>  xlen replication
11696

If it doesn’t - the BSP producer isn’t producing messages! In this case please look at the logs above and see if you have any WARN / DEBUG logs that can be responsible for the disoperation. For quick development iteration and faster network sync - enable a new node key to quickly re-sync with the ethereum network for development and testing by going to the root of go-ethereum and running the bootnode helper.

NOTE: To use the bootnode binary execute make all in place of make geth, this creates all the additional helper binaries that bsp-geth ships with.

./build/bin/bootnode -genkey ~/.ethereum/bsp/geth/nodekey

2. DAS-Pinner Setup

DAS-Pinner serves as the interface to the storage layer of the network. It’s a lightweight IPFS service that stores and pins data on the IPFS network using web3.storage. It is designed to work seamlessly with the DAS Light-Client, enabling the retrieval of data from the IPFS network and its verification using the DAS protocol.
It is meant for uploading/fetching network artifacts (like block specimens and block results or any other processed block data) files of the Covalent Decentralized Network.

Current : Go Version: go1.22 or later , this could change later in time please cross check with operator-releases channel on discord for correct versions.

1. Clone & Build

   Clone and build das-pinner (in a separate folder) We store the block specimens in the ipfs network. Bsp-agent interacts with the das-pinner server to handle the storage/retrieval needs of the network.

   $ cd ..
   $ git clone https://github.com/covalenthq/ewm-das.git 
   $ cd ewm-das
   $ git checkout tags/v0.17.6 ## update if newer release is announced
   $ ./install-trusted-setup.sh
   $ make build-pinner
   

2. Setting up w3cli

   Software Prerequisite - Node.Js version 18 or higher and npm version 7

   • Install w3cli Version 7.9.1: To install version 7.9.1 of the w3cli tool globally on your system, run the following command:

   npm install -g @web3-storage/w3cli@v7.9.1
   

   • Troubleshooting: If you encounter any issues, such as missing dependencies, try clearing the npm cache and reinstalling w3cli:

   npm cache clean --force
   npm uninstall -g @web3-storage/w3cli
   npm install -g @web3-storage/w3cli@v7.9.1
   

   Additionally, you can manually install any missing dependencies.

   npm install -g crypto-random-string 
   

3. Get the agent key, did and delegation proof from Covalent, then you can run the ipfs-pinner ipfs-pinner can be run as a server and allows two functionalities currently -  /get and /upload With this, the ipfs-pinner is setup and can upload/fetch network artifacts.

   Sample Service systemmd file for ipfs-pinner can be found here.

   $ cat .envrc
   export AGENT_KEY="<<ASK_ON_DISCORD>>"
   export W3_DELEGATION_FILE="<< ASK_ON_DISCORD>>"
   
   $ ./build/bin/server -w3-agent-key <AGENT_KEY> -w3-delegation-file <W3_DELEGATION_FILE>
   

3. BSP-Agent Setup:

Install Dependencies

• Install direnv

MacOS 12.x (M1/Intel)

brew install direnv

Debian/Ubuntu

apt install direnv

Fedora

dnf install direnv

RHEL/CentOS

yum install direnv

SLES/OpenSUSE

zypper install direnv

direnv manages and controls sensitive information for the agent, such as Ethereum private keys for operator accounts on the Covalent Network and Redis access passwords. This is crucial since these applications, exposed on HTTP ports, must not log sensitive data.

To enable direnv on your machine add these to your ~./bash_profile or ~./zshrc depending on which you use as your default shell after installing it using brew.

For bash users - add the following line to your ~/.bashrc
For zsh users - add the following line to your ~/.zshrc

bash

eval "$(direnv hook bash)"

After adding this line do not forget to source your bash / powershell config with the following, by running it in your terminal.

bash

source ~/.bashrc