Skip to main content
As discussed in the setup section, there are two ways to setup the node and so accordingly these instructions are below:
  1. Run with Docker Compose - (Recommended method - Beginner)
  2. Build & Run from Source - (Optional method - Advanced)

1. Steps to Run Node with Docker:

1

Clone Repo

Note: The current BRP-Refiner stable version is 0.6.1. For the latest updates or additional details, please check the announcements or discussions in Discord.
Clone the covalenthq/refiner repo.
2

Check global environment variables

3

Set local environment variables

Note: .envrc.local overrides any env vars set with .envrc on calling direnv allow . Note: When passing the private key into the env vars as above please remove the 0x prefix so the private key env var has exactly 64 characters.
  • BLOCK_RESULT_OPERATOR_PRIVATE_KEY: Your personal Block Result Producer (BRP) operator private key
  • AGENT_KEY & PROOF.OUT File: Your personal web3.storage Agent key & Delegation file use by the ipfs-pinner service
  • IPFS_PINNER_URL: Service (ipfs-pinner) used by rudder to access IPFS assets like Block Specimens (service is automatically invoked and run with the in repo docker compose files eg: docker-compose-mbeam.yml)
  • EVM_SERVER_URL: Service (evm-server) used by rudder for stateless execution of Block Specimens into indexable (queryable) Block Results

Start Services

1

Load env vars into the shell.

2
Copy over the delegation proof file to ~/.ipfs repo. You should have gotten this from Covalent in addition to the W3_AGENT_KEY value.
3

Start all 4 services in the background for Moonbeam

NOTE: On a system where an das-pinner instance is already running, check the instruction in the Appendix to run refiner docker alongside.
Monitor the logs for Block Result submissions.
Check step 7 in Run BRP from source section below for a successful Refiner stack run log output with performance metrics.

Deployment As Service Unit:

Here you can find an example of a systemd service unit file that can be used to auto-start/restart of the docker-compose service for Refiner.
After adding the env vars in their respective fields in the service unit file, enable the service and start it.
Note 1: When passing the private key into the env vars as above please remove the 0x prefix so the private key env var has exactly 64 characters. Note 2: In order to run docker compose as a non-root user for the above shown service unit you need to create a docker group (if it doesn’t exist) and add the user (say) “blockchain” to the docker group
Hurray! You have Successfully implemented the BRP Node using Docker Compose

2. Steps to Run Node with Source:

Here, to run the node from source we need to manually run all the three services step by step, follow this guide for implementing
  1. EVM-Server
  2. DAS-Pinner
  3. Block Results Producer

1. Run EVM-Server

The EVM-Server is a stateless EVM block execution tool. It’s stateless because in Ethereum nodes like geth, it doesn’t need to maintain database of blocks to do a block execution or re-execution. The evm-server service transforms Block Specimens to Block Results entirely on the input of underlying capture of Block Specimen data.
1

Clone & Build evm-server

Clone covalenthq/erigon that contains the fork for this particular stateless execution function, Current stable version is v1.3.2-evm, build the evm-server binary
2

Run the generated binary

Note: evm-server occupies the port 3002 on your local at http://127.0.0.1:3002”, so make sure this port in not occupied by any other service prior to start.

2. Run DAS-Pinner

DAS-Pinner serves as the interface to the storage layer of the network. It’s a lightweight IPFS service that stores and pins data on the IPFS network using web3.storage. It is designed to work seamlessly with the DAS Light-Client, enabling the retrieval of data from the IPFS network and its verification using the DAS protocol.
It is meant for uploading/fetching network artifacts (like block specimens and block results or any other processed block data) files of the Covalent Decentralized Network.
1

Clone & Build ewm-das(pinner)

Clone covalenthq/ewm-das and build the pinner server binary
2

Set the environment variable required by 

ipfs-pinner by getting W3_AGENT_KEY & Proof.out from web3.storage and adding it to an .envrc file.
3

Start the das-pinner server

You should get the “w3 agent key” and “delegation proof file” from Covalent. After that, start the ipfs-pinner server.
Note: das-pinner occupies the port 5080 on your local at http://127.0.0.1:5080/api/v1, so make sure this port in not occupied by any other service prior to start.

Debugging: Repo Migrations

In case of the following error
Run the following migrations

3. Run Block Results Producer (BRP):

BRP is the primary orchestrator and supervisor for all transformation pipeline processes that locates a source Covalent Network data object to apply a tracing/execution/transformational rule to and outputs a new object generated from using such a rule. Both the source and output are available through a decentralized storage service such as a wrapped IPFS node. A transformation-proof transaction is emitted, confirming that it has done this work along with the output content ids (ipfs) access URL. To define what these three components are: Source: The Block Specimen that serves as an input to the Refiner. Proof transactions made earlier to a smart contract with the respective cids (content ids) are where the source is checked. Rule: A transformation plugin (or server) that can act on the Block Specimen (source). These can be compared to blueprints that have been shown to produce the data objects needed. Furthermore, anyone can create these rules to get a desired data object. Rule (or server) versions thus need to exist, tied to the decoded block specimen versions they are applied on. Target: The output generated from running the rule over the object that came from the source that is the block result.

Steps to Run BRP:

1

Clone Refiner

Clone the refiner repo
2
Get your BLOCK_RESULT_OPERATOR_PRIVATE_KEY that has GLMR tokens for Moonbeam and is already whitelisted as Block Result Producer operator. Set the following environment variables for the local refiner by creating an .envrc.local file
  • Copy paste the environment variables into this file
3

Load envrc file

Call to load .envrc.local + .envrc files with the command below and observe the following output, make sure the environment variables are loaded into the shell.
  • Once the env vars are passed into the .envrc.local file and loaded in the shell with direnv allow ., build the refiner application for the prod env i.e moonbeam mainnet or dev env for moonbase alpha as discussed before.
4

Get required dep for Refiner app

Get all the required dependencies and build the refiner app.
5

Build Refiner app

  • For moonbeam mainnet.
  • For moonbase alpha.
6

Start the Refiner

Start the refiner application and execute the proof-chain block specimen listener call which should run the Refiner pipeline pulling Block Specimens from IPFS using the cids read from recent proof-chain finalized transactions, decoding them, and uploading and proofing Block Results while keeping a track of failed ones and continuing (soft real-time) in case of failure.The erlang concurrent fault tolerance allows each pipeline to be an independent worker that can fail (for any given Block Specimen) without crashing the entire pipeline application. Multiple pipeline worker children threads continue their work in the synchronous queue of Block Specimen AVRO binary files running the stateless EVM binary (evm-server) re-execution tool.
7

Check logs

Check logs for any errors in the pipeline process and note the performance metrics in line with execution. Checkout the documentation on what is being measured and why here.
Alternatively - check proof-chain logs for correct block result proof submissions and transactions made by your block result producer. For moonbeam. For moonbase.
Note: For any issues associated with building and re-compiling execute the following commands, that cleans, downloads and re-compiles the dependencies for refiner.
If you got everything working so far. Congratulations! You’re now a Refiner operator on the CXT Network. Set up Grafana monitoring and alerting from links in the additional resources section.

Troubleshooting

To avoid permission errors with ~/.ipfs folder execute the following in your home directory.
To avoid netscan issue execute the following against ipfs binary application.
To avoid issues with ipfs-pinner v1.1.0, a small repo migration for the local ~/.ipfs directory may be needed from - here. For linux systems follow the below steps.

Bugs Reporting Contributions

Please follow the guide in docs contribution guidelines for bug reporting and contributions.

Scripts

In order to run the Refiner docker compose services as a service unit. The example service unit file in docs should suffice. After adding the env vars in their respective fields in the service unit file, enable the service and start it.
Note:: To run docker compose as a non-root user for the above shown service unit, you need to create a docker group (if it doesn’t exist) and add the user “blockchain” to the docker group.

Appendix

Run With Existing IPFS-Pinner Service

On a system where an ipfs-pinner instance is already running, use this modified .envrc.local and docker-compose-mbeam.yml
Note: When passing the private key into the env vars as above please remove the 0x prefix so the private key env var has exactly 64 characters.
and start the refiner and evm-server services: