Ori Pomerantz


This document is a tutorial. For reference on the retesteth options, look here.

Retesteth through the Web

The easiest way to run the tests is through the web interface.

Request Helper

To run an existing state test, you can use the request helper. You set these parameters:

Parameter Meaning Sample Value
GeneralStateTests/ test suite stExample
--singletest name of test add11
--clients client to use t8ntool (the value for geth)
--singlenet fork to use Berlin
--vmtrace trace to produce raw
--verbosity log verbosity none

When a test file contains multiple tests you can restrict which ones you’ll run with the -d, -g, and -v parameters.

Request Single File

You can run a single test, either state test or blockchain test, using the request single file option. You specify the test type and then upload a test file. Here are the parameters:

Parameter Meaning Sample Value
-t test suite test type (state or blockchain)
--testfile The file to test you upload this file
--clients client to use t8ntool (the value for geth)
--vmtrace trace to produce raw
--filltests test file type see below

If --filltests is set to none, you need to upload a generated test file. You can find those here (for state tests), and here (for blockchain tests).

If --filltests is set to filltests then you can upload a filler test file, which you can write yourself. This is documented in this tutorial (for state tests) and this one (for blockchain tests).

Custom Command

The command line parameters for retesteth are documented here. You can use this option to run whatever parameters you want.

Retesteth in a Docker Container

If you want to run the tests locally you can run retesteth inside a Docker container.

These directions are written using Debian Linux 11 on Google Cloud Platform (using a 20 GB disk - the default 10 GB is not enough), but should work with minor changes on any other version of Linux running anywhere else with an Internet connection.

  1. Install docker. You may need to reboot afterwards to get the latest kernel version.

    sudo apt install -y wget docker
  2. Download the latest retesteth docker image

  3. Load the docker image:

    sudo docker image load --input dretesteth.tar
  4. Download the script.

    chmod +x
  5. Download the tests:

    git clone --branch develop
  6. Run a test. This has two purposes:

    • Create the retesteth configuration directories in ~/tests/config, where you can modify them.
    • A sanity check (that you can run tests successfully).
    sudo ./ -t GeneralStateTests/stExample -- \
     --testpath ~/tests --datadir /tests/config

    The output should be similar to:

    Running tests using path: /tests
    Running 1 test case...
    Retesteth config path: /tests/config
    Active client configurations: 't8ntool '
    Running tests for config 'Ethereum GO on StateTool' 2
    Test Case "stExample": (1 of 1)
    *** No errors detected
    *** Total Tests Run: 12


    The /tests directory is referenced inside the docker container. It is the same as the ~/tests directory outside it.

    If you get the following error:

    Tests folder does not exists, creating test folder: '/tests/GeneralStateTests/stExample'
    WARNING: /tests/src/GeneralStateTestsFiller/stExample does not exist!
    WARNING: stExample no tests detected in folder!
    Running tests for config 'Ethereum GO on StateTool' 2
    *** No errors detected
    WARNING: /tests/src/GeneralStateTestsFiller does not exist!
    *** Total Tests Run: 0

    try moving the file and cloned tests folder to your home directory (~).

  7. To avoid having to run with sudo all the time, add yourself to the docker group and open a new console.

    sudo usermod -a -G docker `whoami`

How Does This Work?

A docker container is similar to a virtual machine, except that it doesn’t run a separate instance of the operating system inside itself so it takes far less resources. One of the features of docker is that it can mount a directory of the host computer inside its own file system. The --testpath parameter to tells it what directory to mount, in this case ~/tests which you just cloned from github. It mounts it as /tests inside the container.

By default the retesteth configuration files are in ~/.retesteth. However, that directory is not accessible to us outside the docker. Instead, we use --datadir /tests/config to tell it to use (or create) the configuration in what appears to us to be ~/tests/config, which is easily accessible.

Test Against Your Client

There is an instance of geth inside the docker container that you can run tests against. However, unless you are specifically developing tests what you want is to test your client. There are several ways to do this:

  • Replace the geth inside the docker.
  • Keep the client on the outside and keep the configuration files intact
  • Put your client, and any prerequisites, inside the docker and change the configuration files
  • Keep your client on the outside and connect to it through the network and change the configuration files

When we ran the test in the previous section we also created those configuration files in ~/tests/config, but they were created as being owned by root. If you need to edit them, change the permissions of the config files. To change the configuration files to your own user, run this command:

sudo find ~/tests/config -exec chown $USER {} \; -print

If you look inside ~/tests/config, you’ll see a directory for each configured client. Typically this directory has these files:

  • config, which contains the configuration for the client:
    • The communication protocol to use with the client (typically TCP)
    • The address(es) to use with that protocol
    • The forks the client supports
    • The exceptions the client can throw, and how retesteth should interpret them. This is particularly important when testing the client’s behavior when given invalid blocks.
  •, which starts the client inside the docker image
  •, which stops the client instance(s)
  • genesis, a directory which includes the genesis blocks for various forks the client supports. If this directory does not exist for a client, it uses the genesis blocks for the default client.

Click here for additional documentation. Warning: This documentation may not be up to date

Replace geth Inside the Docker

If you want to test a modified version of geth, you can just build it inside the docker container. To do so, recreate the docker container. Before you run sudo ./ build edit the line that specifies the geth repository. Use whatever repository you use to store your modified version of geth.

Client Outside the Docker, Keep Configuration Files Intact

If you want to run your client outside the docker without changing the configuration, these are the steps to follow.

  1. Make sure that the routing works in both directions (from the docker to the client and from the client back to the docker). You may need to configure network address translation.

  2. Run your client. Make sure that the client accepts requests that don’t come from localhost. For example, to run geth use:

    geth --http --http.addr retesteth

    To run besu use:

    docker run -p 8545:8545 -p 13001:30303 \
         hyperledger/besu:latest retesteth --rpc-http-port 8545 \
         --host-allowlist '*'
  3. Run the test the same way you would for a client that runs inside docker, but with the addition of the --nodes parameter. Also, make sure the --clients parameter is set to the client you’re testing.

    ./ -t BlockchainTests/ValidBlocks/VMTests -- \
       --testpath ~/tests --datadir /tests/config --clients geth \
       --nodes \<ip\>:\<port, usually defaults to 8545\>

Client Inside the Docker, Modify Configuration Files

If you want to run your client inside the docker, follow these steps:

  1. Move the client into ~/tests, along with any required infrastructure (virtual machine software, etc). If you just want to test the directions right now, you can download geth here.

  2. Modify the appropriate to run your version of the client instead. For example, you might edit ~/tests/config/geth/ to replace geth with /tests/geth in line ten if you put your version of geth in ~/tests.

  3. Run the tests, adding the --clients <name of client> parameter to ensure you’re using the correct configuration. For example, run this command to run the virtual machine tests on geth:

    ./ -t BlockchainTests/ValidBlocks/VMTests -- --testpath \
    ~/tests --datadir /tests/config --clients geth

Client Outside the Docker, Modify Configuration Files

If you want to run your client outside the docker and specify the connectivity in the configuration files, these are the steps to follow:

  1. Create a client in ~/tests/config that doesn’t have and Typically you would do this by copying an existing client, for example:

    mkdir ~/tests/config/gethOutside
    cp ~/tests/config/geth/config ~/tests/config/gethOutside
  2. If you want to specify the IP address and port in the config file, modify the host in the socketAddress to the appropriate remote address. This address needs to work with the JSON over RPC test protocol.

    For example,

       "name" : "Ethereum GO on TCP",
       "socketType" : "tcp",
       "socketAddress" : [ "" ],
  3. Make sure that the routing works in both directions (from the docker to the client and from the client back to the docker). You may need to configure network address translation.

  4. Run your client. Make sure that the client accepts requests that don’t come from localhost. For example, to run geth use:

    geth --http --http.addr retesteth
  5. Run the test the same way you would for a client that runs inside docker:

    ./ -t BlockchainTests/ValidBlocks/VMTests -- \
        --testpath ~/tests --datadir /tests/config --clients gethOutside

Running Multiple Threads

To improve performance you can run tests across multiple threats. To do this:

  1. If you are using start multiple nodes with different ports
  2. Provide the IP addresses and ports of the nodes, either in the config file or the –nodes parameter
  3. Run with the parameters -j <number of threads>.

Using the Latest Version

The version of retesteth published as a docker file may not have the latest updates. If you want the latest features, you need to build an image from the develop branch yourself:

  1. Install docker.

    sudo apt install -y wget docker
  2. Download the script and the Dockerfile. Make sure to do this in an otherwise empty directory, because the docker builder copies everything in or below the directory where Dockerfile is located.

    mkdir ~/retestethBuild
    cd ~/retestethBuild
    chmod +x
  3. Modify the RUN git clone line in the Dockerfile for repo “retesteth” to change branch -b from master to develop. Do not modify repo branches for “winsvega/solidity” [LLLC opcode support] and “go-ethereum”.

  4. Build the docker image yourself:

    sudo ./ build


    This is a slow process. It took me about an hour on a GCP e2-medium instance.


In most cases people don’t start their own client from scratch, but modify an existing client. If the existing client is already configured to support retesteth, you should now be able to run tests on a modified version to ensure it still conforms to Ethereum specifications. If you are writing a completely new client, you still need to implement the RPC calls that retesteth uses and to write the appropriate configuration (config,, and for it.

There are several actions you might want to do with retesteth beyond testing a new version of an existing client. Here are links to documentation. Note that it hasn’t been updated in a while, so it may not be accurate.

If you want to write your own tests, read the next tutorial.