Generated Virtual Machine Tests

Location /BlockChainTests/ValidBlocks/VMTests

The VM tests aim is to test the basic workings of the VM in isolation.

This is specifically not meant to cover transaction, creation or call processing, or management of the state trie. Indeed at least one implementation tests the VM without calling into any Trie code at all.

A VM test is based around the notion of executing a single piece of code as part of a transaction, described by the exec portion of the test. The overarching environment in which it is executed is described by the env portion of the test and includes attributes of the current and previous blocks. A set of pre-existing accounts are detailed in the pre portion and form the world state prior to execution. Similarly, a set of accounts are detailed in the post portion to specify the end world state.

The gas remaining (gas), the log entries (logs) as well as any output returned from the code (out) is also detailed.

Test Implementation

It is generally expected that the test implementer will read env, exec and pre then check their results against gas, logs, out, post and callcreates. If an exception is expected, then latter sections are absent in the test. Since the reverting of the state is not part of the VM tests.

Because the data of the blockchain is not given, the opcode BLOCKHASH could not return the hashes of the corresponding blocks. Therefore we define the hash of block number n to be SHA3-256(“n”).

Since these tests are meant only as a basic test of VM operation, the CALL and CREATE instructions are not actually executed. To provide the possibility of testing to guarantee they were actually run at all, a separate portion callcreates details each CALL or CREATE operation in the order they would have been executed. Furthermore, gas required is simply that of the VM execution: the gas cost for transaction processing is excluded.

Test Structure

{
   "test name 1": {
       "_info" : { ... },
           "env": { ... },
           "pre": { ... },
           "exec": { ... },
           "gas": { ... },
           "logs": { ... },
           "out": { ... },
           "post": { ... },
           "callcreates": { ... }
   },
   "test name 2": {
       "_info" : { ... },
           "env": { ... },
           "pre": { ... },
           "exec": { ... },
           "gas": { ... },
           "logs": { ... },
           "out": { ... },
           "post": { ... },
           "callcreates": { ... }
   },
   ...
}

Info Section

"_info" : {
         "comment" : "A test for (add 1 1) opcode result",
         "filling-rpc-server" : "Geth-1.9.14-unstable-8cf83419-20200512",
         "filling-tool-version" : "retesteth-0.0.3+commit.672a84dd.Linux.g++",
         "lllcversion" : "Version: 0.5.14-develop.2019.11.27+commit.8f259595.Linux.g++",
         "source" : "src/GeneralStateTestsFiller/stExample/add11Filler.json",
         "sourceHash" : "e474fc13b1ea4c60efe2ba925dd48d6f9c1b12317dcd631f5eeeb3722a790a37"
 },

Info section is generated with the test and contains information about test and it’s generators.

Fields

comment comment from the test source. (can be edited at source)
filling-rpc-server tool that has been used to generate the test (version)
filling-tool-version the test generator (retesteth) version
lllcversion lllc version that was used to compile LLL code in test fillers
source path to the source filler in the test repository
sourceHash hash of the json of source file (used to track that tests are updated)

Env Section

"env" : {
    "currentCoinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
    "currentDifficulty" : "0x020000",
    "currentGasLimit" : "0xff112233445566",
    "currentNumber" : "0x01",
    "currentTimestamp" : "0x03e8",
    "previousHash" : "0x5e20a0453cecd065ea59c37ac63e079ee08998b6045136a8ce6635c7912ec0b6"
},

Env section describe information required to construct a genesis block, or VM env for transaction execution.

  • The fields are always 0x prefixed HEX.

Fields

currentCoinbase author/miner/coinbase address
currentDifficulty transaction executed in a block with this difficulty
currentGasLimit transaction executed in a block with this gas limit
currentNumber transaction executed in a block with this number
currentTimestamp transaction executed in a block with this timestamp
previousHash hash of the previous block (deprecated)

Pre/preState Section

"pre" : {
    "0x095e7baea6a6c7c4c2dfeb977efac326af552d87" : {
        "balance" : "0x0de0b6b3a7640000",
        "code" : "0x600160010160005500",
        "nonce" : "0x00",
        "storage" : {
            "0x00" : "0x01"
        }
    },
    "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" : {
        "balance" : "0x0de0b6b3a7640000",
        "code" : "0x",
        "nonce" : "0x00",
        "storage" : {
        }
    }
},

Pre section describes a full state of accounts used in test.

Its a map of <Account> => <AccountFields>

AccountFields are always complete (balance, code, nonce, storage must present) in this section and can not have a missing field.

  • All values are 0x prefixed hex.
  • Empty code defined as 0x.
  • Zero storage record defined as 0x00.

Fields

address hash HASH20 is 20 bytes ethereum address 0x prefixed
balance VALUE account balance in evm state
code BYTES account code in evm state
nonce VALUE account nonce in evm state
storage map map of storage records VALUE => VALUE
TYPE Empty Length Format description
VALUE 0x00 Any* 0x prefixed hex up to 32 bytes long with no leading zeros.
BYTES 0x Any* 0x prefixed bytes of any length
HASH8 0x00…00 Fixed 8 0x prefixed bytes of length 8
HASH20 0x00…00 Fixed 20 0x prefixed bytes of length 20
HASH32 0x00…00 Fixed 32 0x prefixed bytes of length 32
HASH256 0x00…00 Fixed 256 0x prefixed bytes of length 256
  • Size can be limited by the meaning of field in tests. (like gasLimit ceil, tx signature v - value)

Exec Section

{
    "exec" : {
        "address" : "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6",
        "caller" : "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b",
        "code" : "0x600260021660005500",
        "data" : "0x",
        "gas" : "0x0186a0",
        "gasPrice" : "0x0c",
        "origin" : "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b",
        "value" : "0x0b"
    },
}

Fields

address FH20 The address of the account under which the code is executing, to be returned by the ADDRESS instruction.
origin FH20 The address of the execution’s origin, to be returned by the ORIGIN instruction.
caller FH20 The address of the execution’s caller, to be returned by the CALLER instruction.
value VALUE The value of the call (or the endowment of the create), to be returned by the CALLVALUE instruction.
data BYTES The input data passed to the execution, as used by the CALLDATA… instructions. Given as an array of byte values. See $DATA_ARRAY.
code BYTES The actual code that should be executed on the VM (not the one stored in the state(address)) . See $DATA_ARRAY.
gasPrice VALUE The price of gas for the transaction, as used by the GASPRICE instruction.
gas VALUE The total amount of gas available for the execution, as would be returned by the GAS instruction were it be executed first.
TYPE Empty Length Format description
VALUE 0x00 Any* 0x prefixed hex up to 32 bytes long with no leading zeros.
BYTES 0x Any* 0x prefixed bytes of any length
HASH8 0x00…00 Fixed 8 0x prefixed bytes of length 8
HASH20 0x00…00 Fixed 20 0x prefixed bytes of length 20
HASH32 0x00…00 Fixed 32 0x prefixed bytes of length 32
HASH256 0x00…00 Fixed 256 0x prefixed bytes of length 256
  • Size can be limited by the meaning of field in tests. (like gasLimit ceil, tx signature v - value)

Post Section

Same as Pre/preState Section

Callcreates Section

The callcreates section details each CALL or CREATE instruction that has been executed. It is an array of maps with keys:

  • data: An array of bytes specifying the data with which the CALL or CREATE operation was made. In the case of CREATE, this would be the (initialisation) code. See $DATA_ARRAY.
  • destination: The receipt address to which the CALL was made, or the null address ("0000...") if the corresponding operation was CREATE.
  • gasLimit: The amount of gas with which the operation was made.
  • value: The value or endowment with which the operation was made.

Logs Section

The logs sections contains the hex encoded hash of the rlp encoded log entries, reducing the overall size of the test files while still verifying that all of the data is accurate (at the cost of being able to read what the data should be). Each logentry has the format:

keccak(rlp.encode(log_entries))

(see https://github.com/ethereum/py-evm/blob/7a96fa3a2b00af9bea189444d88a3cce6a6be05f/eth/tools/_utils/hashing.py#L8-L16)

The gas and output Keys

Finally, there are two simple keys, gas and out:

  • gas: The amount of gas remaining after execution.
  • out: The data, given as an array of bytes, returned from the execution (using the RETURN instruction). See $DATA_ARRAY.
$DATA_ARRAY - type that intended to contain raw byte data
and for convenient of the users is populated with three types of numbers, all of them should be converted and concatenated to a byte array for VM execution.
  • The types are: 1. number - (unsigned 64bit) 2. “longnumber” - (any long number) 3. “0xhex_num” - (hex format number)

    e.g: ````[1, 2, 10000, "0xabc345dFF", "199999999999999999999999999999999999999"]````