Aptos Light Client

End-to-end benchmarks

The end-to-end benchmark is meant to benchmark the time taken to send both of the proofs generation request to the Proof Server, have a parallel computation happen and receive the two proofs back. This benchmark is meant to simulate the worst case scenario where the client has to generate two proofs in parallel. It can run the proofs in sequence if the benchmark is running in a single machine, to prevent resource exhaustion.

The benchmark can be found in the proof-server crate. It can be run with the following command:

RUN_PARALLEL="0" RECONSTRUCT_COMMITMENTS=false SHARD_CHUNKING_MULTIPLIER=1 SHARD_SIZE=1048576 SHARD_BATCH_SIZE=0 RUSTFLAGS="-C target-cpu=native --cfg tokio_unstable -C opt-level=3" PRIMARY_ADDR="127.0.0.1:8080" SECONDARY_ADDR="127.0.0.1:8081" cargo bench --bench proof_server

This benchmark will spawn the two servers locally and make two requests in sequence to them. This generates both proofs in the same machine, one after the other. In a production setting, the two prover servers will be in different machines, and the two proofs would be generated in parallel. This returns times that are closer to a production setting, without any resource exhaustion when generating the proofs.

To run the proofs in parallel even though it's running in one single machine, pass the RUN_PARALLEL=1 environment variable when running the benchmark. This is not recommended as the total reported time for generating both proofs at the same time in the same machine will be longer than a properly configured production setting where the proofs are generated by different machines.

The benchmark returns two main metrics for each proof:

• e2e_proving_time: Time in milliseconds taken to send both request to the Proof Server and generate both proofs.
• inclusion_proof:
  • proving_time: Time in milliseconds taken to generate the inclusion proof.
  • request_response_proof_size: Size of the proof in bytes returned by the server.
• epoch_change_proof:
  • proving_time: Time in milliseconds taken to generate the epoch change proof.
  • request_response_proof_size: Size of the proof in bytes returned by the server.

For our production configuration, we currently get the following results:

{
  "e2e_proving_time": 65543,
  "inclusion_proof": {
    "proving_time": 54239,
    "request_response_proof_size": 17520651
  },
  "epoch_change_proof": {
    "proving_time": 65543,
    "request_response_proof_size": 20154137
  }
} 

on r7iz.metal-32xl AWS EC2 instance. The sphinx's commit used is: d392acc

SNARK proofs

To enable SNARK proving, just pass the environment variable SNARK=1 when running:

SNARK="1" RECONSTRUCT_COMMITMENTS=false SHARD_CHUNKING_MULTIPLIER=64 SHARD_SIZE=4194304 SHARD_BATCH_SIZE=0 RUSTFLAGS="-C target-cpu=native --cfg tokio_unstable -C opt-level=3" PRIMARY_ADDR="127.0.0.1:8080" SECONDARY_ADDR="127.0.0.1:8081" cargo bench --bench proof_server

For our production configuration, we currently get the following results:

{
  "e2e_proving_time": 519213,
  "inclusion_proof": {
    "proving_time": 519213,
    "request_response_proof_size": 18600
  },
  "epoch_change_proof": {
    "proving_time": 519162,
    "request_response_proof_size": 28709
  }
} 

on r7iz.metal-16xl AWS EC2 instance. The sphinx's commit used is: d392acc

Looking for optimal Sphinx parametrs

Sometimes it is possible to get some boost in performance on a specific benchmarking machine tuning SHARD_CHUNKING_MULTIPLIER variable. It's optimal value depends on the number of CPU cores and hence must be obtained empirically (e.g. by running e2e bench multiple times varying value of SHARD_CHUNKING_MULTIPLIER). The e2e_bench.sh script is provided in the root of the workspace, specifically for this case. For r7iz.metal-16xl AWS EC2 instance, its optimal value is 32 (for SNARK).