Google cloud EXTREME PERSISTENT DISK (single)/120000 IOPS configuration benchmark/review

  • Run date – March 17, 2023
  • We used the configuration below from Google cloud for this benchmarking in zone us-west4-a
    • One n2-highcpu-64 with Debian 11
    • One 1000GB extreme persistent disk with 120000 IOPS configuration, no snapshot
    • No optimization at OS level
  • Treat this as our review of single extreme persistent disk performance with the configuration above. Note that there is a throughput limit of 2200 MB/s on a single extreme PD; we could not push more than this.
  • Used direct IO and/or sync to avoid cache.
  • Picked the best possible peak throughput numbers and corresponding 99.9 percentile latencies to achieve this bandwidth.
  • We used the same workload (jobs/threads) that we picked for the best peak numbers for 1m block size for sustained run.
  • Whether an application can fully utilize the bandwidth (The advertised “up to” number) really depends on which block size, read/write ratio, sequential or random access workload, that application is using.
    • We can help identify which storage system is optimal for your application in terms of performance.
  • Few observations:
    • Overall latencies are higher than we expected to achieve peak throughput considering the high cost of this performance storage.
    • For some reason 8K sequential read peak throughput and latency is much lower than random 8k read. We double checked this behavior.
    • Latencies of 4k/8k for mixed work load (r/w) are really high.
    • We did rewrites up to 1000GB of data to benchmark sustained writes. Throughput fluctuation between 2200Mbps and around 1500Mbps during sustained runs. Results might change when there is much more writing.
    • Getting the right instance and this extreme PD combination was a hassle. Had to try different regions to get this resource combination.
  • If you are interested, use with caution as ballpark performance numbers could vary depending on multiple factors (current cloud usage in that zone, backing store usage, etc). These are the numbers we got in our run. We are not liable for any loss or damages caused by using this data.