Vultr maybe? 1TB is included for 5 bucks and overages are only $10/TB (a cent per GB used over).
You could even in theory “load balance” instances to bring the BW costs down to $5/TB by rotating instances when they hit 1TB of usage each. Then rinse and repeat every month.
Possibly. I checked out most of the big dogs that have infrastructure in the States and unsurprisingly their bandwidth allocations are pretty low unless you pay the big bucks. @Ryan’s Catalyst Host might be a good location since his packages include 10TB allowance and are in Dallas (currently not covered). VMHaus also has a pretty high b/w allowance with their London offerings.
I doubt I’ll be able to find anything in AUS or some other APAC location (hong kong, south korea, etc.) that will have sufficient bandwidth since it’s quite expensive over there. @Brendan do you have any recommendations for something in your neck of the woods? I really don’t know how much bandwidth something like this would eat up, so I’m wary to give any estimates.
South America would be pretty cool as well, if there’s something down there that’s a reasonable price and also has high bandwidth + link speed.
To stop derailing this thread, I’m saying they’re telling you just what you want to hear, and what they want you to accept.
Hardware RAID will almost always be better for you than Software, except from an administration standpoint.
OpenBSD is probably a bit more secure than your general Linux installation, but guess what? People love their CentOS. It doesn’t matter what someone wants you to believe, you’re going to generally go with your preconceived notions.
You don’t have to agree with me; you don’t have to host with me. You’re welcome to your md raid on ext3 over ceph if you so choose. Just don’t try to be pious about being both cheap, and misinformed.
FWIW, @Friendly and I took this to PM.
I’ll post photos. <3
Thanks @Mason for your work
It has some awesome ideas that I use in mine shell script
btw, what exactly ioping is doing?
Just pushed an update to the repo - GitHub - masonr/yet-another-bench-script: YABS - a simple bash script to estimate Linux server performance using fio, iperf3, & Geekbench.
Notable changes:
Here’s the updated sample output:
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-01-08 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Wed 08 Jan 2020 07:33:21 PM UTC
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
CPU cores : 8 @ 4098.759 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 31Gi
Swap : 0B
Disk : 221G
dd Sequential Disk Speed Tests:
---------------------------------
| Test 1 | Test 2 | Test 3 | Avg
| | | |
Write | 291 MB/s | 286 MB/s | 281 MB/s | 286.00 MB/s
Read | 179 MB/s | 188 MB/s | 179 MB/s | 182.00 MB/s
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 2.93 Gbits/sec | 7.80 Gbits/sec
Online.net | Paris, FR (10G) | 7.79 Gbits/sec | 5.20 Gbits/sec
Severius | The Netherlands (10G) | 8.98 Gbits/sec | 2.53 Gbits/sec
Worldstream | The Netherlands (10G) | 8.65 Gbits/sec | 8.57 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 7.80 Gbits/sec | 9.03 Gbits/sec
Biznet | Bogor, Indonesia (1G) | 752 Mbits/sec | busy
Hostkey | Moscow, RU (1G) | 905 Mbits/sec | 449 Mbits/sec
Vultr | Piscataway, NJ, US (1G) | 448 Mbits/sec | 51.6 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 1.74 Gbits/sec | 1.61 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 1.61 Gbits/sec | 106 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 28.2 Mbits/sec | 476 Mbits/sec
iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 7.78 Gbits/sec | 6.00 Gbits/sec
Online.net | Paris, FR (10G) | 2.86 Gbits/sec | 5.74 Gbits/sec
Severius | The Netherlands (10G) | 6.96 Gbits/sec | 2.38 Gbits/sec
Worldstream | The Netherlands (10G) | 7.29 Gbits/sec | 6.02 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 4.64 Gbits/sec | 8.93 Gbits/sec
Vultr | Piscataway, NJ, US (1G) | 97.5 Mbit/sec | 37.3 Mbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | busy | busy
Hurricane Electric | Fremont, CA, US (10G) | 348 Mbits/sec | 505 Mbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 5714
Multi Core | 19758
Full Test | https://browser.geekbench.com/v4/cpu/15115430
If you have any questions, concerns, comments, or suggestions regarding YABS, please let me know! 
Where’s the butts i/o quotient test?
It’s top priority to add that in. Right after the fsck serial i/o test.
Maybe maxi(host) or g core (labs).
Not many good options or some pschyz reseller.
By reading cf blog posts and by seing cdn providers ( like bunny cdn) there is not much in Latin America that can be had for cheap.
Quite a few new changes have been pushed to the project. The biggest changes were replacing dd sequential disk speed tests with fio random disk speed tests for more real-world results and adding in both IPv6-only and 32-bit support. Give me a shout if anyone has any errors to report and I’ll get those patched up!
Here’s the current output of the script:
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Tue Feb 4 19:04:24 UTC 2020
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
CPU cores : 8 @ 800.098 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 31G
Swap : 0B
Disk : 221G
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4kb (IOPS) | 64kb (IOPS)
------ | --- ---- | ---- ----
Read | 69.37 MB/s (17.3k) | 106.51 MB/s (1.6k)
Write | 69.57 MB/s (17.3k) | 107.07 MB/s (1.6k)
Total | 138.94 MB/s (34.7k) | 213.59 MB/s (3.3k)
| |
Block Size | 512kb (IOPS) | 1mb (IOPS)
------ | ----- ---- | --- ----
Read | 133.52 MB/s (260) | 141.90 MB/s (138)
Write | 140.61 MB/s (274) | 151.35 MB/s (147)
Total | 274.13 MB/s (534) | 293.26 MB/s (285)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 1.41 Gbits/sec | 1.13 Gbits/sec
Online.net | Paris, FR (10G) | 1.44 Gbits/sec | 1.29 Gbits/sec
Worldstream | The Netherlands (10G) | 1.18 Gbits/sec | 1.22 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 805 Mbits/sec | 1.13 Gbits/sec
Biznet | Bogor, Indonesia (1G) | 768 Mbits/sec | 38.5 Mbits/sec
Hostkey | Moscow, RU (1G) | 503 Mbits/sec | 686 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 2.74 Gbits/sec | 2.67 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 3.36 Gbits/sec | 963 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 6.34 Gbits/sec | 3.76 Gbits/sec
iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 1.44 Gbits/sec | 1.25 Gbits/sec
Online.net | Paris, FR (10G) | 1.36 Gbits/sec | 972 Mbits/sec
Worldstream | The Netherlands (10G) | 1.19 Gbits/sec | 1.20 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 826 Mbits/sec | 1.14 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | busy | busy
Hurricane Electric | Fremont, CA, US (10G) | 6.36 Gbits/sec | 2.95 Gbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 5587
Multi Core | 19093
Full Test | https://browser.geekbench.com/v4/cpu/15200550
Thumbs up ! 
Wow <3 will post a benchmark here later of my new sexy vm with @MikeA
Sorry I’m late, here is the bench
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu Feb 6 04:24:14 CST 2020
Basic System Information:
---------------------------------
Processor : AMD EPYC Processor (with IBPB)
CPU cores : 2 @ 3892.686 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ❌ Disabled
RAM : 483Mi
Swap : 255Mi
Disk : 5.6G
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4kb (IOPS) | 64kb (IOPS)
------ | --- ---- | ---- ----
Read | 338.35 MB/s (84.5k) | 833.99 MB/s (13.0k)
Write | 339.24 MB/s (84.8k) | 838.37 MB/s (13.0k)
Total | 677.59 MB/s (169.3k) | 1.67 GB/s (26.1k)
| |
Block Size | 512kb (IOPS) | 1mb (IOPS)
------ | ----- ---- | --- ----
Read | 923.12 MB/s (1.8k) | 930.99 MB/s (909)
Write | 972.17 MB/s (1.8k) | 992.99 MB/s (969)
Total | 1.89 GB/s (3.7k) | 1.92 GB/s (1.8k)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 177 Mbits/sec | 168 Mbits/sec
Online.net | Paris, FR (10G) | busy | 132 Mbits/sec
Worldstream | The Netherlands (10G) | 142 Mbits/sec | 93.2 Mbits/sec
wilhelm.tel | Hamburg, DE (10G) | 128 Mbits/sec | 119 Mbits/sec
Biznet | Bogor, Indonesia (1G) | 148 Mbits/sec | 41.1 Mbits/sec
Hostkey | Moscow, RU (1G) | 125 Mbits/sec | 164 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | busy | 193 Mbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | busy | 189 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | busy | 179 Mbits/sec
Geekbench 4 test failed. Run manually to determine cause.
Thanks for sharing the results! I’m curious why the Geekbench test failed. Maybe it was killed halfway through or something.
Yesterday, I purchased yabs.sh and set up redirects to the script/project page on github.
You can now run the YABS test via:
curl -sL yabs.sh | bash
Curl/wget requests get redirected to the raw script content on github, while other requests (i.e. from a web browser) get redirected to the main github project page.
fio is failing for me. Fresh Debian 10 install:
Generating fio test file...main: line 170: 922 Illegal instruction $DISK_PATH/fio --name=setup --ioengine=libaio --rw=read --bs=4k --iodepth=64 --numjobs=2 --size=2G --runtime=1 --gtod_reduce=1 --filename=$DISK_PATH/test.fio --direct=1 --minimal &> /dev/null
fio disk speed tests failed. Run manually to determine cause.
Here is my output from BuyVM: (Ubuntu 18.04) - No Errors
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu Feb 6 15:36:15 EST 2020
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) CPU E3-1270 v3 @ 3.50GHz
CPU cores : 1 @ 3499.992 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ❌ Disabled
RAM : 481M
Swap : 511M
Disk : 261G
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4kb (IOPS) | 64kb (IOPS)
------ | --- ---- | ---- ----
Read | 13.93 MB/s (3.4k) | 72.47 MB/s (1.1k)
Write | 13.94 MB/s (3.4k) | 72.85 MB/s (1.1k)
Total | 27.88 MB/s (6.9k) | 145.32 MB/s (2.2k)
| |
Block Size | 512kb (IOPS) | 1mb (IOPS)
------ | ----- ---- | --- ----
Read | 157.18 MB/s (306) | 175.35 MB/s (171)
Write | 165.53 MB/s (323) | 187.03 MB/s (182)
Total | 322.71 MB/s (629) | 362.38 MB/s (353)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 686 Mbits/sec | 512 Mbits/sec
Online.net | Paris, FR (10G) | 714 Mbits/sec | 533 Mbits/sec
Worldstream | The Netherlands (10G) | 692 Mbits/sec | 598 Mbits/sec
wilhelm.tel | Hamburg, DE (10G) | 621 Mbits/sec | 442 Mbits/sec
Biznet | Bogor, Indonesia (1G) | 513 Mbits/sec | 83.8 Mbits/sec
Hostkey | Moscow, RU (1G) | 647 Mbits/sec | 531 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 762 Mbits/sec | 739 Mbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 803 Mbits/sec | 629 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 182 Mbits/sec | 122 Mbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 3436
Multi Core | 3308
Full Test | https://browser.geekbench.com/v4/cpu/15206159
Here is my output from AlphaVPS (Proxmox) - ERROR
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu 06 Feb 2020 03:43:40 PM -05
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) CPU L5630 @ 2.13GHz
CPU cores : 16 @ 1658.981 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 15Gi
Swap : 15Gi
Disk : 901G
Generating fio test file...main: line 170: 24676 Illegal instruction $DISK_PATH/fio --name=setup --ioengine=libaio --rw=read --bs=4k --iodepth=64 --numjobs=2 --size=2G --runtime=1 --gtod_reduce=1 --filename=$DISK_PATH/test.fio --direct=1 --minimal &> /dev/null
fio disk speed tests failed. Run manually to determine cause.
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 47.8 Mbits/sec | 874 Mbits/sec
Online.net | Paris, FR (10G) | 908 Mbits/sec | 910 Mbits/sec
Worldstream | The Netherlands (10G) | 916 Mbits/sec | 911 Mbits/sec
wilhelm.tel | Hamburg, DE (10G) | 917 Mbits/sec | 788 Mbits/sec
Biznet | Bogor, Indonesia (1G) | 509 Mbits/sec | 92.7 Mbits/sec
Hostkey | Moscow, RU (1G) | 891 Mbits/sec | 854 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 810 Mbits/sec | 550 Mbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 773 Mbits/sec | 507 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 786 Mbits/sec | 390 Mbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1881
Multi Core | 12372
Full Test | https://browser.geekbench.com/v4/cpu/15206176
I installed fio via apt and modified the script to use this version:
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu 6 Feb 20:51:42 GMT 2020
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) E-2276G CPU @ 3.80GHz
CPU cores : 4 @ 3791.996 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 3.9Gi
Swap : 4.0Gi
Disk : 46G
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4kb (IOPS) | 64kb (IOPS)
------ | --- ---- | ---- ----
Read | 445.04 MB/s (111.2k) | 1.39 GB/s (21.7k)
Write | 446.22 MB/s (111.5k) | 1.40 GB/s (21.8k)
Total | 891.26 MB/s (222.8k) | 2.79 GB/s (43.6k)
| |
Block Size | 512kb (IOPS) | 1mb (IOPS)
------ | ----- ---- | --- ----
Read | 916.91 MB/s (1.7k) | 931.84 MB/s (910)
Write | 965.63 MB/s (1.8k) | 993.91 MB/s (970)
Total | 1.88 GB/s (3.6k) | 1.92 GB/s (1.8k)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 8.70 Gbits/sec | 7.63 Gbits/sec
Online.net | Paris, FR (10G) | 8.95 Gbits/sec | 8.77 Gbits/sec
Worldstream | The Netherlands (10G) | 6.70 Gbits/sec | 7.77 Gbits/sec
wilhelm.tel | Hamburg, DE (10G) | 6.58 Gbits/sec | 7.81 Gbits/sec
Biznet | Bogor, Indonesia (1G) | 829 Mbits/sec | 37.0 Mbits/sec
Hostkey | Moscow, RU (1G) | 923 Mbits/sec | 923 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 1.45 Gbits/sec | 1.58 Gbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 1.29 Gbits/sec | 958 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 10.1 Mbits/sec | 37.1 Mbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 6434
Multi Core | 20747
Full Test | https://browser.geekbench.com/v4/cpu/15206187
Ubuntu 19.10 - ERROR
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-04 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu 06 Feb 2020 12:52:26 PM PST
Basic System Information:
---------------------------------
Processor : Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
CPU cores : 1 @ 2666.748 MHz
AES-NI : ❌ Disabled
VM-x/AMD-V : ❌ Disabled
RAM : 3.8Gi
Swap : 2.0Gi
Disk : 98G
Generating fio test file...main: line 170: 20815 Illegal instruction $DISK_PATH/fio --name=setup --ioengine=libaio --rw=read --bs=4k --iodepth=64 --numjobs=2 --size=2G --runtime=1 --gtod_reduce=1 --filename=$DISK_PATH/test.fio --direct=1 --minimal &> /dev/null
fio disk speed tests failed. Run manually to determine cause.
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed
| | |
Bouygues Telecom | Paris, FR (10G) | 826 Mbits/sec | 222 Mbits/sec
Online.net | Paris, FR (10G) | 851 Mbits/sec | 302 Mbits/sec
Worldstream | The Netherlands (10G) | 825 Mbits/sec | 292 Mbits/sec
wilhelm.tel | Hamburg, DE (10G) | 818 Mbits/sec | 237 Mbits/sec
Biznet | Bogor, Indonesia (1G) | 536 Mbits/sec | 37.1 Mbits/sec
Hostkey | Moscow, RU (1G) | 814 Mbits/sec | 701 Mbits/sec
Velocity Online | Tallahassee, FL, US (10G) | 894 Mbits/sec | 468 Mbits/sec
Airstream Communications | Eau Claire, WI, US (10G) | 905 Mbits/sec | 632 Mbits/sec
Hurricane Electric | Fremont, CA, US (10G) | 125 Mbits/sec | 221 Mbits/sec
Geekbench 4 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1701
Multi Core | 1663
Full Test | https://browser.geekbench.com/v4/cpu/15206199
Debian 10 is the same. apt-get install fio and then modifying the script slightly worked for me.