Yeah, they replied to me with the attack was to tiny, but it was still more than 1gb/s sadly it’s not a webserver in this case but a game server and it was a flood attack so simply filling up my bandwidth. I wonder what i could do to prevent it i know they have a little firewall in their robot interface but it’s limited to 10 records …
Defending games servers can be difficult. Do you know what kind of traffic was used to flood your server?
1 Like
I think it was some CLDAP or some kind of Mixed AMP, so all in Layer 4. The thing is that at the moment i got attacked i couldn’t do anythng… not even connect to my machine.
I think it was an UDP flood with some amplification i think so.
I got one of my NVME disks that has like 5.35 PB of read… and its smart status looks bad the other one also had 4 PB read but the tests seem okay.
smartctl --all /dev/nvme0
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.0-112-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZVLB1T0HALR-00000
Serial Number: S3W6NA0M607975
Firmware Version: EXA7301Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 1,024,209,543,168 [1.02 TB]
Unallocated NVM Capacity: 0
Controller ID: 4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 1,024,209,543,168 [1.02 TB]
Namespace 1 Utilization: 705,093,128,192 [705 GB]
Namespace 1 Formatted LBA Size: 512
Local Time is: Tue Aug 4 09:00:09 2020 CEST
Firmware Updates (0x16): 3 Slots, no Reset required
Optional Admin Commands (0x0017): Security Format Frmw_DL *Other*
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 81 Celsius
Critical Comp. Temp. Threshold: 82 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 7.02W - - 0 0 0 0 0 0
1 + 6.30W - - 1 1 1 1 0 0
2 + 3.50W - - 2 2 2 2 0 0
3 - 0.0760W - - 3 3 3 3 210 1200
4 - 0.0050W - - 4 4 4 4 2000 8000
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
- NVM subsystem reliability has been degraded
SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning: 0x04
Temperature: 53 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 64%
Data Units Read: 10,456,269,776 [5.35 PB]
Data Units Written: 1,116,509,064 [571 TB]
Host Read Commands: 106,660,265,864
Host Write Commands: 19,199,610,651
Controller Busy Time: 1,299,516,066
Power Cycles: 10
Power On Hours: 6,769
Unsafe Shutdowns: 0
Media and Data Integrity Errors: 11
Error Information Log Entries: 11
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 53 Celsius
Temperature Sensor 2: 74 Celsius
Error Information (NVMe Log 0x01, max 64 entries)
Num ErrCount SQId CmdId Status PELoc LBA NSID VS
0 11 2 0x0367 0x4502 0x000 923146742 1 -
What should i do?
Have you tried contacting the support? I think they change drives in such cases for free IIRC.
1 Like
Yeah just got them they said that this error indicates that the connection between the drive and the mainboard is not good.
And it was only used at 64% so they won’t change them but at my eyes 5 PB is a lot for a ssd isn’t it ?
Can’t find any expected life. One site did say max 600TB but seems a bit low imo. But yea 5 PB Is a bit high but not critical.
EDIT: Nope 600 TB seems like most reliable 1 tb ssd can write. (or 1.5m power on hours) So not read. Read issn’t much of a problem for most ssd’s
read doesn’t matter. only writes wear out the cells. each ssd/nvme has it’s own specification on TBW aka “terabytes written” … for the PM981 I did not find any reliable information though.
somewhere it said to be around 1200TB for the 1TB model, so there is quite some room still and even then it does not mean that drive is going to die anywhere soon.
seems like there hasn’t any spare cells been used yet, so no problem as well.
for the errors I think something like that can happen during initial installation, I don’t think there is much to worry about that drive right now. if you are unsure see if you can run an extended offline test via smartctl on it.
2 Likes
It all depends on the amount of DRAM also.
FYI, box has just arrived a couple hours ago. Yipieee.
In case anyone is intrested, 1) disks are brand new, only been online ~3 hours.
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.4.47] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: KXD51RUE1T92 TOSHIBA
Serial Number: ****
Firmware Version: 1CEE6111
PCI Vendor/Subsystem ID: 0x1179
IEEE OUI Identifier: 0x8ce38e
Total NVM Capacity: 1,920,383,410,176 [1.92 TB]
Unallocated NVM Capacity: 0
Controller ID: 0
Number of Namespaces: 1
Namespace 1 Size/Capacity: 1,920,383,410,176 [1.92 TB]
Namespace 1 Utilization: 4,305,784,832 [4.30 GB]
Namespace 1 Formatted LBA Size: 512
Namespace 1 IEEE EUI-64: 8ce38e 02001afc5d
Local Time is: Tue Aug 4 17:02:15 2020 CEST
Firmware Updates (0x14): 2 Slots, no Reset required
Optional Admin Commands (0x001e): Format Frmw_DL NS_Mngmt Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 68 Celsius
Critical Comp. Temp. Threshold: 70 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 7.00W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 - 4096 0 1
1 + 512 0 2
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 31 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 189,415 [96.9 GB]
Data Units Written: 8,415 [4.30 GB]
Host Read Commands: 189,146,503
Host Write Commands: 3,393
Controller Busy Time: 44
Power Cycles: 5
Power On Hours: 3
Unsafe Shutdowns: 0
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 31 Celsius
Error Information (NVMe Log 0x01, max 128 entries)
No Errors Logged
All are the same (brand new, same info).
… 2) Disks tracked down to be using these…
Quite respectiable 
.
Thanks again Hetzner 
now big task to move lot of stuff over, yawns.
2 Likes
Just run
‘’’
terraform init && terraform apply
‘’’
1 Like
End goal is this but not all of existing stuff uses same hypervisor and either manually done or partly in ansible. In the middle of refactoring and sticking everything in code.
1 Like
Just saw a tiny video from hetzner done in 2018 it states that their is 229K servers XD… Hetzner - our way to Finland - YouTube
3 Likes
It would be interesting to see a graph of the development of those numbers, with the cloud and all. I think Hetzner grew a lot since then.
4 Likes
+1
@Hetzner_OL your chance to impress us with some quality marketing figures 
5 Likes
Hahaha normally they don’t give numbers around that’s also why i sended it. It’s pretty rare to hear the numbers for hetzner since logically there is so much … 
3 Likes
Do you know what they show in this post? https://bibliogram.snopyta.org/p/CDeEGAiC4bz
2 Likes
I’m sorry, but we don’t give out concrete numbers and facts.
But I can give you two numbers, in Helsinki we currently have 150,000 square meters of space to grow and the data center park in Falkenstein offers 100,000 square meters of space on which to build. So there is still plenty of room for us to develop.
1 Like
With the right perspective, the inside of a data center is a visual feast. Illustrating the “Four Cs” - Corridors, Cabling, Cooling and Containment.
Well, who knows what’s in this picture?
Well, who knows what’s in this picture?
5 Likes
Do you mean the wagon in the front, or the background? Because the name of the picture can be seen when hovering over it
2 Likes