How Much "Stolen" CPU Time Is Too Much?


#1

From https://scoutapp.com/blog/understanding-cpu-steal-time-when-should-you-be-worried:

Steal time is the percentage of time a virtual CPU waits for a real CPU while the hypervisor is servicing another virtual processor.

It’s shown as st in top.

I have one VPS with dedicated CPU usage (BuyVM), and a few others with “fair share” CPU usage.

What I’m noticing is that the BuyVM and one other VPS with fair share CPU (BudgetNode storage KVM) have close to no “stolen” CPU time (less than 1%), whereas two of the others constantly have stolen CPU > 5% (one of them peaks around 20-30%)

My CPU usage is quite low on all of these, so this just seems like noisy neighbors on the same node. In fact the BuyVM sees the highest load from me, and yet also has the best performance out of all of them in terms of both CPU and disk IO.

Understandably, stolen CPU time is expected with a VPS without dedicated CPU time (“fair share”), but how much stolen CPU time is expected? Is 5% normal? What about 20%? The only time I’ve ever submitted a support ticket about this was on BuyVM, and that’s because the CPU is supposed to be dedicated there (so stolen CPU should be ~0%). Turns out that was an issue on the node, and Francisco fixed it :slight_smile:


#2

Well, according to tests, more then 10% does affect your applications running on the system.


#3

Highly depends on what price you paid :stuck_out_tongue:

I’d expect a ton of steal time on LES NAT VMs for example, but not a $20/mo 2GB KVM instance.


#4

Bumping up the granularity on the graph shows definite spikes on one particular VPS:

Right, that’s what I was thinking… I’m thinking of trying a few other providers around the same price range to see if they have the same amount of stolen CPU time.

Unfortunately OpenVZ doesn’t seem to report steal time (I think the kernel only supports it for KVM and LXC), so it just appears as 0.0 on my NAT VPS. :stuck_out_tongue:


#5

Tbh if you’re seeing any there’s a good chance the provider needs to kick out someone who thinks CPU mining is a good idea. The amount of CPU you have to oversell to see legitimate usage using 50% of it on any regular basis is insane.


#6

General, if you see Steal above 1-2% I would contact the Provider and ask to me moved onto a different node. Mostly it wont happen but if it does, the Provider oversold that not to much.


#7

Like other already said if it’s more than few % ask provider to move node.

You don’t see CPU steal issue that often, I’ve only had the issue once with a VM sold with dedicated core technical it was thread which had steal as high as 40-50% at times.


#8

Should ticket and we can look.

We don’t hard cap by default, but if a node gets busy (some fair share CPU plans not being fair) then we have no problem mass capping.

Francisco


#9

Sorry if I wasn’t clear - My VPS with BuyVM is actually one of the ones where I don’t have this issue. I had ticketed in the past, but haven’t seen the issue come back since then!

Based on comments in this thread, it sounds like I should submit tickets with the other providers


#10

Wouldn’t happen to be a provider I’m intimately familiar with would it? We had some complaints of it when miners went crazy, would like to know if any linger.


#11

It’s not DO :slight_smile:


#12

Not proud to say @Daniel is talking about one of my nodes. Had a few growing pains, they’re getting resolved but unfortunately a couple of customers have felt it. I’m doubling the node capacity in May and implementing automated CPU throttling for people who use more than their fair share.

I’m doing the best I can to rectify @Daniel’s issue, still waiting on him to confirm but hopefully it’s closed out.


#13

Thanks @Brendan! I was being careful to not name any providers negatively, but I guess you found my post :slight_smile: I’m keeping an eye on it and collecting more data (both for CPU and for disk IO), and will report back tomorrow.

I guess it’s worth mentioning that the other VPS with high stolen CPU time (albeit without the large peaks) is with VirMach. However, it’s one of their ridiculously cheap Black Friday sale VPSes (~$39 per year for 8 GB RAM, 60 GB SSD, 10 Gb/s connection), which at their price points I expect to have some amount of oversold CPU.


#14

I got moved to another node yesterday. The stolen CPU time is noticeably lower on the new node. There’s still spikes, but they’re lower than before (although still over 10% at times). Will monitor it for a week to see how often it spikes.


#15


#16

lol, if you think that’s amazing, VirMach had an “8” deal on either Black Friday or Cyber Monday: 8 CPU cores, 8 GB RAM, 80 GB SSD, 8 IPs for $8 per year, with only 8 in stock. I think it was the “@FAT32 special”. I tried to get it but their billing system collapsed and I failed. :stuck_out_tongue:


#17


#18

Let me guess: Overcommit of 1:8 on everything as well


#19

Love thing of you Brendan!

:wink:


#20

good luck to you. I would never use virmach for business because of their high usage policy and fee schedule.

Personally I wouldn’t worry about cpu steal because their high usage policy will suspend your vps way before it becomes an issue.