This topic is just for me to get a little smarter on what would be considered “safe” overselling/overcommitting of resources on a Proxmox host. Not asking for you guys to divulge your industry secrets or anything like that – just trying to get a general idea.
Some context - I currently run a small free VPS host (FreeMach). With my first box, I commit memory ~1:1, meaning each guest’s allocation is dedicated (or near-dedicated). But after running this node for over a year, I’ve noticed that RAM usage really hasn’t ever exceeded ~60%, so I’m thinking perhaps there is room to overcommit.
The second node for this service is going live shortly and I wanted to get some opinions on whether overcommit memory would be safe. The host has 24GB of RAM and I’m intending to allocate 16 2GB KVM guests on the box. So this would result in 1.33:1 of allocated memory to physical memory. Ignoring other factors, such as network and cpu (which I think will be fine under my proposed config), would I be safe in doing this?
I know it’s easier to oversell with containers, but am unfamiliar with KVM guest’s resource usage. Any insight provided would be helpful! I use ZFS on my Proxmox hosts, if that changes the equation at all.
I never overcommit memory on KVM, but I’ll drive containers into the damn swap. I don’t know if proxmox just has a really well optimized stack but my efforts with KSM have proven to be ineffective and at too high of a cost.
Hmm, okay. Maybe I’ll reconsider my approach and cut down the number of VMs then to be closer to 1:1.
Edit: Also might be a smarter move to not oversell since I’m finding users who will actually be utilizing their VMs as opposed to just letting them idle.
This is my current node’s usage with 10x 3GB KVM guests running. ZFS, KSM, and ballooning all enabled.
The ~60% memory usage is what makes me feel like I could possibly stuff more VMs on and still be safe. But maybe there’s something I’m not seeing that would cause issues if I did that.
+5 Machines, I would not expect that someone uses all the 3GB.
Overselling is always risky, LET I would expect that 50% of the machines are at idle or more.
Ask them, tell them, big big interest, we need more memory.
“Can you steal a 32gig stick and put it into my server?”
“Yes sir, this product that is in direct competition with your own services and that I give away for free to your potential clients which won’t buy from you when they get their service from me needs more stuff. Give me more stuff so I can steal more clients away from you kthxbai”
seems fine to me. of course it might get harder the smaller the guests are, because you’ll always need a static minimum and can only balloon into buffers/cache. though setting memory to dynamic 1.5/2GB should have you covered, KSM will take care of the rest.
I don’t overcommit memory on KVM, KSM is always tricky - but if you have Proxmox taking care of it and it works, why not. Ensure you ALWAYS have enough free swap space.
Also KSM can result in a slightly lower overall performance, so if you are CPU bound, it may bring more problems than it solves.
OVZ is a totally different beast than KVM and memory overcommit works differently.
What is possible with containerization isn’t possible with KVM.
ZFS complicates things slightly.
proxmox ZoL allocates 50% of host memory to ARC.
I reduce arc-max to 8GB on my 32GB RAM kvm boxen.
So your current numbers look good by that metric I guess.
What you can oversell slightly on zfs is ZVOL disk sizes. (zfs lz4 is amazing)
IIRC You want to target a MAX of 85% zfs pool true space utilization to prevent everything from crumbling.
Maybe restrict users to template images instead of ISO installs to keep utilization predictable.
Proxmox / KSM default is to enable itself @ 80% (this can be changed easily in a config), if your RAM use is sitting under 80% and KSM reports 0 this means all RAM use is non compressed / dedupe.
If your running alot of Linux VM’s KSM can have a big impact on lowering RAM due to alot of the RAM use being simular enabling the dudupe function of KSM to have a big effect.
If your just using standard mdadm raid then your memory usage shouldn’t have a big impact during rebuild’s e.t.c however if your using ZFS or something else you will need to account for extra RAM there.
As noted by others KSM does have a CPU overhead, so if your node’s are already heavy CPU wise then may not be a good idea. But if load is always light you’d hardly notice the couple of extra CPU % used.
I recently started doing this using solusvm, thanks everyone for the insights for now I’m keeping it 1:1 too but we’ll see after few months if there’s room to expand