Configure Multiple External IPs Within Proxmox

Ok I think I did something wrong.

If I want 2 VMs inside a VPS, I need 4 ips in total???

1st for host
2nd as gateway
3rd and 4th for guest

?

I think I’ve installed Proxmox 3 or 4 times. And I forget the procedure every single time.

1 Like

Three should be enough. The one you give your vps as host is also the gateway for the guests or even the vpss gateway might work.

2 Likes

2004? 2003?

EDIT: I see now. It’s the version number, not a date :sweat_smile:

1 Like

I believe it would work with 2 additional IPs (instead of a small subnet) and then setup the guests to use pointopoint network to the main host IP.

It’s probably not the standard procedure as a /29 might be easier, but if the guest VMs are your own it might be a cheaper option.

2 Likes

Thanks @Falzo @Mr_Tom

Tried:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static

auto vmbr0
iface vmbr0 inet manual
	address x.x.x.125/24
	gateway x.x.x.1
	bridge-ports eth0
	bridge-stp off
	bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
	address 10.0.0.1/24
	bridge-ports none
	bridge-stp off
	bridge-fd 0

And setting x.x.x.126/24 in the guest for vmbr0.

Didn’t work for the guest (using CentOS 7).

that’s not necassirily wrong, but depends on how your provider assigns these IPs. you are probably running into MAC issues here. with the setup you described, the MAC addresses of your guest need to be allowed on the hosts router. usually not working, because filtered as measure against IP spoofing.
that’s why you get virtual macs from hetzner or ovh f.i. for additional IPs.

you could try to use /32 as subnet masks and add pointopoint to the config with the same IP as the gateway on the host (x.1). for the guest also use /32 but gateway and pointopoint being your main vps IP (x.125) …

could be that you need to enable ip forwarding as well and add a route for the guest IPs to vmbr0 as well. or ask the provider for virtual macs to those addon IPs and set these for your VMs :man_shrugging:t2: :grin:

1 Like

As Falzo says, it could be MAC issues. You can go routed to avoid these, or have virtual MAC addresses from the provider.

Also might require

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

In there too.

Is it an additional subnet you’ve got from the provider or just additional IPs?

2 Likes

The next challenge is the VMs within your VM within your VPS.

image

6 Likes

I agree, would especially be interesting, if you manage to have the middle layer VMs not need any public IPv4 then, but pass them through to the deepest one :stuck_out_tongue:

How far can you go ?

I’m lost.

EDIT: I think you helped me to do something similar with my server with Hivelocity. I’m going to check the config and comment later.

Not sure if @seriesn offers this.

Didn’t work.

The host:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static

auto vmbr0
iface vmbr0 inet manual
	address x.x.x.125/32
	gateway x.x.x.1
	pointopoint x.x.x.1
	bridge-ports eth0
	bridge-stp off
	bridge-fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward

auto vmbr1
iface vmbr1 inet manual
	address 10.0.0.1/24
	bridge-ports none
	bridge-stp off
	bridge-fd 0

The guest:

IPADDR=“x.x.x.148”
PREFIX=“32”
GATEWAY=“x.x.x.125”
SCOPE="peer x.x.x.125

Just additional IPs

EDIT: Added the route-eth0 file in the guest:

x.x.x.125 dev eth0
default x.x.x.125 dev eth0

Still not working.

EDIT2: Internal network does not work either :sob:

try make it a seperate bridge for each IP, like in:

auto vmbr2
iface vmbr2 inet static
	address x.x.x.125/32
	bridge_ports none
	bridge_stp off
	bridge_fd 0
	bridge_maxwait 0

	up route add -host x.x.x.126 dev vmbr2

and maybe change the vmbr0 block to:

auto vmbr0
iface vmbr0 inet static
	address x.x.x.125/32
	gateway x.x.x.1
	pointopoint x.x.x.1
	bridge-ports eth0
	bridge_stp off
	bridge_fd 1
	bridge_hello 2
	bridge_maxage 12
	post-up echo 1 > /proc/sys/net/ipv4/ip_forward
	post-up echo 1 > /proc/sys/net/ipv4/vmbr0/proxy_arp

obviously use vmbr2 in the settings for your guest. the guest config seems good, though I am very bad with centos.

no guarantues though, as said before, there usually is not the one way to do it :man_shrugging:t2: :grin:

PS: for your local to get access to the public you need at least a nat rule, like

    post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE

Yello sir,

You should be able to simply add the additonal IP’s within proxmox control panel, since these IP’s are routed to your box :).

If you want and can wait till like 8 AM EDT (Jul 25th), I can lend a helping hand :slight_smile:

3 Likes

image

sorry, I just couldn’t resist :wink:

6 Likes

Can’t blame me. Has been ages since I have done a proxmox setup lol.

2 Likes

Please dont touch that networking crap in proxmox, it fucks up the entire shit you configured.

If you do, it’s like a proxy mocks you :wink:

Here’s what I’ve got on a setup where the additional IPs are on a different subnet to the main IP.

Host:

allow-hotplug eth0
iface eth0 inet static
	address 12.23.34.45/25
	gateway 12.23.34.1
	dns-nameservers ip1 ip2
	dns-search domain.com
	post-up echo 1 > /proc/sys/net/ipv4/ip_forward
	post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

auto vmbr0
iface vmbr0 inet static
	address 12.23.34.45/25
	bridge_ports none
	bridge_stp off
	bridge_fd 0
	up ip route add 11.222.333.441/32 dev vmbr0
	up ip route add 11.222.333.442/32 dev vmbr0
	up ip route add 11.222.333.443/32 dev vmbr0

auto vmbr1
iface vmbr1 inet static
	address 10.20.30.1
	netmask 255.255.255.0
	bridge_ports none
	bridge_stp off
	bridge_fd 0
	post-up iptables -t nat -A POSTROUTING -s '10.20.30.0/24' -o eth0 -j MASQUERADE

And then on the guest

auto ens18
iface ens18 inet static
	address 11.222.333.441
	netmask 255.255.255.255
	pointopoint 12.23.34.45
	gateway 12.23.34.45

Not sure if this is the most “correct” way to do it, and I use a different setup for additional subnets, but that does work.

3 Likes

probably more correct than mine :joy: as you keep eth0 as such and vmbr0 being a routed bridge device.
I am always using vmbr0 as bridged bridge (using vmacs with hetzner/ovh on that) and leaving the main interface without IPs at all.
however if one looks closely it’s not so different at all :man_shrugging:t2:

@imok if you want I can have a look later today and see if we can figure a way to make it work…

2 Likes