As a lot of NethServer Users are contemplating using / or are already using Proxmox as a virtualization environment this page will contain advice, know-how and generic tips. All this is still WIP (Work In Progress) as of 10th May 2020.
Andy_Wismer
Different Concepts / Scenarios / Topologies
Here are a few typical scenarios for using Proxmox:
- A Simple Home Network
- A Network for SME small and medium enterprises
- A fully redundant Network
Most of these concepts are just as valid in other virtualization environments, it doesn't matter which (VMWare, Xen Hyper-V…). A separate backup / storage is always a good idea. The saying - don't put all your eggs in the same basket - is quite valid here.
Redundant Networking using BONDING or faster (10 GBE) networks works for everyone and all platforms.
A few premises on the networks below: We're using for this docu the network 192.168.191.0
Network: 192.168.191.0
Subnet Mask: 255.255.255.0 (CIDR notation: /24)
Gateway / Firewall: 192.168.191.1
Proxmox Server 1: 192.168.191.61
This is intended as an example, use or choose your own network. Don't use eg the same network as your employer, or you're never going to get a VPN there running. Change the 3rd number in the IP adress (here the 191) to anything of your choice, but try to avoid networks you might connect to. Also avoid the networks 192.168.0.0, 192.168.1.0, as these 2 networks are used by a lot of devices on the market. You might buy a new Wireless adapter, only to discover after connecting it up, that your internet isn't working anymore, as both use the same IP 192.168.0.1…
Gateway / Firewall
You can use your existing firewall / router provided by your Internet provider, at least for starters.
For more advanced uses, I'd strongly suggest using a dedicated firewall. You can build your own, using for example hardware by PC-Engines (3 or 4 NIC models) or some other suitable brand and installing suitable firewall software / OS, like OPNsense (free open source like NethServer, NO paid for enterprise version with closed source features). Or you can buy a good firewall, like the OPNsense series, and support the distro.
I also have friends running their home network behind a virtualized OPNsense firewall (This could also be your NethServer…), inside Proxmox. This runs stable and surprisingly well (After a Proxmox reboot, I can access the LAN with VPN in 3-5 minutes…), but I wouldn't advise using a virtualized firewall for a company, unless the situation requires it (Like virtualized, hosted networks).
Note:
These concepts are designed for long term. The hardware may be replaced, but the concept stays valid.
→ Your new car still has the same positions for gas and brake pedals, and still uses a steering wheel in front of you… They still don't give you the option for a Joystick as a steering wheel replacement.
A simple Home Network:
Who is this for?
This scenario is suitable for Home Users, small companies and Power Users.
Requirements
- Time - about one day max!
- Internet connection
- Suitable Hardware (see below)
Hardware
A usable “Server” for Proxmox can also be a good PC. However: A PC is NOT a server, just like a Ferrari car might be fast and powerful, but it's just the wrong choice to pull your plough in a field if you happen to be a farmer…
Besides which, the most powerful PCs available on the present market happen to be “Gamer” PCs, and these are designed to be replaced in half a year, a year max! Gamers want the latest hardware and Graphics!
Servers are built to run 24*7, have generally a higher energy efficiency for long term usage, and provide less issues in long time usage, also come with IMPI (HP-ILO) for management. Disk containers / RAID are other benefits. My suggestion would be to use a “real” server, new or second hand off the bay. My personal choice would be HP Proliants (20+ years experience), but a Dell Server might be just as good.
- Min. RAM: 16 GB (Proxmox uses 1-2, the rest is for VMs and LXC Containers)
- Min. CPU: 2/4 cores, Intel or AMD Virtualisation support
Disks in the server depend on if you're planning to use shared storage (NAS) or local storage (Your Server). Performance depends strongly on what RAID your server has or uses. A lot of so called RAID are actually FakeRAID, without any intelligence, using CPU cycles to replace in Software a “real” RAID Controller. My experience says in most cases NAS storage is better… I put in a small SSD for Proxmox system use, all the rest is stored on NAS.
NAS
A NAS is used in Proxmox environments as Backups - and also for storage, depending on the environment. Choose a “built” NAS, NOT an old PC with more / newer disks. This is usually slower, less reliable (It's old!), eats much more power and produces usually much more noise! A good choice is one having more HD bays than currently required, to make adding on disks in future relatively painless. That means: Don't choose a 2-Disk NAS! 4 or more is the rule! Synology, FreeNAS, QNAP are good choices for a NAS.
This NAS can also be used to store media, data, and anything else one could put on a NAS.
The NAS should have an independent backup, ideally daily of the whole NAS. An external USB3 Disk is a good suggestion. This can protect you against outages of your NAS but also against Crypto-Ransomeware, simply as this disk isn't accessible from any PC or from the network. Only the NAS internal processes can use that disk. (Make sure you set it up as such, and not “share” that backup over your network!).
If using shared storage - your VMs live / run off the NAS - get more performant disks, like WD Red Pro or SSDs.
A medium sized Network for SMEs
A fully redundant High Availability Network
Specific Proxmox Tips
Proxmox Backups & /tmp
This also should be set:
nano /etc/vzdump.conf
change this:
tmpdir: /tmp
→ Improves Backup times to NAS. The temp file is used to create the zipped backup, more efficient wenn done locally, instead of over NFS.
Proxmox & HP ROK Server Licenses & Dell equivalent
If you buy a new HP or Dell Server, there’s often the option to get a discounted MS Server thrown in. If needed, this can be a good price discount.
HP calls this ROK, Dell has it’s own name. Proxmox can support this quite easily!
The Trick is installing dmidecode:
apt-get install dmidecode
Then run dmidecode, eg as follows:
dmidecode > dmi.txt
Note: dmidecode produces a “lot” of output, that's why I put it into a file…
This will give you the HW license, to copy paste into Proxmox.
TIP: search for UUID…
Handle 0x008A, DMI type 1, 27 bytes
System Information
Manufacturer: HPE
Product Name: ProLiant ML350 Gen10
Version: Not Specified
Serial Number: CZJ123456H
UUID: 36373738-3132-5A43-4A39-123456789548
Wake-up Type: Power Switch
SKU Number: 812345-123
Family: ProLiant
The info is all there and labeled.
Now just fill in these data (copy & paste) to the SYMBIOS settings of your specific VM as shown:
Note: If using migration to another hardware proxmox, the SYMbios settings get migrated too!
This means that your Windows will “see” valid hardware, so NO issues. If using a KVM CPU, this even works after migrating from Intel to AMD hardware!
This works very well!
Dell Specific
On some Dell server you still get the 'ROK.vbe' script error during setup phase, to fix it we need to pass additional args:
smbios type=11,value=x
You can't add it via WebGui in Proxmox, but we can easily edit the config file of the vm.
ie. your VM ID is 600, edit file /etc/pve/qemu-server/600.conf
and add at the bottom:
args: -smbios type=11,value=x
Save and start the VM.
Nethserver in Proxmox: enlarging your Nethserver disk
This takes only a few minutes!
A full backup by Proxmox before is a good precaution!
Under the Proxmox GUI:
1. Click on the NS7-VM > Hardware > Hard Disk (xxx) > Resize disk > 1500
On the NS7 Console:
2. fdisk /dev/sda > p > d > 2 > n > p > 2 > First sector <Enter> > Last sector <Enter> w
3. reboot the server If you get this:
WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8)
4. pvresize /dev/sda2
5. lvresize -l +100%FREE /dev/VolGroup/lv\_root
6. xfs\_growfs /dev/VolGroup/lv\_root
Finished! Not really needed, but a Reboot and check is not a bad idea…
Note:
This method will work for almost all known VM virtualizations, including VMWare, XEN, VirtualBox, Hyper-V - at least on the NethServer side.
Downloads are very slow with Nethserver in Proxmox
You aren't the first trying to download an ISO file after setting up your Proxmox and Nethserver network and noticing slow downloads. This can occur if you're trying to test with downloading the NethServer ISO Image. NethServer provides their ISO Images from Sourceforge (AFAIK), and you get very different speeds, depending on your location. I may download in a couple of minutes, others, eg in the US with good Internet connectivity take more than 4 hours to download the same 1 GB ISO Image.
→ Test with something else, eg an ISO Image from Ubuntu!
If your download there is fast, the problem is not your Network, it's Sourceforge!