Exploring Proxmox from a VMware User's Perspective

all right friends it's time for the next installment on evaluating alternative hypervisors now that broadcom is making VMware untenable to use for hom laabs and small to medium businesses in the last video I focused on xcp as an alternative to esxi if you haven't seen that video check it out over here now it's time to focus on proxmox and compare and contrast proxmoxve to our once beloved VMware let's get to it hey there home labers and self hosters Rich here this video is the second video focused on evaluating your options if you're coming from the world of VMware and esxi with this video in particular being focused on proxmox Fe again my background is in VMware so I'm going to be looking at this from that perspective and giving my thoughts and opinions along the way as we always do I think we should start with some background on proxmox and the history of the product first the origin story of proxmox begins in 2005 with brothers deepar and Martin Mau two Linux developers who founded the company proxmox server solution gmbh proxmox has evolved significantly over the years into three main products proxmox virtual environment also known as PVE proxmox backup server and proxmox mail Gateway the first staple release of proxmox ve was initially released in 2008 the aim was to offer an easyto ous scalable virtualization platform that could manage both virtual machines and containers in a single system over the years proxmoxve has undergone numerous updates introducing features such as lab migration High availability and software to finded storage and networking these features have made it a strong competitor in the virtualization space challenging other established platforms like VMware and hyperv proxmox has continued to evolve with regular updates that introduce new features improvements and security enhancements recent versions have focused on improving usability scalability and integrating Cutting Edge Technologies like SEF for distributed file storage and ZFS for file systems the future direction of proxmox includes further developments in virtualization technology container management and security features along with a strong emphasis on usability and automation to make managing complex distributed systems simpler and more efficient proxmox server Solutions Prides itself on being an independent and profitable company that counts only its customers and supporters as its investors again after the drama we've been experiencing with broadcom it's refreshing to see a project and Company that's focused on the actual users and customers of their product and not profit first stockholders that are indifferent about the products and the people that use it but history aside how does PVE compare to esxi or even V Center for that matter well let's take a look let's start with architecture both proxmox ve and vmore esxi are type 1 hypervisors however there are key differences in their deployment methodologies esxi is very Lightweight by Design only containing a minimum footprint to function and once booted runs solely from Ram all of the VMR esxi kernel is closed Source proxmox ve is based on Debian Linux and once the boot process loads the kernel and other necessary components into RAM PVE continues to use disc storage for various operations needs onto performance while performance is highly dependent on what's running in your workloads these days performance between the two hypervisors looks to be nearly equivalent esxi has very widely reported maximum support limits for Ram and host per cluster and PVE does not now let's talk about usability both esxi and PVE have built-in management web goys for host in virtual machine management however that's where the similarity ends esxi embedded HTML 5 guei solely focuses on singular host in Virtual machine management any extended functionality requires V Center in contrast pv's built-in management interface has essentially the same basic host virtual machine and additional lxc container management functionality but also provides extended features for proxmox like clustering live migration and backup functionality once again it's tough to directly compare these two management interfaces for a few reasons for one the proxmox UI is more than just a single host VM and lxc container management system you certainly can manage those things and a whole lot more there's no vcenter equivalent for proxmox in terms of additional software the built-in guey does all of that you can build and manage clusters distributed file Services live migrations backups and more just from that interface now on to features esxi and Venter offer a wide range of advanced features including distributed resource scheduling High availability fault tolerance vmotion which is vmware's term for live migration of VMS storage v motion and API control however all of these features require additional Li Ling to unlock by default the best you get with esxi is basic VM management out of the box without V Center proxmoxve also supports clustering live workload migration live storage migrations High availability and workload balancing based on utilization for the most part proxmoxve is nearly feature complete with the licensable features of VMware out of the box for free now let's talk about scalability esxi is very well known for its scalability and is used in some of the largest virtual environments in the world esxi is fully capable of managing thousands of VMS without issue and when adding vcenter into the equation has the ability to scale clusters of virtual hosts out to hundreds of hosts with many thousands of VMS no sweat proxmox also features scaling but its adoption in very large or complex environments is less common compared to esxi proxmoxve natively supports clustering and will service and manage a large collection of hosts running large quantities of VMS and lxc containers and for the business-minded viewers let's talk about support VMware offers extensive professional support training certifications an extremely well-maintained public knowledge base and a large community behind it that being said with the changes in broadcom's new ownership it's unclear exactly how that will affect product support access to the KB and so on proxmoxve being open source relies more on Community Support however proxmox server Solutions gmbh the company actively behind development of proxmoxve offers four different levels of support premium standard basic and community and provides various options and service depending on your business requirements and needs and lastly cost here's where the rubber meets the road and for vmore esxi the most recent changes being made to licensing by broadcom Will effectively put VMware out of the reach of most people it's very likely the reason you're watching this video now and to add insult to injury as of February 12th 2024 the VMware esxi free hypervisor is no longer available proxmox ve is 100% free and open source with all features available without cost Professional Services are available for a fee with their premium tier of support costing at the time in this video € 1,020 per socket per year but overall it will be dramatically more costeffective especially for smaller cluster deployments of multiple proxmox Fe hosts or those with budget constraints all right overview out of the way let's look at some real world examples of the two side by side along with my thoughts on both as I mentioned earlier both hypervisors have a built-in web management gooey so no additional software is needed to manage your host once installed the first step here is to to take a look at the consoles of both and compare them here are both consoles side by side and right away we see a huge disparity between the two we'll start with the incumbent esxi the esxi console provides some information about the physical host like esxi version number OEM Hardware manufacturer processor type count and speed and RAM amount on the host as well as the URL for web management in terms of configuration pressing F2 and logging into the host provides you with very basic management functionality in essence the most you can do is configure or change your management interface settings enable support functionality like enabling SSH or local promp access enabling lockdown mode and that's about it via F12 you can manage shutting down or rebooting the host itself outside of these functions VMware expects anything beyond this to happen from within the built-in HTML 5 management web guiy in contrast the proxmox ve console is Spartan with a small block of text in the top providing you with the URL to access the built-in web guei and below a log Lo in prompt to log into the host with the root user and pass that you set up during the initial installation beyond that all management for the host needs to be done via the web gooey the disparity between these two is pretty Stark and coming from the world of VMware I wish proxmox provided more information or a simple text based menu UI for the host via the console I've been in situations where network connectivity was unavailable and you needed to do troubleshooting making that process easier via built-in menu system is invaluable I'm sure I'm going to hear in the comments about how you can do everything you need to do VI the command line once you're logged in and I get that but not all support Engineers have deep Linux knowledge so providing some basic ease of use from the console is a real value ad a6i wins here but consoles aside let's dig into their respective management goys and the differences between the two now as I mentioned earlier both systems have a built-in web goey but those goys are not exactly Apples to Apples because e6i essentially does just VM and host management only and proxmox is an entire management Sol solution morein to V Center as I did last time I'll be calling out the differences as we move through this if you watched our video on xcp andg you can skip to the next timestamp as we'll be showing the same example of the esxi web guy we did in the last video this is the web management UI for esxi landing on the main dashboard you're given a summary of all of your hosts State usage V switch and Port group configs data stores mounted and free space system information and add a glance graph of overall CPU and RAM usage on the host on the bottom in the recent task pane are all the active and recent tasks that are running or have occurred on the left is your navigation pane where you can dig deeper into your host configuration and monitor your performance Hardware events logs and more in the virtual machine tab you get a full list of the running VMS on the host details about storage usage guest OS config host name CPU and memory usage when you drill down on a specific VM you get health utilization and configuration information snapshot and access to the console as as well as all the standard VM management functions like power suspend restart and Hardware configuration options in the storage tab you get an add a glance view of your mounted data stores their utilization and type as well as the ability to create new data stores across the top you view and manage your virtual storage adapters devices and so on when drilling down into a configured data store you view further information about the storage type location host connected virtual machines connected to it and with the data store browser at the top browse the storage system upload and download data and otherwise manage the data store as desired the last tab to touch on in the web guey is the networking tab at a glance you see all of the port groups configured on your host and drilling down into any of them provides you with detailed information about the port group VMS connected V switch that serves the group and a visualization of how the VMS with network interfaces assigned to that Port group connect to the physical Network backing out to the virtual switches Tab and viewing your configured V switches you get an incredibly Rich level of information about all virtual networks on your hosts the VMS connected to them and how they connect to your physical networks the main networking tab is also where you can view and manage physical network adapters on your host VM kernel Nicks TCP stacks and firewall rules this is the web management UI of proxmox there are a lot of design cues that feel like they were inspired by esxi and venters management goys and I think that's a good thing on the left we have a navigation panel similar to VMware with a drop- down menu that provides three different views of your system the different views feel more reminiscent of vcenter than ESX I which makes sense since proxmox like xcp features all the functionality of Venter compared to esxi across the top we have shortcuts to access documentation create VMS or lxc containers and access various user settings at the bottom of the screen we have the task list and cluster log again very reminiscent of esxi and Venter user interfaces on first login we land on the data center summary page and begin to get a glimpse at the UI differences that proxmox has compared to esxi with the nested tabs that are context based depending on the primary tab you choose in the navigation menu on the left the data center tab provides a holistic view of all connected host statistics cluster configurations SEF user management and permissions High availability configurations and management software to find networking configuration and a lot more changes to settings here can affect multiple hosts that are part of the collective data center dropping down one level we land on our host name proxmox and the associated management configurations and settings that directly affect our host only similar to the data center level we have a search tab useful for searching for VMS containers and network and storage resources on the host the summary tab provides detailed information about the host such as CPU usage load averages RAM and storage usage IO delay swap usage and a good overview of the CPU details kernel version repo status and more proxmox also provides great graphs of historical resource utilization with a drop- down option to select greater time frames next is the Note Tab for keeping notes which is probably useful to some somebody followed by the shell tab which gives you direct console access to the host without the need to bust out SSH to do so xcp has a similar feature and I think this is great the system tab gives you a full list of running demons on your proxmox host and allows you to start stop and restart them easily as well as a view of the syis log of the host for troubleshooting the network subtab provides you with a list of all the dedicated physical network adapters Bridges and other interfaces that currently exist on the host for proxmox this is going to be the simplest place to manage your network connections on your post as previously mentioned proxmox also has a full sdn stack built into it but managing connections here is the most straightforward approach certificates is for managing and replacing CTS on your host DNS allows you to add change remove DNS servers from your network stack host provides you direct access to the host file on the underlying debn OS I suppose if you wanted to add static entries over using DNS having access directly would be convenient options allows you to configure start on boot delay and wake on land functionality time allows you to add change remove and TP time servers from your host and the CIS log tab gives you real-time access to your host's running CIS log with options to view the live syis log or use time and date options to select a time span super helpful for troubleshooting issues the updates tab shows you a list of packages that are out of date and need to be updated and provides an upgrade button at the top to kick off the update the next stop is the repositories tab that shows you all of the app repositories your host uses for updating including by default the PVE Enterprise repository which is only accessible with an active sub description you can see I've also added the non-production SEF repo here as well which is necessary to install SEF on the host this leads me to a complaint proxmox is free and fully open- sourced yet the default installation has an app depository enabled that is only accessible to paying customers and until you disable that repo you'll be constantly nagged about it yes it is easy to disable that repo but why not just leave it off by default and automatically enable it when you apply a subscription key to your host moving on the firewall tab allows you to create firewall rules options and view log related to the firewall esxi does have basic firewall functionality built into it but it's buried and nowhere near as prominent as it is in proxmox diss allow you to manage the local diss attached to your host you can create and manage lvms thin provisioned lvms directories and create ZFS pools for storing your VMS and containers this diss tab is only for physical dis Management on your host to manage connecting NFS ice scuzzy and a variety of other network storage types you need to access the storage tab backup and data center next is the SEF tab SEF is an open source distributed storage system that provides object block and file level storage and is built into proxmox which makes it super easy to build out clusters of hosts that share storage and benefit from seph's self-healing capabilities and more I installed seph on this host but I only have a single proxmox host running and I haven't fully configured it yet which is why I have a health warning next is replication proxmox also features native replication of workloads between two or more proxmox hosts task history shows you a full list of the tasks and events that have occurred on the host this is a full list view of what you see in the task cluster log panel at the bottom of the screen and lastly is the subscription tab this is where you'd add your license key if you're a paying customer and so on clearly there's a lot to the prox MOX management UI and we're only really focusing on host level management since we're trying to make a comparison between esxi and proxmox and not vcenter in prox MOX so let's take a beat here and focus on Virtual machines specifically since we haven't touched on them quite yet the UI and prox MOX Aggregates all of the VMS lxc containers Network and storage resources under the host in the server view pane let's take a look at how proxmox manages VMS and then we'll build one out so we can get a feel for the differences and similarities to esxi building a VM is pretty straightforward though swinging up to the top right and clicking create VM starts the wizard that walks you through the VM creation process you'll give your VM a name that you can't have any spaces in that name then onto OS you'll choose your install ISO which you'll have needed to upload to a storage location ahead of time and then choose your OS of choice unlike VMware when you choose to install a Linux OS you're only required to choose between which kernel version your VM is using however for Windows you can be specific on which version of Windows OS you're planning to install selecting Solaris and the other category gives you no additional versions to choose from in the system tab there are a bunch of options for Hardware emulation to provide down to your VM from a variety of different graphics cards including VMware compatible which is interesting to see machine types and bios options between basic bios Boot and ufi interestingly when you choose ufi you also need to choose where to store your ufi boot dis something you didn't need to do in esxi another nice thing to point out here is that proxmox has native TPM support for VMS as well something that esxi without Venter does not here too you need to choose where to put your TPM data but this makes more sense from a security standpoint proxmox has quite a few storage controllers to choose from including yet again another VMR option I'm starting to see an interesting Trend here building out storage devices for your VM is pretty straightforward as well choose the bus type dis size and you have options for disc cache types something that doesn't exist in esxi interestingly there's even a tab for allowing you to configure bandwidth limits for your vm's storage device I find this neat but problematic especially if you make configurations here and forget you did them and then run into performance issues with your VMS provisioning your vcpus is also pretty straightforward choose your socket and core count but then you can also choose your CPU type I'm kind of fascinated by this because compared to e6i this sort of thing was forbidden I'm not entirely sure what would happen if I told my VM that its CPU was an epic Ram CPU when its underlying physical CPU is actually an Intel E5 2680 V4 but I suspect nothing good can come of that the memory tab is just that selecting the amount of memory to provide to your VM the network tab again is pretty straightforward you choose the interface that serves your host enter a VLAN tag if you want proxmox to tag that vm's traffic for a specific VLAN and then you can choose from four four different virtual Nicks including a very familiar vmx net 3 which for us VMware people stands out right away the last T is just a summary of all of the options we've selected and clicking finish kicks off the VM creation Now we see my newly created VM on the left named Oracle vm2 and if we swing over to the console tab we can start the VM by clicking start now from here life is essentially the same as it would be for a VM created in esxi so that's a comparison of the two interfaces now I'm going to talk about the things I like dislike and the good and bad again coming from my vmore esxi perspective feel free to disagree with me in the comments of this video or dare I say on our Discord server anyway after using proxmox for a solid two weeks now and getting comfortable with all facets of host in VM management I can say that it will effectively satisfy your needs as a replacement for esxi it has all the general bells and whistles features and options and honestly a lot more out of the box so let's talk about the things I don't like first off let's talk about the UI my first impression of the proxmox UI is that it's a hot mess it is super busy there are options on top of options and when you first log into it you really don't get a good feeling on where to start there are Oddities like right clicking in some areas will give you a proxmox context menu but others will give you the browser context menu I understand why there are options on top of options in proxmox and that's because they provided you essentially all of the management functions you'd be able to do on a Linux box in the guey but do we really need to be able to edit the host file on the UI just asking another thing is the constant nagging about not having a subscription when you log into proxmox boom nag when you go to check for updates boom nag and that is even with the Enterprise repo disabled I get it subs pay for engineers to make proxmox and people need to make money to live but being reminded continuously about it isn't going to get you more subs which also leads me to reiterate my complaint about having the Enterprise app repost enabled by default turn them off by default why are you making that a manual step for your users make it so that when users purchase and enter a support key they get that turned on automatically don't make it their problem the next thing I wish proxmox had was a real console interface I said it before and I will say it again there are times when a virtual host may not be accessible via the network for whatever reason and you have to interact with it via the console expecting everyone who administers proxmox to have a deep enough Linux knowledge to be able to drill down and fix something broken by the command line isn't a reasonable assumption if proxmox wants to be fully accepted by businesses as a replacement for VMware they need to have multiple user-friendly means of managing their boxes because an IT department isn't going to be fully stocked with Linux nerds oh and then there's the upgrading issue patch management for your host works well and I appreciate the appt integration in the guey patching to keep your host updated and secured easily is a necessity these days and proxmox has done an excellent job of making that easy but upgrading to the next proxmox version isn't easy the process from moving from version 7 to version 8 is messy and their Wiki page is long and daun in comparison to esxi update process it's a mess and the risks are very high another thing is that there isn't an easy way to migrate your VM workloads from esxi or Venter into proxmox and there needs to be referred to the last video where xcp added the feature to migrate workloads entirely automatically from VMware to xcp this is going to be a necessity for anyone serious about moving from VMware and my last complaint I don't know why proxmox supports lxc containers I really don't see the advantage of lxc over traditional virtual machines sure you can shave down even more RAM usage and you can share gpus easier between lxc containers but you can't live migrate than between hosts so they're really not fault tolerant and I feel like being reliant on the underlying kernel is a security risk compared to a fully self-contained virtual machine and if I'm being honest my big disappointment was finding out that when people say prox MOX has native container support they're not talking about application containers like Docker or k8s which is kind of what I was hoping for I'm sure I'm going to catch hell in the comments for that but I mean I can't be the only person who thought containers meant application containers right thankfully that's really about all the complaints I have proxmox has a ton going for it I had a really hard time not slipping off the esxi versus proxmox subject because so much of proxmox reminds me of vcenter from its navigation panel to all the VMware virtual Hardware options like VMware compatible VGA VMware par virtualized scuzzy and VMware vmx net 3 all of which I'm not sure how they're able to include since I thought that that was closed source but regardless that makes running a migrated VM workload a lot easier you can create clusters of hosts live migrate workloads around and even enable High availability and that's the stuff of Venter and it is built in oh and proxmox has native VM backup functionality built in no need to have third- party backup software to protect your workloads again another thing that required a license to do in VMware so in summation if you are coming from the world of VMware or assumed to be X VMware person or company using VMware look looking for an alternative direction to migrate to proxmox certainly has a ton going for it and is worth checking out one last thing there are even more hypervisors out there if you have an interest in a different hypervisor and want us to take a look comment below or get in our Discord and that friends will do it for this video if you liked it throw us a sub and a like and if you have a beef with anything I've said here let me know in the comments below special thank you to our YouTube members you guys help keep the lights on and we thank you for it if you'd like to help support the channel consider becoming a member or buy some of our sweat it all helps us keep making these videos and now that you finish watching this video how about checking out this place over here of other great self-hosting and H Lab videos we've done in the past if you're looking for your next great H lab idea we can help

2024-03-01 10:44

Other news