How to Plan a Homelab

If you have an interest in IT then a homelab can be a great way to improve your skills, and they can be a lot of fun. They can get pretty out of control though if you let them; so here are my tips to help you build a cost-effective homelab that can actually save you money, whilst improving your skills.

This is not going to be a step-by-step on how to install your homelab. There are plenty of manauls for that, and they all by nature have to assume that you want your homelab to look exactly like theirs. No, this is going to be about helping you plan for a homelab that fits in with your lifestyle, your budget, and meet your needs.

Let's answer the obvious question first - what is a homelab and why would I want one? A homelab is a set of IT equipment in your home that replicates the same sort of functionality you'd expect to find in a professional infrastructure. It acts as a safe space to play with new technologies, somewhere you can run personal projects, and it will help you work towards certifications. It is not a replacement for a test environment at work. You can play with new ideas and technology, sure; but what you can't do is replicate your actual business systems. At least I hope you can't! And you shouldn't either. A homelab is supplemental to what you do at work.

Maybe your employer uses Windows and you want to play with Linux. Do it in your lab. Maybe you want to mess around with home automation or media streaming. Do it in your lab. This is for people with an interest in technology and their own personal development. If you're sat there thinking that your employer should be paying for this all then maybe it isn't for you. If you're an employer sat there thinking "Hey, this is great. I can save money - I don't need a test environment anymore." then when your employees are forced to experiment on your live systems, you kind of brought it on yourself.

Topology. How complex?

The general rule I follow with a homelab is keep it simple. Benches and benches full of equipment might look impressive but they're going to cost you more in terms of running costs, physical space, and noise. All of which I'm going to assume is at a premium in your home. Unless you actually live in a datacentre; and going by some Google searches, some of you guys clearly do think you live in datacentres.

In order to keep that footprint as small as possible you will almost certainly want to use virtualization. That allows you to run multiple logical servers inside one physical box. There are scenarios where you'll need specific physical devices, such as studying for certain certifications; but these days that's the exception rather than the rule. Most things today can be virtualized, and in a professional environment most things will be virtualized, so that's definitely the way you want to go for your lab.

The simplest topology you can start with is a single box that contains a hypervisor and local storage.

If that sounds unimpressive, then remember by the power of marketing you can legitimately refer to that as a hyper-converged infrastructure. That comes with the obvious downsides of having all your eggs in one basket, but this is just a lab, remember. If you're worried about high uptime then I have to question whether this is actually a lab, because that's starting to sound more like production.

If or when your lab breaks then you should view this as an opportunity. You're the only person who's being inconvenienced, so now you've got the chance to learn how to fix it so that when it happens in the real world, you know what to do. My recommendation, therefore, is to start with a single hyper-converged box; and you scale out if your learning needs demand it. There are reasons you might need to scale out. If you're working towards a virtualization or a storage exam for example, you might need to replicate a highly available environment. So you might need multiple host servers, or shared storage, or replicated shared storage.

Bear in mind though, that as soon as you start doing that you effectively double, triple, or even quadruple your running costs. With a modern CPU that supports nested virtualization, you can actually run a hypervisor inside another hypervisor. That could allow you to run all of that inside a single box - which could be fine if you just need it temporarily for an exam.

If you're just starting out in IT and you don't have any equipment - no spare computer, or budget for second-hand kit, then there are still things you can do. If you have a copy of Windows 10 Pro then you can run a cut-down version of Hyper-V on it. If you don't have Windows 10 Pro, then you can always install a copy of Oracle VirtualBox or VMware Workstation Player - each of which is either free, or has a free edition. They will both allow you to run virtualization on top of your existing computer. If your computer can't handle that either, you can always dual boot into a different operating system. If your computer can run Windows, it can run Windows Server, and if it can run that, it can also run Linux.

Hardware. What kit should you use?

Once you've decided on your topology, the next question is: what hardware are you going to use? The specific components might vary a little depending on software compatibility, but what I'm really focused on here is what type of hardware should you use, and where could you get it from?

One of the best ways to finance a homelab is to use second-hand kit. That could be an old computer you have lying around, something you got off eBay, or even some proper enterprise kit that's been retired from your place of work.

Careful with that last one. I know a lot of IT pros have the opportunity to come into possession of decommissioned enterprise kit; but decommissioned or not, make sure you have the permission of the previous owner. Because if you don't, that's stealing, and I'm not advocating that. And for goodness' sake, please make sure there's no data on it!

Second-hand enterprise kit can be a really good choice for a homelab, and it generally has quite a bit of life left in it. That's because a lot of businesses will get rid of their kit after about five years. That's not because it's broken - that's a risk mitigation. They're getting rid of it to reduce the likelihood of them being impacted by a hardware failure; but for a homelab you can afford to take more of a risk. Remember what I said before: if it breaks, it's only you impacted, and now you've got the opportunity to learn how to fix it without a few hundred users screaming at you.

Although second-hand enterprise kit can look like a good deal, don't forget to factor running costs into your calculations. In the space where I keep my homelab I have a nice deep shelf that's about 19 inches wide. It is crying out for some enterprise-grade rack-mount kit, and I had some available to me when I put the lab in. But I did the numbers, and they weren't good. For the same price as running one of those servers for three years, I was able to build and power a lab for six years using mostly brand new kit. The main reason for that is I used commodity desktop hardware rather than actual servers. That might not sound as exciting, but it's a lot more practical for several reasons.

Most servers are designed to sit in a datacentre, not your house. They're not really designed with families and quality of life considerations in mind. As well as sucking up a lot more juice, they can make a lot of noise. The small ones in particular can sound like they're getting ready to take off! First and foremost, this is home. So be realistic but where you've space for this, and whether your family will appreciate the sound of a 1U server's fans screaming as it installs updates.

Also bear repair costs in mind. If a desktop beaks, you can quite easily get replacement parts fairly cheaply. If a server breaks, the parts are a lot more expensive.

From a functional point of view, there's basically no difference between a server and a desktop. Both will serve your needs equally well. The advantage of a server is about reliability and management; which is really important for a business that's reliant on that thing to keep running, but less so for a lab.

Speaking of hardware costs, I really can't recommend buying new server hardware for your home. I mean do you really need to spend money on a Xeon processor, ECC memory, and SAS hard drives for your home? Well, I know some of you guys are like "Oh yeah, of course you can - it's fun!", and yeah, I can't really criticize that if it's a hobby. Some hobbies are only justified because they're fun, and not for any other reason; but if we're looking at this critically... that's just a waste of money. For me, I'd stick with commodity desktop hardware.

If you have a steady stream of server kit available for maintenance, then fair enough, but you'll probably still pay more for running costs. If you don't have, or don't want to use second-hand kit, then one of the most cost-effective ways to get a homelab up and running is to build your own from desktop parts. I say "build it yourself" because if you're trying to use a desktop as a homelab, one of the biggest constraints you're going to have is how much memory you can fit into it. If you want to run a sizeable lab and you're trying to get that off the shelf, you're going to end up staring at some kind of ridiculous gaming PC with a massively expensive graphics card and enough RGB to rival Las Vegas.

"Homelab" isn't exactly a configuration you're going to find at PC World, and because of that I'm suggesting you build it yourself. If you don't want to do that, you can always use a website that lets you pick out the parts and they'll build it for you.

Software. Pick your platform

Here you have plenty of options. I'm not going to be too prescriptive, but I will mention a few free ones. If you were to buy all of the software that goes into a typical enterprise environment, you'd need to take out a mortgage! Fortunately, a number of vendors are sympathetic to this, and they will give you free or drastically cut down in price versions of their software; provided you don't intend use it in production. They're not being as charitable as you might think. Their intention is that you like the software, so you bring it with you to work, and then they can collect a licensing fee. So they're not doing it out of the goodness of their heart, but you may as well take advantage.

Hypervisor. Virtualisation software

The main choice you have to make is what hypervisor are you going to use? In some ways it might not matter, if you're more interested in what sits on top of the hypervisor; but unless you never deal with virtualization at your job I suggest you be intentional with this choice. Let me give you an example. Most of my customers use VMware. A small number use Hyper-V, and anything else is basically an edge case for me. Based on that I might build a VMware lab because that's what I'm going to use the most of at work. Or, alternatively, I could say I do so much VMware, I'm sorted for that. I'm going to do a Hyper-V lab instead, and keep my skills sharp on that as well. There's no right or wrong answer here, but have a think about what you want to get out of your lab, and then be intentional about your choice to help you achieve that.

In terms of free options, you're pretty spoiled for choice here. KVM and Xen are open source, and Hyper-V and VMware both have free editions available. If you're running Windows already, you might already look at Hyper-V as free. "Included" is a better way of putting it because Hyper-V is included in Windows, but Windows isn't free. That said, there is a fully free version of Hyper-V called Microsoft Hyper-V Server, and you don't need a Windows licence for that. It's basically Windows Server Core, with all of the roles apart from Hyper-V removed. It is fully featured, but if you want to manage that with a graphical interface you're going to need a copy of Windows to run that on.

VMware is the undisputed king of virtualization, and they have a free edition of ESXi. VMware is used in most enterprises, so it can be a really good choice for your homelab. Unfortunately though, the free version of VMware (unlike Hyper-V) is not fully featured. Notable missions are a lack of clustering and storage APIs. The lack of clustering might not matter if you're just running a single box in your lab. The lack of storage APIs basically rules out the use of virtualization aware backups. You can still back up your VMs if you want using traditional agent-based methods, but they're just not as good. VMware also tends to be a bit more picky about specific hardware compatibility than, say, Hyper-V. Windows Server has the advantage that its desktop sibling has to run on pretty much anything; so there are drivers for pretty much anything. VMware doesn't have that desktop operating system equivalent to steal drivers from, so support is more limited. As a general rule, server hardware is probably going to work, but if you're running it on a desktop you should do a bit more research to make sure it's compatible.

I don't mean to say you need to stick rigidly to the hardware compatibility list for a lab. To give you a sense of things, I have two desktops at home that at some point have had VMware ESXi installed on them. One of them worked straight out of the box. The other one needed an add-in network card because the onboard NIC just didn't work with it. The other big decision is your storage platform.

Storage. Software for your data

Storage is obviously important as it underpins a lot of everything else, but most IT pros spend less time working with their storage than they do with their VMs or their hypervisor. There's also a lot more options out there - it's more difficult to narrow this down to a few big players as it is with virtualization for example; and chances are you're going to work with a lot of different storage vendors over your time. For that reason you can be a little bit less intentional with this choice, unless you happen to specifically work with storage.

If you're sticking with a single box then you don't need to do anything complicated - you can just use local storage. If you're scaling out in the Windows world, then Storage Spaces Direct is included, and will do the job. For anything else (or for Windows for that matter), you could use something like FreeNAS, which is open source and built on FreeBSD. There are a number of software-defined storage vendors out there. If you use one for work, it might be worth sticking with something familiar. If you don't have a preference thought, then between local storage, Storage Spaces Direct, and FreeNas, you can all your bases covered.

Other Software Options. Licensing for R&D

Free software options aside, it's worth talking to some of the big vendors to see if they have an NFR or "Not for Resale" licence available for IT Pros. Sometimes you need to be a partner or pass some exams for this, but not always. Some of them are quite happy for IT Pros to learn their software in the hope they'll bring them into production, where they can get a licensing fee.

Another option is to talk to your employer. Sometimes they might have some licences available, and they'd be willing to let you use some of them because they're going to see a benefit from you improving your skills. Some vendors like Microsoft have specific licensing programs for research and development, so if you have an MSDN, or I think it's now a Visual Studio subscription, that might cover you for all of your needs. If you're a student at university, they're also pretty good at targeting you guys. Microsoft want all of the IT Pros and developers of the future to come into the Microsoft fold, so they'll quite often give away free software to students. If that's you, you may as well take advantage of it.

Finally, don't forget about evaluation copies. If you're working towards a certification and you just need the software for a limited time, an evaluation copy will probably do what you need.

Cloud. Why build it yourself?

One option I haven't mentioned yet is cloud. These days you might rightly question why someone would choose to build an infrastructure in their home when they can just use the cloud, and it's a good question - particularly if you have free credits from something like an MSDN subscription. If you want to do something temporarily, then the cloud can be a good bet; and if you want to learn about cloud services then it's pretty much essential. The main problem with the cloud is cost. Particularly if you're going to be running services on a permanent basis. There's also the issue that it doesn't teach you anything about the hardware, but mainly it's about cost. With a homelab, you want to keep things as cheap as possible. That means reusing the second-hand kit, and using consumer-level hardware. You just can't do that with the cloud. I can't take the hard drive out of my old desktop and then tell Amazon I don't want to pay for their cloud storage because I'm going to plug this in instead. It doesn't work like that.

To give you an example, I built my current homelab for about £300, reusing some existing hard drives. I have since upgraded it with some solid-state, so for the sake of argument let's say £400. I intended that lab to last me for six years, although honestly with the SSD upgrades it's going to last me longer. I run services in that lab permanently. There are things I need going 24/7. So if I was to take the same services and put them in the cloud, ignoring practicalities like bandwidth and latency; how much would that cost me over six years? Using the Azure calculator, taking advantage of burstability where appropriate, and reserved capacity where appropriate, it works out at... £80,000! Spread that across a few hundred people over six years and it probably looks quite reasonable; but am I as an individual going to pay that for my lab? Am i heck! I'm going to need a lot more subscribers before I can afford that! In fact, I could claim my lab is cost neutral, if not profitable; and that's before taking into account potential earning improvements by being better at my job. If I got rid of my lab I'd have to pay roughly the same amount for cloud storage for my files. Throw in media streaming services, home automation, the ability the hosts personal projects such as this; and it's very easy to build a compelling business case for a homelab environment.