Proxmox Thread - Home Lab / Virtualization

MarioBros said:
Hey Proxmox experts.

Disaster stuck 2 days back, I was trying to install a smart switch and had to turn off my mains power at home, once the mains was off within 5 mins my UPS failed and my Proxmox server was powered off.. once the server was booted back I noticed that the proxmox local-lvm status was unknown... the local-lvm had all my VM & CTs including most important HomeAssistant server...

I tried troubleshooting going thru few forums and google but nothing worked... I followed one of the guides which mentioned about changing the content type in the Prxmox hardware storage settings and rebooted the server and that is when I realized I had f**ed up, the server never came back online :( I lost all my VM and CTs and had no backups of these (which was my mistake, only positive note is my Homeassistant backup was available on GDrive)

Last 2 days I have spent hours with fresh installation of Proxmox and all the VMs & CTs

How could I have avoided this? what is the best way to backup Proxmox ?
Click to expand...
I feel really sorry to know this.

Hopefully some experts would share good suggestions regarding how such a disaster could be prevented.
 
MarioBros said:
Hey Proxmox experts.

Disaster stuck 2 days back, I was trying to install a smart switch and had to turn off my mains power at home, once the mains was off within 5 mins my UPS failed and my Proxmox server was powered off.. once the server was booted back I noticed that the proxmox local-lvm status was unknown... the local-lvm had all my VM & CTs including most important HomeAssistant server...

I tried troubleshooting going thru few forums and google but nothing worked... I followed one of the guides which mentioned about changing the content type in the Prxmox hardware storage settings and rebooted the server and that is when I realized I had f**ed up, the server never came back online :( I lost all my VM and CTs and had no backups of these (which was my mistake, only positive note is my Homeassistant backup was available on GDrive)

Last 2 days I have spent hours with fresh installation of Proxmox and all the VMs & CTs

How could I have avoided this? what is the best way to backup Proxmox ?
Click to expand...
To my knowledge, your physical volume or logical volume is lost, but not deleted. You'll need to locate it and bring it back online. If you're using Logical Volume Management (LVM), recovery is bit easy.

Use logical volume scan command to get it back.
 
  • Like
Reactions: alpha5555
MarioBros said:
Hey Proxmox experts.

Disaster stuck 2 days back, I was trying to install a smart switch and had to turn off my mains power at home, once the mains was off within 5 mins my UPS failed and my Proxmox server was powered off.. once the server was booted back I noticed that the proxmox local-lvm status was unknown... the local-lvm had all my VM & CTs including most important HomeAssistant server...

I tried troubleshooting going thru few forums and google but nothing worked... I followed one of the guides which mentioned about changing the content type in the Prxmox hardware storage settings and rebooted the server and that is when I realized I had f**ed up, the server never came back online :( I lost all my VM and CTs and had no backups of these (which was my mistake, only positive note is my Homeassistant backup was available on GDrive)

Last 2 days I have spent hours with fresh installation of Proxmox and all the VMs & CTs

How could I have avoided this? what is the best way to backup Proxmox ?
Click to expand...
Not an expert, but Proxmox Backup Server is usually the recommendation I see for this. It is also fairly easy to setup and you can automate it to do it every X days.

In my case, I do it the manual/stupid way. I turn off the server, plug in a liveusb, boot into a linux distro that supports zfs out of the box like CachyOS, then I use dd to clone the drive. I do this once a week. The advantage of this is that if the server ever fails, I can get back running with a single dd command.
 
  • Like
Reactions: alpha5555
variablevector said:
Not an expert, but Proxmox Backup Server is usually the recommendation I see for this. It is also fairly easy to setup and you can automate it to do it every X days.

In my case, I do it the manual/stupid way. I turn off the server, plug in a liveusb, boot into a linux distro that supports zfs out of the box like CachyOS, then I use dd to clone the drive. I do this once a week. The advantage of this is that if the server ever fails, I can get back running with a single dd command.
Click to expand...

Hey can u let me know the process for manual backup? any guide on video please

DewlanceWebHosting said:
To my knowledge, your physical volume or logical volume is lost, but not deleted. You'll need to locate it and bring it back online. If you're using Logical Volume Management (LVM), recovery is bit easy.

Use logical volume scan command to get it back.
Click to expand...
oh man I wish I knew this earlier, I went with a fresh install again :(
 
I've had 50+ unexpected shutdowns over the years across both 7 and now 8 and this is something that's never happened, all my drives are SSDs.

To be clear, the local-lvm did not span on to the spinning drive? It was confined to the main ssd?

Were you able to access the web ui? Or log in to the root console directly on the node?
 
  • Like
Reactions: Party Monger
MarioBros said:
Hey can u let me know the process for manual backup? any guide on video please


oh man I wish I knew this earlier, I went with a fresh install again :(
Click to expand...
Boot into an liveusb that comes with zfs installed. Do "zpool export -a" to export the zfs pool first. Then use "sudo dd if=/dev/sda of=/path/to/backupdrive/zfsbackup.img status=progress" where sda is the drive where the zfs filesystem exists. I'm not sure how this would work if your zfs pool spans across multiple drives, since I just have the one drive. And be careful while using dd, if you get the inputs/outputs switched by accident, you might delete everything.
 
Last edited:
  • Like
Reactions: MarioBros
rsaeon said:
I've had 50+ unexpected shutdowns over the years across both 7 and now 8 and this is something that's never happened, all my drives are SSDs.

To be clear, the local-lvm did not span on to the spinning drive? It was confined to the main ssd?

Were you able to access the web ui? Or log in to the root console directly on the node?
Click to expand...
on my new server while testing this has happened about 5 times and same, no issue. The whole reason im testing is to see what to do when such issues strike.
Any suggestions are welcome. For work related tasks I've only figured out in machine backup to cloud or local storage on RPI4
 
I've had plenty of grub failures after unexpected shutdowns, but those are easy to recover from. Maybe that's what's happened here?

In those cases, Proxmox won't boot at all and you're left looking at a grub rescue prompt with seemingly no way forward.
 
Faraz said:
Can proxmox run on a 2600x without a gpu?
Click to expand...
You will need a gpu to install it, and if you ever need to troubleshoot something and require access to the server over a monitor. Otherwise you don't need one at all. Practically speaking, it is good to have a spare gpu handy or at least be able to swap one in from another pc.
 
  • Like
Reactions: Faraz
variablevector said:
You will need a gpu to install it, and if you ever need to troubleshoot something and require access to the server over a monitor. Otherwise you don't need one at all. Practically speaking, it is good to have a spare gpu handy or at least be able to swap one in from another pc.
Click to expand...
That actually works out. It will be hosted at my old workplace, and I don't want anyone interacting with it. Thanks!
 
  • Like
Reactions: variablevector
codwapeace said:
Where to buy them? Any idea?
Click to expand...

Amazon. 20K for unmanaged, 25k for managed.

https://www.amazon.in/Ethernet-Multi-Gigabit-Unmanaged-Fanless-MDIXSupport/dp/B097RD15N8/ - change options to choose which you want. I will be taking a SFP plus 10G version.
Faraz said:
That actually works out. It will be hosted at my old workplace, and I don't want anyone interacting with it. Thanks!
Click to expand...

In my ~15+ years of virtualization, that is a bad idea. Lock it down but not limit access by crippling it. You can get some old entry level GPU cheap and keep it.
 
Last edited:
  • Like
Reactions: alpha5555
@Faraz
> That actually works out. It will be hosted at my old workplace, and I don't want anyone interacting with it. Thanks!

How legal is this, that company can raise a serious legal case against this if something unexpected goes wrong ?
 
TEUser2K1 said:
@Faraz
> That actually works out. It will be hosted at my old workplace, and I don't want anyone interacting with it. Thanks!

How legal is this, that company can raise a serious legal case against this if something unexpected goes wrong ?
Click to expand...
Well, from a strictly legal standpoint, the workplace company may claim breach of privacy, stealing of property, unauthorized usage of property, breach of trust, misappropriation of property, etc at the top of my head. If they turn out to be hot headed, they may also claim that your usage led to damages in their operations, and claim compensation for the same (this you won't be able to defend well because whatever they may claim, you won't have any way to rebut). This is me only suggesting without going into details of the workplace or their business and being an advocate.
 
  • Like
Reactions: TEUser2K1
TEUser2K1 said:
@Faraz
> That actually works out. It will be hosted at my old workplace, and I don't want anyone interacting with it. Thanks!

How legal is this, that company can raise a serious legal case against this if something unexpected goes wrong ?
Click to expand...
It's my own device and I have the owners permission to host it. I just don't want his employees tinkering with my property. Chill :tearsofjoy:
AwAcS said:
Well, from a strictly legal standpoint, the workplace company may claim breach of privacy, stealing of property, unauthorized usage of property, breach of trust, misappropriation of property, etc at the top of my head. If they turn out to be hot headed, they may also claim that your usage led to damages in their operations, and claim compensation for the same (this you won't be able to defend well because whatever they may claim, you won't have any way to rebut). This is me only suggesting without going into details of the workplace or their business and being an advocate.
Click to expand...
Nothing illegal going on here. I built the company's network. I still on occasion provide support to them. I have their full permission to host my device on their network. I mostly use the machine for eve-NG, storing encrypted data and for wireguard.
vivek.krishnan said:
Amazon. 20K for unmanaged, 25k for managed.

https://www.amazon.in/Ethernet-Multi-Gigabit-Unmanaged-Fanless-MDIXSupport/dp/B097RD15N8/ - change options to choose which you want. I will be taking a SFP plus 10G version.


In my ~15+ years of virtualization, that is a bad idea. Lock it down but not limit access by crippling it. You can get some old entry level GPU cheap and keep it.
Click to expand...
Does it impact performance in anyway? I have a 1650 connected to it at the moment.
 
Last edited:
  • Like
Reactions: TEUser2K1
Faraz said:
It's my own device and I have the owners permission to host it. I just don't want his employees tinkering with my property. Chill :tearsofjoy:

Nothing illegal going on here. I built the company's network. I still on occasion provide support to them. I have their full permission to host my device on their network. I mostly use the machine for eve-NG, storing encrypted data and for wireguard.

Does it impact performance in anyway? I have a 1650 connected to it at the moment.
Click to expand...
Unless you're running something gpu accelerated, like transcoding or gaming, no. Based on the things you mentioned, you don't need a gpu at all after intial setup.
 
  • Like
Reactions: Faraz
I have a question, what is the *current* definition of a homelab? Yes it's hardware to run software but what are the specifics here? Also many people who work with computers run one but maybe others do too. So how would you define a homelab today?
 
Top Bottom