Linux Your experience with BTRFS?

If you use any of the OCI compliant runtimes (docker, cri-o, moby etc.) be aware that there's an issue where btrfs subvolumes are not removed after a container is destroyed.

I wrote about this issue last year on my blog : https://nanibot.net/posts/docker-and-btrfs-enemies/

Relevant github issues:
github.com

Docker gradually exhausts disk space on BTRFS · Issue #27653 · moby/moby

Description Steps to reproduce the issue: Install docker on a btrfs system Use docker intensively for a while, and make sure to remove & recreate containers, rebuild with no cache, restart the syst...
github.com github.com
github.com

docker does not remove btrfs subvolumes when destroying container · Issue #9939 · moby/moby

I receive the following error when deleting a container which created a btrfs subvolume (as happens when you run docker in docker). # docker run --rm fedora:20 sh -c 'yum -y -q install btrfs-progs ...
github.com github.com
booo said:
Read that article, seems to me that he made is test to validate his confirmation bias.

Lets see, when you compare RAM and swap; disk access is (say roughly) 500 times slower. so a cache miss and swap-in would be 500 times lower and even swapping out an unused page would create an io which is very very slow.

in older days when you had 640kb RAM then having a swap which is roughly twice the size of RAM would give you an illusion of 3times the memory. but but a very slow memory. and when you wanted huge chunks of data then computer will give you the same error message as when it didn't have the swap. but in today's age, we have boat loads of ram and ram is relatively cheap. so just add more RAM which is way way faster than swap disk.

Now, coming to ramdisk based swap; swap in and swap outs will only add one more memory transfer where you move pages from here to there in the memory. how is that supposed to improve performance? I mean swapping is creating unnecessary movement of pages in ram. Actually if you move pages across numa nodes, it will be much more costlier operation than just copying memory in the same numa node. I mean like let the page live where it is instead of moving it in the name of swapping and add more dma calls...

Personally anecdote time... I have done extensive io tests on enterprise servers for a very long time like saturating network links etc in SAN environments. never used swap on my servers/virtual machines. I dont think swap is a good thing in today's world where you could easily slap a 32G extra ram. if an application using more memory then probably it is a badly written application.




Anyone looked into xfs? its a fairly advanced filesystem too...
Click to expand...
+1, using swap on servers is a no-no. When your program runs out of memory it should always be preferred to let it be killed by the OOM killer instead of relying on swap. In addition to what booo mentioned (about disk access being slow), note that because of this the program might take longer to fail (typically, you would want it to fail-fast instead). Also note that fedora (not sure about other distros) moved to swap on zram since Fedora 33 I think. (https://fedoraproject.org/wiki/Changes/SwapOnZRAM)

About XFS: I run OpenShift/okd clusters for fun. I moved to ext4 a few weeks ago. Ever since, I've noticed that my VMs are struggling with disk IO (which did not happen on XFS). Not jumping to conclusions here but I would like to move back to XFS so that I can rule out any possibility of my hardware dying.
 
Last edited:
Back
Top