Just wanted to share with you my recent upgrade. I received a Mellanox ConnectX-7 VPI card from GrabnPay. Beast of a card with 400GbE/NDR speeds, and perfect for my PCIe Gen5 board. If anyone’s building their lab or upgrading, I highly recommend checking it out here: grabnpay.in. Support and delivery were top-notch.
Curious if anyone else is using this with an OSFP module?
The fuck? WTF are you self-hosting? This sounds like r/HomeDataCenter shit. Is this some self-promo thing with the links — never heard of this site before.
Visit my site!
nanibot.net
Cool. I run a homelab too! 2 x AMD EPYC 7601 and 256GBs of RAM.
But nowhere near 400GbE lol. Can you please explain your usecase?
@gowtham07, do you have access to the internet backbone?
I am curious about your use-case.
Though I would like to explore high-speed networking if my wallet allows 
, maybe someday.
Did you just get one, or multiple?
Because there isn’t much you can do with just 1 card.
Would love to know your setup, amd what you use it for.
Also, I see similar cards on ebay for much cheaper.
This thread got me hooked
I find it funny that the OP himself never answered the question posed in topic. I just hope its not a promotion for that site.
Well, considering he has a total read time of 1 minute since the migration, I’d wager yes.
For those of you actually looking for faster networking options, although its not 400G, 40G or 10G (or even 100G, relatively speaking) is pretty cheap now.
I have a couple of Mellanox CX3 cards for 40G, and Intel x710 cards for 10G.
This blog is a good read, and lists some of the options.
hey, do any of you know, where to get access points that enables the use 6Ghz band in India region, India or Import ??
Tell me more!
How must did they cost and where did you get them?
Do you have point to point connections or did you get a switch as well?
Are you running copper or fiber?
How much did the cables cost?
Almost all of it was from ebay.
The Mellanox 40G cards came as a pair, with included DAC cables.
It cost me 90USD (price+shipping) + 2.6k customs.
The 40G is p2p between my NAS and main hypervisor.
I got 3 Intel x710 cards for 11k including shipping and customs. They are connected to a Mikrotik CRS305 switch for my 10” rack setup.
They are connected to the switch using copper DAC cables, and I also have a few interconnects that use fiber.
The 10G DACs cost around 15USD/p.
The LC-LC OM3 cables cost ~10USD/p
The Intel 10G transceivers cost around 5USD/p
That is super cool. I’ve recently started to feel limited by the 10G link between my desktop and nas, it’s nice to know that 40G is within reach.
I got my Aquantia 10G cards from a system builder a few years ago, they were a few thousand each and are MSI branded.
Do you have a specific use-case for the high speed networking your TMM cluster?
The HUD is nice. Can you elaborate a bit more on how that was setup? Is it a spare tab repurposed?
The 10G cards are mainly used by ceph. The higher speed to the NAS also helps for other types of storage.
The HUD is just a 7” display that connects via display port to one of the machines.
I use one of those mini keyboard+touchpad combo things to control it.
In the pic, its displaying beszel. But I occasionally use it to display homeassistant/grafana/headlamp, or even put videos for my cat to watch 
