You are right in that there are a lot of variables involved from one setup to another.
Having said that, there are enough and more people globally and plenty people locally (with an Indian accent) who rely on voice commands to manage a very large (50-100) set of nodes with a high degree of accuracy.
Speaking for myself, close to 100 end-points with a degree of accuracy thats good enough for say the house help to ask Siri (homepod) to switch off the bedroom fan on a daily basis when she comes around for cleaning.
Between the automation rules and voice commands, it’s been a very long time (years in fact) since any of us used physcial actuations for home control.
If there is one (make it two) thing i would say, the most important thing to do when you take the plunge ot go this route is to ensure that a) your home network is super stable/reliable and b) your automation design/plan is well thought through
