I have set up my Ally with IoT LTSC, drivers and the store, nothing else. It is significantly better for gaming as well if you don’t use the device for regular laptop/desktop work.
It is absolutely a bad idea now to go with any new Windows 11 releases or feature drops.