You could combine both the plans
make a total of 4 VLANs - 1 each for WAN1, WAN2, LAN1, LAN2 on the smart switch, and keep 1 port for trunking
setup the same properly in the pi4 with the lone ethernet port to trunk with all interfaces on different VLANs on the smart switch using something like...