You are absolutely correct. The issue is with the router's DNS only.
After trying a lot of troubleshooting steps, I manually added DNS inside IPv4 settings and all of the sites loaded fine. Also everything seems to be working fine on mobile phones, even desktop sites load fine on mobiles, but...