Hahaha....I thought people are referring to the router itself.
Most of my devices are products from the Indian market. I have a Amazon Fire tab which could be categorised as a USA device, but the issue isn't limited to that.
I have tried resetting once, didn't help. Next I tried upgrading the firmware. Haven't really seen the issue since then. But I'll have to wait until the weekend (that's when all the devices are at home at the same time) to find out if it has actually solved the problem.
Thanks for the tip. Now I know I'm not the first one to face this issue.