How much ashwagandha? Need to take between 2 gram to 5 gram for to be effective.
You can also try Inositol (aka Vitamin B8) (500mg to 1000mg) before bed. It's natural substance found in a lot of things. It gives one of the best dreams. You'll see a full blown hollywood movie in your dreams. It'll make you crave to go to bed early. You can also add some magnesium (more important mineral than calcium or iron). All these including calcium you are already having with milk can be a best supplement stack for sleep. If you are still unsatisfied then add some melatonin (sleep hormone) too.
Is that like sleep apnea?