I used to fight terrible insomnia like you're describing, but I did figure out what works for me. My naturopath explained to me that my adrenal glands have been overactive due to having no real energy to work with (chronic fatigue.) My adrenal glands hadn't been able to calm down enough to let me sleep. This created or increased my anxiety, which was yet another thing keeping me awake. I had tried to calm my anxiety, but since the anxiety was only a symptom and not the root cause, nothing was working.
Now the trick is to actually calm the adrenal glands, which takes a lot of effort. For me, it involves calming all of my health issues so I can increase energy and let the adrenals rest. I'm not there yet, but I AM sleeping - and often sleeping well! My naturopath has me taking a lot to fall asleep with: 3 each of two different herb based pills, melatonin, and 2 Kavinace. 9 pills total, at minimum. I can take two more of both of the herbal pills if I need extra help. The point is that the brain needs to be overpowered by the pills so that it can't win in a fight to stay awake. Melatonin usually isn't enough for people with insomnia because it isn't strong enough. During the day, I take all sorts of vitamins, including a couple B Complexes. I also am required to do 45 minutes of cardio a day (which I can't do, I end up doing about 15 minutes, lol,) in order to not only wear out my body so it rest, but also for healing purposes. The won't heal if it isn't being used. I can't have any starch or sugar (no fruit or honey, even,) and my caffeine intake must stay as minimal as I can. I'm allowed to indulge with a very small cup of coffee or 85% dark chocolate every once in a while.
But I do as much as I can for sleep. I take hot relaxing baths before bed. My favorite is a detox bath: ground ginger and kelp mixed with Epsom salt and raw organic apple cider vinegar. I'll add lavender essential oil, as it is a very calming scent. You can keep it simple and just do Epsom salt with a calming essential oil too, but I like the benefits of the other ingredients.
I recently bought an aromatherapy diffuser/ humidifier that I keep next to my bed. I need the humidity anyway, but the aromatherapy helps A LOT. I add "tranquility" or "sleep aid" oil blends and let it run all night.
How much time do you spend in front of a screen at night, including ebooks? The blue light from screens actually suppresses melatonin production and stimulates you. Have you heard of "computer glasses?" They have special lenses that filter out blue light. I wear them whenever I look at any screen after 5 pm, and they're helping a lot. My pair was about $20, but there is a high quality brand that sells for around $80 if you need better quality.
I need to address my pain before going to sleep, because pain is stimulating. All my sleeping pills help calm pain because they're nerve suppressants, and the baths help a lot because they draw out toxins that cause pain and replenish my body with essential nutrients, but when I need more help, I use an acupressure mat. It's a soft mat with hundreds of non-sharp spikes that you lay on, and it should come with a neck pillow with the spikes too. I lay on it for 10-20 minutes before bed and it helps decrease my back pain substantially. It works by greatly improving circulation and helping the chi to flow in your body (which is what acupuncture is for.) I bought my acupressure mat for about $30.
Another thing that helps is drinking organic (but more importantly, non-bleached) chamomile with lavender tea an hour or two before bed. I drink it early enough that I can pee before going to sleep, ha! It is very calming, and the heat is relaxing anyway. Also, don't go to bed hungry. A little bit of protein in the evening can help you sleep, because it's the slowest nutrient to break down, so it doesn't give you quick energy. I will eat a few spoonfuls of raw almond butter within an hour of going to bed.
I hope all this helps you! I'll get which herbs are in my pills in another post shortly.