As a similar "Boy Scout" of sorts, the fear is/was real. I didn’t experiment with so much as nicotine or alcohol until I’d tried “stuff” with the supervision of an experienced "sitter"; I ended up having some of the best times of my life in the safety and context of home and friendships. Combined with my own life experiences with drug abuse and addictions, I was able to build a healthy relationship with those substances that didn't result in dependency or abuse.
In the time since, my views have changed dramatically on these substances, and I'd like to try more of them. However, my personal moral compass prevents me from using substances outside of a legally permissible setting, at least at present - and that's something I'm fine with.
Ultimately, the taboo side of things is something the individual has to grapple with on their own. I can only commiserate with your frustrations, not help overcome them unfortunately. My only other advice would be to use any substance only to amplify good vibes, never to cope with bad ones.
If all you do is chase a lost feeling, you're missing out on what's in front of you now.