My ex has won parental responsibility but has now failed a drug test and wants more access. Courts arnt helpin?

My ex has made numerous allegations about me in court, such as being a alcholic and a drug addict. My child had to be examined by a doctor as he made allegations to caffcass. I took a drug test which obviously I passed. My ex now wants more access despite failing a drugs test. He sees her every two weeks, but plays mind games. I dont feel supported by caffcass. Due in court again soon any advice please,