I find all intrusions into freedom of speech a little worrying, as I think it might be better to know who the hate filled bigots are, and it’s easier to challenge those ideas in the open. And like you, I worry how far such legislation might go, and don’t believe any ideas should ever be ringfenced from criticism.
FWIW religions and the religious have far more to fear from other religions and the adherents of other religions, than from secularism or atheism per se.