New clearomizers often have a primer on the wick (properly called sizing on fibers) that can taste just nasty. Depending on the maker of the wick - or when its made etc, the primer can be quite heavy and hard to get rid of. If you find your new toys still have a bad taste - soak the heads in alcohol, or my fave - denture cleaner tablets, for a few hours. Rinse in hot water. That will get rid of all the icky primer taste.
If I have encountered the primer problem I also get rid of the juice that was in the tank with the head - it never tastes right to me if I try to save it. I just clean the heads and start with fresh new juice.
Denture cleaning tablets clean up the silicon caps too and get rid of the smell too.
I have no problem ditching bad juice. I'll try soaking the plastic parts in alcohol. If that does not help I'll go get denture tablets. It's the little silicon cap that smells so bad and could be from the primer-I did wash before using the first time. I don't remember the smell till after I used juice in it the first time.