It´s nothing new, people that are attracted to the way you look (especially if you have a certain picture that turns him or her on) and seem to base their entire motivation to be with you on that. It seems silly but it happens, more often than you think. A sexy outfit can get you a lot of attention, but is that attention for you or for what you´re wearing? If you can answer that question then it shouldn´t be that difficult to decide what you do with that type of attention, because if someone is just attracted to the way you look then that´s not someone worth spending time with.
Latex means sex? To some people it does, wearing latex is nice but some people have such a strong desire for latex that it makes them completely blind to anything else. Which might lead to an awkward situation when he or she wakes up the next day in bed with someone they´re not attracted to (without the latex). But it sucks if that is what it´s all about for people that are interested in you, they like what you wear but they don´t seem to really care for who you are. I guess these casual sex encounters could work for some time but I take it most people want more out of a relationship!
So it´s a bad thing to be attracted to the way someone looks? I´m not saying that, but don´t let that be the main reason why you´re attracted to someone. Don´t forget that the person wearing latex or whatever type of clothing seems to take your fancy is more than that outfit, so even if that is a huge turn on for you I would suggest not to focus too much on that. It´s nice to meet people that have more to say than how hot you look in that outfit, because that´s not a turn on for the other (well not for most people).
Of course it´s nice if you can share a fetish, for example you both love latex and if you both wear it then you can have amazing moments together. But remember not to make it all about latex or sex for that matter, there is more to a relationship than just sex! Sure it plays a part but if you base your relationship on just sex then you´re missing out on a lot of other things. But I guess we all have our own needs and ideas about that, let´s just hope you find someone that has the same way of looking at it as you!