There was a programme recently on British radio about the work of the German philosopher Wittgenstein who studied language and the way in which we use it. One idea that he put forward was that there can be no thought without words. The thought doesnt come first and then get translated into words; rather the words come first and can then be placed together to become thoughts.
I have reservations about the exclusivity of language in this context. It would be possible to think musically for example, but only if one had first experienced sound and tones. A gymnast develops a skill through 'thinking' with his body, but only by gradually building up the knowledge through actually performing it.
It does occur to me that this might well be the same with sexuality too - so that first comes the experience and specific sexual attraction develops out from that. By experience I do not mean sexual abuse necessarily or indeed sexual activity of any kind but other people and our experience of them and the context in which we relate to them. (ie pleasant or painful connotations).
People that we meet when we are children gradually mould the way in which we see others and, necessarily, ourselves as well. It also therefore figures that people that we DON'T meet would have an effect also. If I never get to know a woman to whom I am attracted - or if my relationship with my mother is one-sided and I feel dominated and threatened - then that may well colour all my relationships with women. In other words, when I meet a woman later, I will not come with an open mind but with a whole set of previous experiences which will colour my view of her (and, following that, her view of me which may well be just as significant.)
If a child falls into a swimming pool and nearly drowns and then doesnt go near a pool for too long afterwards, his fear of water may well become overwhelming and very hard to alter later. Presumably, albeit the other way round, this is how fetishes develop: if a child has a really pleasurable experience he will seek to repeat that and it may mean that he comes to associate things linked to that pleasurable experience with the pleasure itself - in a sort of iconic or symbolic way.
If a child has a sexual experience very early on, the degree to which that experience is either pleasurable or painful will surely colour his view of similar situations later and he will either seek them out because it was pleasurable or run in the other direction if it wasn't.
My point being that the things that happen to us change us in ways we probably cannot really control or understand.
Having said all that, I can see how it would be very hard to prove this. I'm only going on personal experience.