Of course it makes a difference. I can fantasize that my lover has a huge dick independently of how big it is in reality. Fantasy hurts no one.
"My own wife told me of a past lover who she was very interested in until she went to bed with her. She said that she was very embarrased but she hardly felt a thing. Is she wrong?"
Oooh Thomas, I think you got your pronouns mushed. You meant "with him" right? Your wife needs to do some Kegels (Exercising muscles around the vagina so as to allow the vagina to grip/feel better.)
Joanna