I like Wittgenstein's solution to this: it's the beetle in the box.
Suppose everyone had a box with something in it: we call it a "beetle". No one can look into anyone else's box, and everyone says he knows what a beetle is only by looking at his beetle. --Here it would be quite possible for everyone to have something different in his box. One might even imagine such a thing constantly changing. --But suppose the word "beetle" had a use in these people's language? --If so it would not be used as the name of a thing. The thing in the box has no place in the language-game at all; not even as a something: for the box might even be empty. --No, one can 'divide through' by the thing in the box; it cancels out, whatever it is. That is to say: if we construe the grammar of the expression of sensation on the model of 'object and designation' the object drops out of consideration as irrelevant.
Phil Inv, section 293.
Miles