It is clear that the minds of Inuit work somewhat differently from those of English-speakers, or they would speak English; language is a mental production. If, living outdoors in an artic climate, it would be strange if they did not have a great many words for snow, just as young American males have a great many words for automobiles; people can be expected to create large conceptual and vocabulary systems for things they are familiar with and deal with every day. I don't see anything odd in the 400-words-for-snow theory, whether it's correct or not.
However, that doesn't reflect much on the Sapir-Whorf hypothesis, at least as Whorf expressed it. Whorf explicitly stated that he thought syntax, not "lexations" (dictionary entries) might be expected to influence thought, since syntax is much more restrictive than vocabulary. For instance, in most Indo-European languages, the speaker is compelled to assign activities or states, as expressed in verbs, to a particular time domain (tense). Thus, speaking English, one must choose from "I am running, I run, I ran," etc., placing the act in a particular time relationship with the present moment. This assignment is _required_ in English, but not in some other languages (e.g. Chinese). On the other hand, we assign a time dimension to nouns only sparingly and with some difficulty (_ex-president_, _bride-to-be_). One can see how this situation might lead to a difference in the way the referents were thought of; those assigned to nouns might be thought of as eternal and unchanging, whereas those assigned to verbs would seem, maybe, fluid and uncertain -- yet the assigment is often pretty arbitrary. "White" is a an adjective in English, but a sort of verb in Japanese. It could be a noun -- we could have _the_white_, if we hung out with Plato _to_leukon_, an eternal thing lending its whiteness-quality to mere worldly objects, something far different from a transitory sensation of the appearance of the snow on Mt. Fujiyama.
I don't know that there's any objective evidence for Whorf's hypothesis, but it's certainly not absurd on its face, and it does not concern how many words the Inuit have for snow.
In any case, I doubt if the language one speaks would affect one's ability to do mathematics very much, since mathematics does not work with way any natural language does, and it often deals with non-linguistic intuitions which are cast into symbolic form only with considerable difficulty.