I don't believe it is contested at all in psychology that 'thinking' (or at least answering - the behavioral outcome of thinking) can be assessed. If this cannot be measured, then it seems the tools to measure it (statistics) would be false - as well as its foundations - validity (content and construct validity), reliability (inter-rater and test-retest reliability) and probability itself.
Why would probability be any less valid when applied to psychology than to physics? Peter Jay
-------
There are a lot of philosophy of science issues here that are being glossed.
Statistical probability recasted as statistical mechanics works at a certain level of physics because the point masses are assumed to have no intentions of their own. The same model of zero intention or will breaks down when it is applied to human systems. However individual wills and intentions are constructed, they exist and are an integrated part of the system. You can't assume the random motion model works. Particles are subject to forces and given direction and magnitude. But there seems to me to be a marked distinction between mathematically defined forces and a highly fluid system of wills, intentions, plans, goals, and the other characteristics of people and their societies. Then there is the problem of time. The laws of forces and particles are supposed to be based on measured and known constants and hold forever. Niether of these are accurate for people and human systems.
These assumptions about human systems don't necessarily imply that statistical methods do not have some application to understanding how society works, and hence are valid in some applications. What's required is a kind of art, craft, critical judgement of when to use the application.
The underlying problem is that psychology has to develop an explanatory theory of mind, and as far as I know they haven't. This seems to me to be the same problem with cognitive sciences.
What is put forth above, is what I think is called the behavioral school model, and so that's why I thought it was contested territory. We used to call the UCB Educational Psychology Dept the rat-theory crowd back in the day.
Then there was my own experience as a student counselor. I knew many of the students I was serving would have mediocre grades and their high school scores would be low, if they did take any tests. I went through their transcripts and interviewed them. The student service program specifically got control over our student's entrance requirements, which we automatically waived, and substituted our own. What was I looking for? People who wanted to go to school and had some motivation to do so. Then I'd ask them the simple and non-trick questions of what they wanted to learn, what they thought their major might be, what their interests where and so forth.
We made some mistakes, but these were usually politically motivated by the conviction that we could get a dead body through UCB, if we worked at it. We did almost the next thing, getting people through who didn't belong in college.
I found the key factor was time, slowing down the process, and lightening the load, and academic and social service support, peer counselling, so students could adapt to the education system and the pace at Berkeley. Nobody turned out to be Einstein, but there are some good teachers, a pretty good philosopher of Heidegger, an economist, an architect, a couple of city planners, a state dept official, a state level director of vocational rehab, a PhD in biophysics, and a lot of lawyers. All these people did not qualify for entrance, and or couldn't have gotten through without our kind of support so I think our system based on the judgement of former students doing one on one interviews, worked pretty well.
CG