rise and fall of p-value: a lesson to be learnt.

Last year, after many years of p-value abuses, the american society of statistics in a revolutionary move published an instruction on how to use p-value. YOU HAVE TO read this before hand, if you ever gonna use p-value. But, here I want to pay attention to a more general mentality, which I believe is the caused of this misuse of p-value.

Since you may not have enough time to glance over the paper, let me just briefly summarize my understanding in few words, although this does not replace the whole manual. It basically says that the value of p-value for large p, does not have any significance by itself. Whenever, p is less than a particular threshold (say <0.05) then we can say that it is less likely your data with a true null hypothesis. Or more scientifically, you can strongly reject the null hypothesis, that is, there is an effect (null hypothesis is assuming that there is no effect). However, the size of effect is not determined by p-value. Additionally, you should always consider that the unusualness of your data.

In modern science, especially, during last 50 years numerical value became more important than ever. Previously, we had qualitative and quantitative understanding, now the former is losing the race to the quantitative universe. Crudely speaking, for me quality is not necessarily non-mathematical or numerical, but something that you can not specify it with a finite set of numbers. IQ, SAT score, GPA are just a few examples of many, which is a single number to specify Intelligence, or Academic preparedness, or academic valuation of someone (which of course, is a sham). In this particular universe, only (finite) number matters, and it usually take while to come up with a good MEASURE which turns multi-variate systems into a single (or a few) number. In particular, dealing with large set of events and data, is one of those areas that turning knowledge into number is usually hard. So the measure, such as p-value come to rescue!!

Now, all scientist in life sciences, environmental sciences, and psychology who does not want to spend enough time to study to understand statistics with underlying assumptions, just take the equation and leave the rest (it is always said: the devil is in the detail). Few years ago, there was a research paper titled: “Why Most Published Research Findings Are False”  which was really disappointing, later on there was very depressing while courageous statement in Nature article in 2014, by Steven Goodman (statistician@Stanford) stating that: “The wake-up call is that so many of our published findings are not true.”.  In the light of these findings, most likely we should rethink and reevaluate all those suggestions by FDA and other agencies, which are heavily based on those studies! The depth of disaster is so deep that nobody wants to even scratch the surface.

Anyway, I do suggest not only restricting the daily usage of p-value, but also leave the whole mentality that everything can be turned into a single number! Sometimes, it is just more than that. I don’t say we should not try to understand things quantitatively, in contrast, what I am saying is that sometimes quantifying with a single number is over simplification and underestimate the reality.

 

Why Do Many Reasonable People Doubt Science?

Some days ago, I read an article in the national geographic website, with the same title (here). This article first starts by stating that people, even scientists, ignore science since in accepting scientific facts we “cling” to our “naive beliefs”.  There are several examples, in the article which the author argues they are just scientific facts, but people have the hard time believing in them, simply because human nature is naive!

Recently, I came across several articles with the same type of logic, and discussions about the “naive beliefs”. I find the reasoning used in this article and similar ones ridiculous. In another post, I explained what is wrong with “scientific facts”. However, please let me explain in some more details why people doubt science. First of all, it happened to all of us when we started our day by looking at the weather forecast. However, once we looked outside, we saw something completely different! So you could obviously see there are many unexpected things that can happen which was not included in our scientific models. Everybody sees this shortage and understands that, particularly, when we are dealing with large multivariable systems.

Somewhere in the article says that “We’re asked to accept, for example, that it’s safe to eat GMO (genetically modified organisms)…because, the experts point out, there’s no evidence that it isn’t and no reason to believe that altering genes precisely in a lab is more dangerous than altering them wholesale through traditional breeding.” This statement is wrong at so many levels.  First of all, when we are talking about our health we tend to stay close to our tradition simply because we know that it has been working fine for many many years. Second, I have seen no clear data to rule out the correlation between these many new diseases and GMOs. For example, we don’t know at what time scale we should expect to see the correlation. It may take more than 10 or 50 years for something to affect us.  Or even worse, it is not clear for what type of disease we should look for. Finally, we don’t know what could be the effect of GMO on environment, the time scale for the environment could be much longer than 50 or 100 years. Just as an example, look at the history of DDT. 

People doubt science due to the failures of science throughout the years, not because they cling to their naive beliefs. They doubt science because of many things that can go wrong in one scientific research. As I emphasized in previous posts (e.g., here),  science is not for believing, but it is a process to falsify our understandings. I should say, although I agree that scientific process is the only trustworthy method to investigate each and every question, we have all the reasons to doubt “scientific facts”, particularly those sciences which rely heavily on statistical inferences.

Most scientific papers are probably wrong!

I remember talking to a friend of mine mentioning the fact that I am really skeptic about the results of the studies in the multi-variable fields with no underlying theories, such as life sciences, social sciences, and earth sciences. My argument is as following, there are many parameters to change, and since we have no theory to compare the experimental predictions with, we could easily get misled by the results. In fact, in most of these fields we can’t even answer the basic questions, such as how many parameters are there to estimate the right sample size!

Recently, I came across this article in newscientists which basically proves that many scientific results are probably wrong! What I like about physics, in general, is that we have theoretical guidelines which help us to better understand the experiments. Of course, our theories are based on many approximations and assumptions, however, it is still unreasonably accurate. In fact, the main difference between theoretical physics and applied mathematics is how to lay down different approximations. This is one of the main reasons that, I believe, we have a steady progress in physics, and sorts of a random walk in other fields. By the random walk, I do not mean that we are not making progress, of course, we are. However, there is no sense of direction in what we are doing, at least by an outsider like me!

How natural is our mathematical world?

Looking at the geometry, we instantly realize that we are used to assume everything is composed of straight lines. Or the mathematical analysis is full of linear assumptions here and there. However, in reality finding a straight line or a linearly interacting system is almost out of reach. This difference may seem  abstract at the first sight. However, the effect of our mathematical assumption is perfectly vivid in manmade products, such that, by looking at a particular object we could easily distinguish manmade from natural. This deviation is clear since all of manmade products are engineered based on the mathematics that does not share the same symmetry with the nature around us.

This is also true, for our modeling. All of our models are linear interaction model, which absolutely is far from what happen in the real life. As it is nicely mentioned by Wigner, we are so lucky that our models can roughly explain what happen in the real life. Additionally, This type of thinking is not limited to the science or the engineering, but it is also extended to the social sciences such as philosophy. We should understand that sometimes what we call perfect or ideal, is not necessarly the ideal of nature around us. We should also note that what we learn in the school is not necessarily what happen in the world.

I finish this post by asking, can there be a different mathematics, more inline with the nature, where we can relax a straight line or linearity assumptions?

 

Possibilian: from fundamentalism to atheism

Thanks to Ali jan for sharing this nice ted talk with me. It is an inspiring talk and this is one of those subjects that I wrote something about, e.g. here or here. To start, in my opinion, he touched two interesting points, first, he clarified that our belief system is a byproduct of our cultural system, and second, he gave a good description of “what really the science is”.
His main message was: listen to all ideas, filter them by the toolbox of science, and then consider the other possibilities. Finally, he emphasized on staying away from making a strict choice when there is a little known and a lot unknown. That’s very good and nice, however, there are several drawbacks here:
1) We live for a rather short time and during this time we are forced to make a decision about some main fundamental questions  (no matter how much we understand), from which almost all religions are coming from, such as where are we coming from, where are we going, and why are we here! How we answer them would change our behaviour in this very world during our short life span. Willingly or unwillingly we are answering to those questions and behaving accordingly.
2) He also nicely mentioned about the role of science as a filter, however, he did not emphasize that science by itself is an ever-changing land of possibilities which open-minded-ly contradict itself from time to time. It means that the toolbox/filter by which we falsify other theories sometimes is changing so drastically that some of the impossibles become possibles and vice versa. For example, in life science, before the discovery of DNA, it was so absurd (scientifically) to talk about a fundamental quality which is transferred from parents to children (called fetrat in Islam) which would determine the most of child character! However, nowadays, it is believed that almost everything is inherited from parents, and education, societies, etc can do a little to change that! So, basically, science is making random walks in the phase-space of possibilities.
3) The way the decisions are made in his “possibilian” approach is biased towards science, so I would rather call this new religion as scientism. Since, the righteousness of possibilities are measured with (modern) science and not the other means. Of course, there exist other possible toolbox/filters. For example, to treat a certain disease you may choose to use modern science or ancient medicine. So, possibilian is rather a larger pool than what is explained in the talk!

On the nature of time?!

There is an interesting podcast on the recent book by Lee Smolin, a well known theoretical physicist at Perimeter Institute.  http://physicsworld.com/cws/article/multimedia/2013/sep/23/lee-smolin-on-the-nature-of-time

here I briefly explain the issue with time.  Well, classical physics suggests a particular direction for time, i.e, everyday is distinguishable from yesterday. If you start a movie from one side to the other, it is distinguishable from playing it backward. However, the governing laws of nature are invariant in time. Additionally, Einstein theory of relativity, merged the nature of space and time together. So, spatial dimension the same as time dimension (we know that space does not have an intrinsic direction as time has, right!). Anyway, this distinction between (current) governing laws of physics and our natural world (at large scale) inspired a lot of discussions to better understand the nature of time. one of these ideas put forth by Lee Smolin which you can briefly hear in this podcast. He usually writes a concise and readable books and I am pretty sure his new book will be the same (in case you are interested. )

Criteria for a good introductory book.

Finding a good and reliable introductory book is usually hard, since we don’t know much about the subject. However, here I will present some criteria independent of the subject which could be useful, specially for math, physics, and computer science. At the end, I provide some examples. (Here, by a good book I mean something that once you are done, you would learn about the subject, and you would know what you don’t know).

The first aspect of a good book is that, it should be consistent in definitions. I believe that every chapter and every subject should be started with clear and concise definitions. Sometimes, we use terms in our everyday language which does seem to have a global meaning while it does not.  For example, the word “morality” does not have the same definition everywhere.

The second criteria is that in every chapter there should be a clear picture of what is going on, starting with the definitions and ending with what is desired.  Meanwhile giving the motivation is very helpful, however, what is a “must” is that it should contain all the details, even if something is skipped it should be mentioned that this particular part is omitted for given reason(s). I have seen in many books that some approximations are made without any justification why the approximation is valid, or without providing the range of validity of the given equation or subject.

The third criteria of a good introductory book is that,  it should contain a list of meaningful and relevant (preferably solved) exercises.  Just being an introductory does not mean it should have meaningless and useless exercises or examples. Especially, it is preferred that examples through each chapter would be continuos. Most of introductory books lack this particular criteria, where they use “simple-minded” examples.

One of the most important subjects in modern physics, is quantum mechanics. The “Quantum Mechanics” by Cohen-Tannoujdi is one of the best exemplary of instant of a good introductory book. For learning Python, “Python Programming” by John Zelle is rather a good book, however, it lacks the clear definitions. Usually, the O’Reilly is a good source of useful introductory and advanced books in computer science. In future posts, I will post a list of useful books that I came across during my ten years of university studies.

Physicist Kenneth Wilson dies at 77

I recently found out that one of my favourite scientist, K. Wilson, died last week. You can read about him here.

His work on phase transition and renormalization group was one of beautiful theories in theoretical physics, which inspired me learning physics more and more. The idea of renormalization and its development can be found in Wilson’s nobel lecture: http://www.nobelprize.org/nobel_prizes/physics/laureates/1982/wilson-lecture.pdf .

Baker-baker paradox!

It has been a while that I am thinking how our memory works. I have some ideas which is mostly based on my personal taking from my own experiences. However, I was not able scientifically explore this topic more, unfortunately, despite the fact that my wife gave me a book as a birthday present about the brain last year.

It possibly happened to all of us that we can remember some words, names, or memories much easier/faster than the others. Why is that? maybe if we could understood the procedure behind this, we would be able to memorize things better. My understanding is that, we tend to forgot things which have less connections to the other part of our memories. On contrary, we tend to remember things easier which has more nodes or connection in the brain.

Apparently, this is something that has been known for a while in psychology and social sciences under baker-Baker paradox. Roughly speaking, if we would be able to manageably associate each new peace of information in the present network of our memories, it is more likely to be able to remember it later.

Recently, I came across this interesting TED talk on how to train our brain to memorize better, which is practically based on the above simple concept. http://www.ted.com/talks/joshua_foer_feats_of_memory_anyone_can_do.html

I should call your attention to the fact that the above tricks work best for memorizing stuff over a short period of time, however, they do not form a (long term) memory.  Hence, another relevant question is that how long does it take for a memory to slip away from our brain? Additionally, how we can improve our long term memory? This will be a subject of another post in near future.

PS: Based on comments that I received from friends, I would start to write smaller post for convenience of viewers. Thank you all for your comments.

Computer vs. Brain.

When I was growing up there was computer boom in Iran. So, almost every students in the major cities was aware of computer and its abilities. Or at least they had some advanced calculators to numerically or symbolically calculate equations and integrals.

It was since the high school where I started to hear this nagging noise complaining that if we can solve these equations or integrations by a computer why we should do it? Honestly, I was fan of solving integrals, and equations analytically and making relevant approximations myself. I never bought and used calculator for calculations to the end of my university degree, at least for tractable problems, although I was constantly using computer to double check my solutions or to create graphs and figures. However, I could not give a clear answer to those classmates who were constantly complaining about that and refusing to learn advanced calculus, since it seemed unnecessary.

After some years, now I believe that I have an answer to this complaint. Setting the beauty of finding solution and [temporary] feeling of achievements aside, solving some challenging but tractable equations, or integrations analytically, first it helps to appreciate  how the mathematics practically works. Second, it improves our brains imagination by closely observing the dynamics between similarity and differences which leads to the final solution. By gaining this ability we would be able to also observe similarity and differences in facing real world problems.

In my opinion [and many others], our general world view is directly related on our perception of happiness and life. How we live depends on how we think we should live. All these questions would obtain different solutions if our brains are trained to see many different possibilities by looking at a certain problem.

In conclusion, I believe that, solving the tractable problems using our brain not only helps to be innovative in facing new mathematical problems in our future careers, but also it would have life changing consequences, since we can see life with higher resolutions. Relying on computer too much, particularly, for kids would lead to replace our dynamic and innovative brain which is capable of creating art, music, math, etc, to some body part which only controls our involuntary moves.

For me the late Richard Feynman was one of those people whom solving simple yet chanllenging mathematics gave him different views in life. Some of his views about the future of science and technology is started to come to reality just recently. I encourage all of you to the following BBC interview with Feynman: http://www.youtube.com/watch?v=EKWGGDXe5MA