Sure, our pattern recognition machinery is wonderful at picking hypotheses that are more likely to be correct - and we have very little idea how that process works. What was going on in Einstein's mind what he dreamed up General Relativity? Who knows.
But when it came time to verify his work, the verification process itself could not be subject to "the gut", intuition or instinct. This process must be as objective and as free from human bias as we can make it. This is what I meant when I said the scientific method strives to remove the gut from the equation. I was probably not specific enough.
The ultimate arbiter of hypothesis is reality. There either is, or isn't a snake in that part of the jungle. It might be wise to avoid all of the "evil" spots, just in case, but that doesn't mean that all of them are in fact dangerous.
Similarly with more positive observations based on spurious correlations - such as folk medicine. Certainly not all folk medicines and practices are effective, but based on long experience few appear to be actually harmful, so it probably can't hurt to try them - especially if you believe they will work (placebo). And if that's all your arguments about alternative medicine amount to - that's well and good. Let's please just confine ourself to the realm of placebo effect and possibly real (and yet unverified) benefit. Just because it works for you doesn't mean it will work for someone else, and neither does it speak to the efficacy of standard medical treatments for the same ailments.
The scientific method amounts to either: noticing that something happens and checking to see if it happens every time or theorizing that something should happen then checking to see if it happens every time.
As any reasonably perceptive person goes through their life they will notice that sometimes x happens around when y happens. If you attempt to verify that it happens regularly and predictably you are a scientist and basing your beliefs in science - even if there is no supporting papers
It is true there is some room for personal bias to cloud the equation - and that is also true for the scientist in the lab. A published peer reviewed paper is a nice check to remove some of the bias but that doesn't mean the layperson should deny their own scientific abilities.
"As any reasonably perceptive person goes through their life they will notice that sometimes x happens around when y happens. If you attempt to verify that it happens regularly and predictably you are a scientist and basing your beliefs in science - even if there is no supporting papers"
No, not really. Observing your own personal experience is highly subject to personal biases, and perhaps more importantly, further subject to interpretting random outcomes as significant - you will have a very difficult time producing significant results from a sample size of one. Even if something happens to you five times in a row, the probability that this represents a random outcome is still quite high.
We, as individuals, also tend to forget negative results. How many times did we take zinc and still get our spouse's cold?
So, unless you are extremely rigorous in recording your observations, you really aren't conducting science in any meaningful sense. And even if you are, you probably will never be able to collect enough observations to be able to claim that your results aren't due to chance.
Here is a great example of where personal observation and testing can lead the scientists. I have other examples as well, although I think one will do.
Shanaya is asthmatic. When I met her 15 years ago she was having asthma attacks daily or even multiple times per day. She has had asthma since she was two. Asthma kind of freaks me out because when you treat it with the pharmaceutical it actually causes long term damage to your breathing tube. It also sucks to hear your wife wheezing, or worse yet to not hear it but know that she is having an attack but trying to hide it so as to seem normal. The goal of this life experiment would be to figure out what is causing the asthma and eliminate it. Now she has an asthma attack less than 12 times per year.
I am an observer. I observe more than I talk, I learn through observing and so I just naturally started to notice asthma being caused by certain smells. Diesel exhaust was one of them - every time we were driving behind a diesel truck with that strong diesel smell she would start wheezing. After a while I realized I could switch the air intake to recycle and roll up the windows to avoid this trigger.
Next we started noticing second hand smoke was a big trigger - anytime someone walking in front of us was smoking asthma would be right there with us. This is much harder to avoid but was good to know and be aware of. In close quarters this is even a trigger if the smell lingers on the clothing of smokers or is on their furniture.
We were under the belief that that cat and dog dander were major triggers. We noticed though that when we were around cats a lot she didn't actually get asthma. So maybe it is a trigger but a mild one. We even lived in a house with a cat for a month with no major attacks.
I think I heard on the radio once that lavender was a trigger so I asked her to smell some once. Yikes, instant reaction and a bad one. This is confirmed repeatedly because lavender is everyone. It's in dish soap, laundry soap, perfume, air fresheners. One time a friend left some unmarked lavender laundry soap in our house and I used it twice to was diapers. Shanaya was allergic to the whole house for a week. I re-washed everything twice and got the carpets out of the house and she was fine again.
Here we are today and Shanaya just had a mild attack. She has been having sensitive lungs for a couple of weeks and we need to figure out what the trigger might be. Some thing have changed recently and there are some new expected triggers around. It's spring, she has had a cold recently, there are cows in the back yard, I was playing with the outdoor dog today and we currently have guests staying with us that live with a smoker. It could be any one of those things, a combination of them, or something completely different. My job is now to figure out which it is.
The process for determining if something is a mild trigger goes something like this: remove the suspected object for a while and see if the asthma goes away. If it does we re-introduce the object and watch for problems. We can do this a few times to verify that it is that object and not some other random thing. If we remove an object and there is still a problem it doesn't necessarily mean that it isn't causing the problem! There could be multiple triggers. So we need to keep removing newly introduced items until the asthma is clear, then add them back in to see what it was.
If I had notes from years past I would know if spring was a trigger or not. I suspect it might be but I have never really tested for it and due to the nature of spring is incredibly hard to test for with the method I use. Notes would be great for this.
The other items I've identified and eliminated didn't need notes, scientists, white coats or labs. Applying some of the principles of the scientific process is all that was required to determine, with enough accuracy, what the triggers are and eliminate them. The scientists I'm sure have listed all of the items I know about as possibly triggering asthma and that certainly helped this process along, especially with lavender, but they have also listed many things that are a trigger for some people but not for Shanaya.
How did the scientists get their list of asthma triggers? I expect they asked a bunch of asthmatic people and their doctors what triggered their asthma. It's a different trigger for everyone and in this case a sample size of one is perfectly appropriate and very effective.
Oh but I can claim that my results work for me and really when talking about what works for me that is pretty important!