Many people don't consider scratch tests reliable in the diagnosis of food allergy, for two sorts of reasons. One reason is that some people who exhibit apparent allergy symptoms in response to ingested foods are non-reactive to the same foods in scratch tests. Another point is that your digestive system severely transforms the food that passes through it. Presenting more-or-less whole food to your broken skin is not the same as presenting more-or-less digested food to your gut.
Immune reactions that aren't IgE-mediated are sometimes called sensitivities. If you react to certain foods but don't have positive laboratory tests, many allergists will say that you're sensitive, rather than allergic. Other allergists either define allergy symptomatically or distrust the available laboratory tests, and they would still say that you're allergic.
Intolerance is yet another term for food problems, often used to describe reactions which don't involve the immune system at all. Lactose-intolerant people, for example, lack the enzyme needed to digest lactose ("milk sugar"). This is a completely different problem from being allergic to casein (a protein found in milk), but both problems mean that milk is bad news. Intolerances often cause intestinal distress of some sort.
Lay-people tend to label any unusual or acquired reaction to any foreign substance as an allergy. The distance between this common understanding and the variously restricted definitions of allergy among professionals leads to a lot of misunderstanding and hard feelings.
The International Food Information Council has their own pages on Food Allergy, with a number of sub-topics. The On-line Allergy Center has their own page on Food Allergies.