In Eric Brands' Traditional Chinese Medicine blog, he complains that Bensky's Materia Medica and other texts on herbalism state ginseng should not be taken with coffee or other stimulants but do not list any citations to back up this claim. He did extensive research and found no evidence in classical texts or modern databases to support the contraindication. He says that "primary sources, evidence-based decisions, and critical thinking" are essential to the future of TCM.
It is most likely that the reason for this caution is due to the popular conception that ginseng is itself a stimulant. However, ginseng is not a stimulant drug in terms of modern pharmacology; it has both stimulating and inhibitory constituents, which gives it an overall regulating and balancing effect on the body. The idea that ginseng=stimulant and thus should not be combined with other stimulants is based on a flawed and simplistic understanding of the actions of ginseng. Certainly this simplistic thinking is not the standard that professional practitioners are shooting for, and it damages our profession if we espouse opinions that cannot be backed up with either science or tradition."
Read the complete article:
http://www.bluepoppy.com/blog/blogs/blog1.php/2009/06/12/ginseng-coffee-herb-drug-interactions-an

This is a very interesting topic. Personally I agree that the idea of ginseng should not be taken with coffee is quite wrong. Just like the idea of eighteen incompatible herbs and nineteen herbs of mutual antagonism, now more and more authorities in Chinese medicine field have confirmed it is wrong.