When sensitive to cow's milk, the most conservative approach is to avoid all forms of cow's milk, including ghee.
When sensitive to cow's milk, can I eat ghee?
Adair
