As per the scriptures the three "Ashrams" or stages in life were originally intended for three grades of intelligence

As per the scriptures the three "Ashrams" or stages in life were originally intended for three grades of intelligence thus:

             Religion: low intellects had to do 'karmas' works, ritual actions, chanting of mantras and indulging in bajans and prayers etc.

             Middle intellects:  Yoga: taking yellow robes, going to caves, ashrams etc. 

             High intellects: Philosophy: who wanted truth are concerned with no external rites or sanyas but depend solely on intelligent inquiry for their path. 

Gaudapada says that:- The merciful Veda teaches karma and Upaasana to people of lower and middling intellect, while Jnana is taught to those of higher intellect.

This clearly indicates that religion, which is based on individual conduct, prescribes karma and Upaasana to people of lower and middling intellect, therefore religion is for the lower intellect. And wisdom is for those are capable of inquiring into their own existence to know and realize the ultimate truth, which is Brahman. 

As indicated in ISH Upanishads: - By worshiping gods and goddesses you will go after death to the world of gods and goddesses. But will that help you? The time you spend there is wasted, because if you were not there you could have spent that time moving forward towards Self-knowledge, which is your goal. In the world of gods and goddesses you cannot do that, and thus you go deeper and deeper into darkness.

It clearly indicates that:-If the human goal is to acquire Self-Knowledge or Brahma Gnana or Atma Gnana  then why one has to indulge in rituals and glorifying the conceptual gods, goddesses and gurus[which is karma]  to go in to deeper darkness. Instead   spend that time moving forward towards Self-knowledge, which is one’s prime goal.