Bg 4.20

tyaktvā karma-phalāsaṅgaṁ
nitya-tṛpto nirāśrayaḥ
karmaṇy abhipravṛtto ’pi
naiva kiñcit karoti saḥ

Synonyms: 

tyaktvā — having given up; karma-phala-āsaṅgam — attachment for fruitive results; nitya — always; tṛptaḥ — being satisfied; nirāśrayaḥ — without any shelter; karmaṇi — in activity; abhipravṛttaḥ — being fully engaged; api — in spite of; na — does not; eva — certainly; kiñcit — anything; karoti — do; saḥ — he.

Translation: 

Abandoning all attachment to the results of his activities, ever satisfied and independent, he performs no fruitive action, although engaged in all kinds of undertakings.

BG 4.20

Act in detachment. Criteria is not to give up karma but give up attachment, giving up selfishness.

Right now he is not talking about devotional thing or about Himself, he is talking about Nira-ashraya, Kirshan is not focusing about Himself. Krishna speaks about Himself in response to Arjuna’s question but from 4.16 onward Krishna is basically talking about Karma Yoga. From 3.35 Krishna is talking about this…3.35 he told act according to ones dharma, a slight detour came as Arjuna has asked a question in 3.36 What makes on act sinfully, so 3.37 to 3.43 was answer to that question, then when Krishna was explaining the history of that knowledge Arjuna asked a question how could have you spoken to Sun god in 4.4 to response to his question Krishna talked about his supreme position from 4.5 to 4.15 but essentially Shri Krishna is talking about Karma Yoga. And in this connection He says yes a person may act but if he is not selfishly motivated he will not be bound.  So here Nira-ashra doesn’t refer to taking shelter of Himself till now Krishna has not introduced to himself as param asaraya. That he will introduce in later chapter and in 18.66 He will say serva dharman paritayaja…here he is talking about being detached…

There are two aspects, to be detached from worldly things and be attached to Krishna, so right now the focus is be detached. So nirasharaya means don’t take shelter of anyone, means donot become attached to anyone materially. The same point Shri Krishna has talked about in 3.19.

In BG 3.19 … such a person doesn’t need to take shelter of anyone else means don’t become attached to anyone else materially, such a person will not be bound.