Ending a long wait by patient listeners at the Netaji Indoor stadium, the icon with effortless ease slipped into the popular rendition after about an hour into the concert.
Penned by Indian Urdu poet and freedom fighter Maulana Hasrat Mohani, "Chupke Chupke" was sung by Ali in his baritone voice for B.R. Chopra's "Nikaah" (1982).
The rendition has since attained cult status among moviegoers in the subcontinent and even three decades later, it continues to evoke a huge response, as was seen during Tuesday's concert.
Ali's magical voice, his mastery over enunciation and fluidity with tones and lyrics entranced the packed hall.
There was never a dull moment as the ustad kept up an engaging conversation with the audience through explanations of the shayaris and ghazals he sang, in his characteristic lively and jovial demeanour.
However, spectators which included the creme de la creme of Bengal, seemed to be overwhelmed with a strong dose of nostalgia, when Ali began "Chupke Chupke".
Filmmaker Mahesh Bhatt, who recently roped in Ali to sing for his play "Milne Do", was also in attendance.
Applause welcomed the rendition which saw his son Aamir joining in.
"It feels good as an artist when the audience enjoys and listens carefully," said Ali.
After cancellation of his concert in Mumbai due to threats by the Shiv Sena, Ghulam Ali in November had expressed disappointment asserting "never to return to India".
Chief Minister Mamata Banerjee offered Kolkata as a venue following the cancellation of the Mumbai concert in October.