Different groups of Rafi fans are celebrating the day in their own way.
"With his vocals transcending all human barriers and frontiers, Rafi-ians, as we like to call ourselves, based in Mumbai have formed the Rafi Foundation since 2007 and strive to bring Rafi sahab's music on a common platform, " said Binu Nair, the founder of the foundation.
Nair said that a group of fans has already offered flowers at the Mohammad Rafi Chowk in Bandra in northwest Mumbai.
"We will go to his residence where his three daughters and a son live and cut a cake in his honour, " he said.
"We have also organised a musical evening in his honour where nearly 3, 000 fans will enjoy the evening amidst Rafi songs played by a 60-piece orchestra, " he added.
Nair said at least 10 public shows have been organised in Rafi's fond memory across Mumbai.
"We have lost count of the private gatherings at residences of Rafi fans in his memory, " he said.
Another big celebration is slated at Rang Sharda auditorium in Bandra organised by Bharatiya Janta Party (BJP) leader Ashish Shelar.
"We have been remembering Rafi sahab on his birth and death anniversaries and have been organising musical evenings since many years now. This is a small way to acknowledge his legendary contribution in the field of Indian music, " Shelar said.
Rafi gave Indian cinema 40 golden years of haunting melodies like "Chaudhvin ka chand", "Teri aankhon ke siva" and "Tum jo mil gaye ho".
Born in Punjab, he began his tryst with music by chanting the notes that a "fakir" (Muslim seer) sang in his village. The family later moved to Lahore.
Rafi trained under Ustad Bade Ghulam Ali Khan, Ustad Abdul Wahid Khan, Pandit Jiwanlal Matto and Firoze Nizami. He sang with K.L. Saigal at a concert for the first time at the age of 13. That was the turning point in his life.
He was honoured with the Padma Shri in 1967. He died of a heart attack in Mumbai July 31, 1981, at the age of 56.