The composer bagged the best compilation soundtrack for a motion picture and another one for "Jai Ho" for the best motion picture song at the 52nd Annual Grammy Awards.
He won the award with veteran lyricist Gulzar and Tanvi Shah who wrote the Spanish lyrics for it.
"This is insane, god is great again, " a beaming Rahman said after receiving the award at the biggest show in the music world. The other Indian contenders, Zakir Hussain and Amjad Ali Khan, could not manage to win the golden gramophones.
"Insane eh? I never even dreamt about winning all these awards, " Rahman posted on his Twitter page shortly after receiving the award.
Just a year ago, Rahman's music for "Slumdog Millionaire", a screen adaptation of Indian author and diplomat Vikas Swarup's novel "Q & A", had propelled him to the record books as India's first double Oscar winner.
Besides the Oscars and now the Grammys, the Chennai musician's composition for the film has also won him other prestigious international awards like the British Academy of Film and Television Arts (BAFTA), Critics Choice Award and the Golden Globe.
"We all are very proud of him. This magical musician has brought more honour to India, his family, fans, friends and his entire team. Such a young man has made his mark on the music map of the world. All of us should put him on our shoulders, " said Gulzar who shares the prestigious award with him.
"I am proud of Rahman as a special friend and composer who made India proud in the world of music. It's his true dedication towards innovative music and his connection with divine energies. Bravo Rahman!" exclaimed filmmaker Subhash Ghai who roped in the composer for his films "Taal", "Kisna: The Warrior Poet" and "Yuvvraaj".
Director Imtiaz Ali added: "It fills me with pride. Also makes me think if he is getting this much accolade for 'Jai ho', wonder what the reaction of the West will be when they discover the rest of Rahman? I'm waiting to see the man go from strength to strength."
The composer's sister Reihana said winning awards was "a way of life" for her 44-year-old brother.
"It's a good start of the year. It was Rahman's birthday on Jan 6 and now he wins Grammys. It's a way of life now since he has won so many, " Reihana told NDTV after her brother won the award.
"When he was going for the Golden Globes last year, he was doubtful about his win and so were we, but after he got that, it has been awards and awards for him, so we were quite sure that he will get the Grammy as well, " she added.
The song was crooned by Sukhvinder Singh, Mahalaxmi Iyer, Vijay Prakash with support from Rahman and Shah.
Global honours may have come his way only since "Slumdog Millionaire", but the unassuming composer has been contributing to world music for quite a while. He composed music for the "The Lord of the Rings" theatre production, for instance, as well as for Andrew Lloyd Webber's "Bombay Dreams".
His talent first came to the fore with "Roja" in 1992, and there has been no looking back since for the musician, who has just been awarded the Padma Bhushan, India's third highest civilian award.
The accolades have not stopped.