The actor, in his mid-60s, was first taken to Jewel Hospital in Irla in western Mumbai Thursday morning but later transferred to Lilavati Hospital, Bandra.
Joy was the typical "lover boy" of the 60's Bollywood with his bright eyes, gelled hair and enjoyed a huge fan following.
His combination with music composer O.P. Nayyar and playback singer Mohammed Rafi saw many successful films like "Ek Musafir, Ek Hasina" (1962) and "Phir Wohi Dil Laya Hoon" (1963).
His very first film, "Love in Simla" (1960), catapulted him to stardom. At the time, Joy was in his final year of graduation.
In 1968, he tried his hand at production with "Humsaya", but the film did not do well.
He also directed "Love in Bombay" (1974), "Chhaila Babu" (1977), "Saanjh Ki Bela" (1981) and "Umeed" (1989).
Actresses Kajol and Rani Mukherjee are his nieces.