Media reports indicated that the Indian High Commission advised the 83-year-old writer to return to India.
Gulzar and Bhardwaj went to Lahore, Pakistan, Monday to record a qawwali for upcoming Bollywood film "Dedh Ishqiya" and the lyricist was also expected to attend the Karachi Literature Festival - but their trip was cut short.
"I had gone (to Lahore) with Vishal, he had a recording there. My baba Ahmed Nasim Qasmi was there and he died sometime back. I had not gone there after his death. So, I visited his grave and offered my prayers, " said the 83-year-old here Wednesday in an interview.
Gulzar, who was born in Dina town in the Jhelum district, which is now in Pakistan, added: "I visited my birthplace after 70 years."
The two came back Wednesday via Amritsar. Gulzar says that it was an emotional turmoil that led him to come back to India.
"After having gone to baba's graveyard, I was emotionally stressed and I started feeling uncomfortable. Vishal saw that I was uncomfortable and he said it would not be right for me to continue with the journey. He postponed the recording and brought me back because I was not mentally ready to get hospitalised there, " he said.
He says they were treated really well by the artists there.
"Here you people are proudly saying that we were asked to leave. They kept us with love, they are our friends, artists, poets musicians. They got our visa done in two days. The Indian High Commission would have told us if there was something like that. If you create such rumours, it is very bad, " Gulzar said.
Being produced by Bhardwaj, "Dedh Ishqiya", a sequel to the 2010 critically acclaimed film "Ishqiya", is being directed by Abhishek Chaubey with Arshad Warsi, Naseeruddin Shah and Madhuri Dixit in the cast.