The house is located in Pabna in Rajshahi division along the border with India's West Bengal state.
The court directed the government to take legal action against the grabbers and submit a report in 10 days, the Daily Star reported.
The court passed the order following a petition by NGO Human Rights and Peace for Bangladesh.