Pretty long story...Wiki covers it well.
https://en.wikipedia.org/wiki/History_of_Kashmir
However, the fact is that the Pandits were evacuated after a major terrorist action that also targeted many Muslims. The Janata Dal government at the centre had formed a coalition with the BJP and the Home Minister was a BJP member. They decided it would be best to evacuate the Pandits out of Kashmir. However, they were then abandoned. It was after Congress regained power that they were provided any relief and rehabiltation.
Then when the BJP rose to become the single largest and most pwerful party in India, they decided to use the Kashmiri Pandit crisis to their own ends. While promising to facilitate their return, the BJP has actually used it as excuse to remove environmental, residential, and ownership restrictions on resources in Kashmir primarily for exploitation by BJP's biggest corporate benefactors. The plight of the pandits is only brought up to trigger an anti-Muslim agenda and is otherwise ignored.