Gani was born in Peja, Kosovo in 1958. His family were Albanian Muslims, but at school, being part of the former Yugoslavia, he was taught atheistic Marxism. He studied English Language at university and, as a good student, was invited to become a member of the Communist Party. It was an offer you could not refuse. So he went to Communist Party meetings, but at home he observed the Muslim feasts, though he did not really believe in God at all.