Should we ban public displays of non-Christian faiths?

A speaker at the recent Unite the Kingdom march in London called for banning all public displays of non-Christian religions. But is he right?

Here's what he said: “Ban any type of public expression in our Christian nation from other religions. Ban Halal, ban Burqas. Ban mosques, ban temples, ban shrines - we don’t want those in our countries."

The crowd cheers, but should we?