Goff not banning far-right speakers, but won't provide venue