A few people that want to do harm can pack a lot of equipment in a small leaky boat to do that harm. Explosives are pretty well controlled and monitored in Australia but if you can source and transport them from offshore then the ability to do that damage is significant.
I have absolutely no issue with legitimate refugees seeking asylum in Australia and we should do everything we can to assist them. But if the means of their arrival in Australia presents a security risk to those already here then there should be serious restrictions on that method of travel.