The Defence Ministry and the Armed Forces will not be deterred by baseless allegations against them, says its caretaker minister Datuk Seri Hishammuddin Tun Hussein.