KUALA LUMPUR—When Hussein Juhar, a Rohingya Muslim from Burma’s western Arakan (Rakhine) State, saw the crew of the boat that would smuggle him to Malaysia, he knew he was in trouble. The six men, all armed, were ethnic Arakanese—the very people he was fleeing from. The 30-year-old decided to leave in mid-October after security forces raided his home during sectarian violence between Buddhists and Muslim Rohingyas that left around 100 dead and 100,000 displaced.