The company, founded by two former Microsoft Research employees, uses unsupervised machine learning to discover malicious behavior.