The main criteria of failure:

Tresca (The Maximum Shear Stress)
Von Mises (Shear Strain Energy)

These are used for ductile materials but for brittle materials;

Maximum Principal Stress (Rankine)
Mohr Fracture Criterion
are used.

(Reference: `Mechanics of Engineering Materials` by P.P. Benham andR.J. Crawford provide a good introduction.)