Multi-Modal Medical Imaging Analysis With Modern Neural Networks