Alibaba DAMO Academy releases ClinFusion open medical AI

Alibaba DAMO Academy released ClinFusion on GitHub. ClinFusion is presented as a vision-centric multimodal large language model system — an artificial intelligence architecture capable of processing both text and visual information simultaneously — built for holistic medical understanding.

Deploying multimodal artificial intelligence in clinical practice involves addressing vision-centric challenges. Medical systems must process heterogeneous 2D images and native 3D volumes while aligning evaluation protocols with clinical workflows. To resolve these limitations, the developers created a compositional and cascaded vision encoder architecture. This structure features a Cascade Spatial-Aware Locality Fusion operator that unifies diverse 2D and native 3D medical image understanding within a single fused encoder.

The project incorporates a vision-grounded evaluation framework consisting of MedIF-Bench for instruction-following assessment and a region-of-interest-grounded method for clinical report generation. According to GitHub documentation, the open-source release includes model weights in 8B and 32B parameter variants, alongside codebase utilities designed to support retrieval-augmented and tool-assisted clinical workflows.

Developers can access the open-source repository and model checkpoints on GitHub and Hugging Face. The release provides installation scripts, configuration templates for evaluation tasks, and guidelines for running inference on custom 2D and 3D medical datasets.