In this demo, we will cover multi-modal data ingestion at scale into an AI-native format, and subsequent model training and fine-tuning workflows. We demonstrate processing several terabytes of healthcare data. We then illustrate advanced data manipulation techniques, including creating virtual dataset views and implementing custom sampling strategies. The demo concludes by showing how this curated data can be queried in natural language and then streamed directly into a multi-GPU training workflow for multi-modal LVMs (Large Vision Models). Throughout the process, we will demonstrate how to maintain comprehensive data lineage for AI, enabling robust version control and facilitating continuous model improvement.

‍