DMT

Overview

DMT (Data-Models-Tests) is a validation framework for computational science that treats scientific validation not as a pass/fail test but as a structured, reproducible debate. It provides a methodology to build, evaluate, and disseminate computational models through the formal separation of data providers, model adapters, and validation writers.

The Philosophy

DMT is grounded in a central tenet: "There is no correct model. But some may be useful." The framework reimagines the interaction between Data, Models, and Tests to mirror the open, debate-driven process of traditional science.

On Models: Useful Approximations

Models are treated as experimental objects — just as a biologist probes a brain slice in a wet lab, a computational scientist should be able to probe a digital model. The framework requires models to expose interfaces for measurement, turning the model itself into a subject of scientific inquiry.

On Data: Integration, Not Just Validation

The framework makes experimental data consumable — readily accessible through standardized interfaces rather than locked in papers or proprietary formats. It also enforces the statistical rigor of separating verification data (used to build the model) from validation data (used to test predictions).

On Tests: Validation as Continuous Debate

A model is never "fully validated." Validation is a continuous process where new experimental data triggers new testing cycles. DMT formalizes this by defining validation as a computational implementation of a scientific comparison.

Architecture: The Three-Party System

DMT's core innovation is decoupling the three participants in scientific validation:

1. Data Interface Author — provides experimental data and readers
2. Model Adapter — writes a wrapper that lets any model be measured by the validation framework (Adapter Pattern)
3. Validation Writer — implements the test: methodology, statistical comparison, and judgment logic

This separation enables many-to-many relationships: one validation applied to multiple models, one model subjected to multiple validations. The output is a Report — a rich, reproducible artifact containing not just a score but the scientific context, methods, data, and plots.

Technical Architecture

• Adapter Pattern — Interface (the contract), Adapter (the translator), AIBase (automatic interface extraction via @interfacemethod)
• Plugin-based extensibility — new models, data sources, and validations plug in without modifying core code
• Scientific narrative engine — automated generation of structured reports documenting every validation
• Configuration-driven — YAML definitions for validation campaigns

Impact

• Used to validate brain circuit models at the Blue Brain Project against experimental measurements across multiple scales (cellular, synaptic, network)
• Enabled systematic comparison of model predictions vs. experimental observations
• Framework is domain-agnostic — applicable to any field where computational models must be validated against empirical data (climate science, drug discovery, materials science)

Technical Stack

Python · Adapter Pattern / Plugin Architecture · Abstract Base Classes / Metaprogramming · YAML configuration · Automated report generation · Matplotlib / Seaborn visualization

← Projects