The Challenge
nPhase.ai, formerly RedCap Cloud, needed to manage large volumes of clinical research data from multiple sources, transform it through complex ETL pipelines, and make it available for analytics at scale. Traditional relational databases couldn’t handle the analytical query workloads, and building data pipelines required significant engineering effort for each new data source.
Further downstream, the work that turns collected trial data into an FDA submission — mapping it to CDISC standards, deriving analysis datasets, programming every table, listing and figure, and validating the result — is specialist, slow and largely manual, and every step has to stand up to 21 CFR Part 11 scrutiny.
Our Approach
For the data platform, Daiviksoft built a visual ETL tool with a drag-and-drop interface for building data pipelines, on ClickHouse as an OLAP data warehouse, Kafka for real-time data streaming and Kubernetes for orchestrating the microservices. The system ingests clinical research data, transforms it through configurable pipelines, and serves it through analytics dashboards.
For nPhase’s Statistical Computing Environment (SCE), Daiviksoft builds the platform that carries clinical data from raw collection to submission-ready output: SDTM mapping, ADaM dataset generation, table, listing and figure (TLF) creation and integrated Pinnacle 21 validation, multi-tenant and built for 21 CFR Part 11. AI sits inside that workflow rather than beside it. Statistical programs for TLFs and ADaM datasets are generated with AI, and a vision model extracts table layouts from shell documents into an editable design.
An AI assistant works alongside users inside the application. It answers questions about projects and studies through a controlled read layer rather than raw database access, and it can drive the interface — navigating, opening dialogs, filling in fields. Anything that changes durable state, such as saving, submitting or deleting, needs the user’s explicit confirmation, and attachments shared with it are treated as untrusted input. In a regulated environment, that guardrail is the difference between an assistant and a liability.
Key Deliverables
- Visual ETL tool for building data pipelines without code
- ClickHouse OLAP data warehouse for high-performance clinical analytics
- Kafka-based real-time data streaming pipelines
- Kubernetes-orchestrated microservices for scalability
- Data source connectors for clinical research systems, including REDCap (Research Electronic Data Capture) integration
- SDTM, ADaM and TLF automation with integrated Pinnacle 21 validation
- AI-generated statistical programs for TLFs and ADaM datasets
- Vision-model extraction of TLF shells into an editable design
- Statistical analysis plan to Analysis Results Standard (ARS) conversion
- AI assistant with confirmation-gated actions and controlled data access
- Multi-tenant architecture built for 21 CFR Part 11
Results & Impact
nPhase.ai gained a scalable data platform capable of handling clinical research data at volume. The visual ETL tool reduced the time needed to build new pipelines from weeks to hours, while ClickHouse provided sub-second query performance on analytical workloads that previously took minutes. The Kubernetes deployment scales with growing data volumes.
On the statistical computing side, AI now generates the statistical programs behind clinical tables, listings, figures and analysis datasets inside a governed workflow, with Pinnacle 21 validation and a full audit trail behind every output.