Open security lakehouse with agentic SOC workflows
A governed Delta and Parquet security foundation that combines continuous telemetry, detection-as-code, historical investigation, and AI-assisted SOC workflows.
ReadArchitecture discussions
Practical patterns for dependable data platforms.
A focused collection of system designs, with the problem, data flow, operating model, best practices, and trade-offs made explicit.
From the architecture desk
Architecture library
Workload-isolated Databricks lakehouse migration
NBCUniversal’s production migration replaces shared reserved capacity with independently scalable job compute and a common platform for SQL, engineering, and ML.
ReadUnified observable platform for energy-market data
A Dagster asset graph turns heterogeneous meter, weather, market, and strategy data into observable, dependency-aware products for forecasting and operations.
ReadCatalog-managed, multi-engine Delta Lake
Unity Catalog becomes the transaction coordinator, credential broker, and policy control plane for managed Delta tables written by heterogeneous engines.
ReadManaged Airflow with DAG-level execution isolation
Checkout.com retains Airflow’s familiar control model while moving patching, scaling, dependency isolation, and per-DAG execution boundaries into a managed platform.
Read