Infrastructure

Data Engineering

24 tools · 5 subcategories

The pipelines and platforms that move, transform, and govern data at scale: processing engines like Apache Spark, Kafka, and dbt, ingestion tools like Airbyte and Fivetran, lakehouse formats like Iceberg and Delta Lake, data catalogs, and synthetic data generators like Gretel AI. They feed the training corpora, fine-tuning sets, and RAG knowledge bases everything else depends on. AI-specific labeling lives in AI Data & Human Feedback.