Real-Time Data Lakehouse Ingestion
Stream live data directly into queryable Apache Iceberg tables. This guarantees immediate data availability for high-performance SQL analyses, e.g. in Dremio Lakehouse, or processing in Spark.
STACKIT Intake reliably captures your data streams and stores them directly in Apache Iceberg tables within your Dremio Iceberg REST catalog. This enables direct data import via Kafka protocol into the STACKIT Data & AI platform. Intake automates the path from raw event streams to structured data and eliminates complex data engineering tasks. This allows your team to focus on valuable insights while we operate the infrastructure, security and scaling in the sovereign STACKIT Cloud.
Applications for STACKIT Intake (Beta)
With STACKIT Intake, you can implement the following use cases, among others:
Accelerated decision-making
Turn operational events into actionable business insights in just a few minutes. Without complex ETL processes, business units gain immediate access to real-time data and drastically reduce the “time-to-insight.”
Future-Proofing Data Infrastructure
Avoid vendor lock-in by using open-source standards such as Apache Kafka and Apache Iceberg. This keeps your data portable and protects your long-term technology investments.
Achieving Digital Sovereignty and Compliance
Migrate sensitive data streams to a 100% European hosted environment. Regulated industries modernize their data ingestion while easily complying with the strictest GDPR requirements.
Features of STACKIT Intake (Beta)
- Use Intake Runners for reserved capacity and specific intakes to easily define your customized ingestion pipelines.
- The technical foundation provides seamless support for the Apache Kafka protocol, JSON message processing and direct Iceberg REST catalog integration.
- Data is instantly transformed into queryable tables through schema intelligence that automatically creates Iceberg tables and evolves them as new fields are added
- 24-hour message buffering prevents data loss, while built-in idempotency guarantees duplicate-free storage in the target table. In addition, an automated dead letter queue (DLQ) collects all data that cannot be delivered due to format or schema problems.
- The service can be seamlessly integrated into developer workflows, allowing you to manage pipelines via the STACKIT portal, CLI and Terraform.
Benefits of STACKIT Intake (Beta)
- You can transfer your data streams to Apache Iceberg tables with a delay of just a few minutes, creating a database that can be queried in near real time for SQL analyses and data-driven insights.
- You no longer need to manage complex Kafka brokers, as STACKIT takes care of scaling, security and maintenance completely in the background.
- Your data streams are protected by a 24-hour buffer window that guarantees reliable delivery even during maintenance work on downstream systems.
- You ensure maximum compliance with a solution in which your data is stored entirely on a strictly GDPR-compliant European cloud infrastructure.
- You can dynamically adjust your throughput capacity at any time from 1 GB/h to up to 20 GB/h to suit your actual business volume.
Contact form
Would you like to find out more about STACKIT cloud solutions? Do you have questions about cloud services, would you like an individual price estimate or would you like to find out more about our company? Use our contact form to get in touch with our experts directly. We will be happy to help you!