Data Warehouse Migration
Unified stream and batch data processing that's serverless, fast, and cost-effective.
• Fully managed data processing service
• Automated provisioning and management of processing resources
• Horizontal autoscaling of worker resources to maximize resource utilization
• OSS community-driven innovation with Apache Beam SDK
• Reliable and consistent exactly-once processing
Benefits
Streaming data analytics with speed
Dataflow enables fast, simplified streaming data pipeline development with lower data latency.
Simplify operations and management
Allow teams to focus on programming instead of managing server clusters as Dataflow’s serverless approach removes
operational overhead from data engineering workloads.
Reduce total cost of ownership
Resource autoscaling paired with cost-optimized batch processing capabilities means Dataflow offers virtually limitless
capacity to manage your seasonal and spiky workloads without overspending.
Key Features
Autoscaling of resources and dynamic work rebalancing
Minimize pipeline latency, maximize resource utilization, and reduce processing cost per data record with data-aware resource autoscaling. Data inputs are partitioned automatically and constantly rebalanced to even out worker resource utilization and reduce the effect of “hot keys” on pipeline performance.
Flexible scheduling and pricing for batch processing
For processing with flexibility in job scheduling time, such as overnight jobs, flexible resource scheduling (FlexRS) offers a lower price for batch processing. These flexible jobs are placed into a queue with a guarantee that they will be retrieved for execution within a six-hour window.
Ready-to-use real-time AI patterns
Enabled through ready-to-use patterns, Dataflow’s real-time AI capabilities allow for real-time reactions with near-human intelligence to large torrents of events. Customers can build intelligent solutions ranging from predictive analytics and anomaly detection to real-time personalization and other advanced analytics use cases.
Solution Partners
