Unified stream and batch data processing that’s serverless, fast, and cost-effective.
Fully managed data processing service
Automated provisioning and management of processing resources
Horizontal autoscaling of worker resources to maximize resource utilization
OSS community-driven innovation with Apache Beam SDK
Reliable and consistent exactly-once processing
Ensure RPO and RTO objectives are met for backup and recovery times.
Use Cloud Storage for backup to reduce your data center backup footprint.
Use backups for ransomware recovery, test/dev clones, and analytics.
Minimize pipeline latency, maximize resource utilization, and reduce processing cost per data record with data-aware resource autoscaling. Data inputs are partitioned automatically and constantly rebalanced to even out worker resource utilization and reduce the effect of “hot keys” on pipeline performance.
For processing with flexibility in job scheduling time, such as overnight jobs, flexible resource scheduling (FlexRS) offers a lower price for batch processing. These flexible jobs are placed into a queue with a guarantee that they will be retrieved for execution within a six-hour window.
Enabled through ready-to-use patterns, Dataflow’s real-time AI capabilities allow for real-time reactions with near-human intelligence to large torrents of events. Customers can build intelligent solutions ranging from predictive analytics and anomaly detection to real-time personalization and other advanced analytics use cases.
