High-level architecture of Kinesis Data StreamsThe following diagram illustrates the high-level architecture of Kinesis Data Streams. The producers continually push data to Kinesis Data Streams, and the consumers process the data in real time. Consumers (such as a custom application running on Amazon EC2 or an Amazon Data Firehose delivery stream) can store their results using an AWS service such as Amazon DynamoDB, Amazon Redshift, or Amazon S3. 
Relaying critical messages from high-throughput IoT devices
Amazon Kinesis Data Stream is a fully managed, scalable, and durable real-time data streaming service designed to continuously capture, process, and store large streams of data. It enables you to ingest high-volume data such as log files, website clickstreams, and IoT telemetry from hundreds of thousands of sources, providing a low-latency pipeline for analytics, application monitoring, and machine learning use cases. Producers send data records to the stream, while consumers process these records in real-time, allowing for immediate insights and actions without the need to manage infrastructure.
Event-driven reference architecture with IoT sensor data
Amazon Kinesis Data Analytics is a fully managed service that allows you to process and analyze streaming data in real time using standard SQL or Apache Flink, enabling developers to build powerful applications that transform, aggregate, and analyze data as it arrives to gain immediate insights from fast-moving streams without managing any servers.
How Launchmetrics improves fashion brands' performance using Amazon EC2 Spot Instances
Amazon Kinesis Data Stream's main role is to provide a highly scalable and durable real-time data streaming service designed for continuously capturing, processing, and storing massive volumes of data records from numerous sources, enabling applications to perform immediate analysis and react to data as it arrives.
What is the difference between Amazon Kinesis Data Stream and Amazon MSK?Amazon Kinesis Data Streams is a fully managed, serverless AWS-native service designed for real-time data ingestion and processing, offering high throughput and minimal operational overhead for use cases tightly integrated within the AWS ecosystem, whereas Amazon MSK provides a managed service for Apache Kafka clusters, enabling users to leverage the open-source Kafka API and its rich ecosystem, ideal for those with existing Kafka applications, specific Kafka configuration needs, or hybrid cloud strategies requiring more direct control over their streaming platform. Architecture for email threat reports and analytics
Amazon Kinesis Data Firehose is a fully managed service that automatically scales to prepare and load streaming data into analytics stores and other destinations like Amazon S3, Amazon Redshift, Amazon Elasticsearch Service, and Splunk. It does not inherently have a dependency on Amazon Kinesis Data Streams; while Firehose can be configured to use a Data Stream as a source, it can also ingest data directly from various producers. For instance, applications running on Lambda or web servers can send data directly to Firehose using its PutRecord or PutRecordBatch API operations without needing an intermediary Kinesis Data Stream.
Modern data streaming architecture with Kinesis Data Streams
Real-time analytics architecture for time series
Enriching & replaying data in real time for event-sourcing microservices
|