Data Integration
Amazon Kinesis Data Streams
Amazon Kinesis Data Streams is a managed real-time streaming ingestion service that captures records from producers and durably stores them in shards for replay by consumers. Throughput is provisioned per shard (on-demand mode auto-scales), records persist hours to 365 days, and data is processed via KCL consumer applications or pulled by Firehose, Lambda, and managed analytics services.
AWS
Service information
Shortname: Kinesis
Huawei equivalent shortnames: DIS
Keywords: stream, ingestion, realtime
Differences vs Huawei
DIS provides a comparable partitioned stream abstraction (REST/SDK/Agent) but with different throughput and retention boundaries. DIS documentation states millisecond latency, millions of concurrent connections, and per-partition writes of hundreds of GB/day, with data cached up to 7 days depending on configuration. AWS Kinesis supports shard-level AWS-managed scaling, on-demand mode, and record retention of hours to 365 days. DIS scaling is achieved by manually adding partitions; verify the equivalent of Kinesis automatic shard splitting/merging and extended retention against current Huawei docs before assuming parity.
The producer/consumer APIs and integration surfaces differ. AWS uses the Kinesis Producer Library, Kinesis Client Library (KCL), and AWS SDK record APIs, with native handoff to Firehose, Lambda, Glue, and MSK Connect. DIS exposes its own REST APIs and Huawei SDKs/agents and lists connectors to OBS, MRS, DWS, CloudTable, DLI, and FunctionGraph for dumping and computation. Existing clients written against the Kinesis data plane API are not wire-compatible with DIS; producer/consumer code must be rewritten against the DIS SDK and re-targeted to Huawei downstreams.
Operational responsibility and the service boundary differ. Kinesis Data Streams is a dedicated ingestion-only primitive; downstream ingestion is delegated to Firehose or Lambda, and Kinesis Analytics is a separate service. DIS folds collection, limited in-stream preprocessing (CSV/JSON to Parquet/CarbonData conversion), and managed dumping into one service, reducing Kafka/Flume build effort but narrowing the separation between ingestion, transformation, and analytics layers. Map these boundary differences carefully when porting multi-service Kinesis architectures.
Migration to Huawei
Assess each stream by shard count, throughput, retention, consumer fan-out, and downstream sink (Firehose, Lambda, analytics). DIS is the core Huawei equivalent for the ingestion primitive; for broader integration, security, and governance needs, compose it with CDM (batch movement), DLI (stream/SQL analytics), and DataArts Fabric. Validate DIS partition throughput, 7-day retention ceiling, multi-AZ storage, and regional availability against your workload before committing to cutover.
Repoint producers by replacing KPL/AWS SDK calls with the DIS Agent or DIS SDK against the new stream endpoints; recreate stream names, partition keys, and sequence ordering expectations. Rewrite consumers from KCL checkpoints to DIS checkpoint semantics, and rebuild Firehose delivery flows as DIS dump/connector jobs targeting OBS, DWS, CloudTable, or DLI. Note that there is no documented one-click Kinesis-to-DIS path; migration is a rebuild of producer and consumer code.
Validate record ordering, partition parallelism, replay, and exactly-once vs. at-least-once delivery per consumer after replay, using identical payload schemas. Run dual-write or replay-based shadowing against DIS to compare latency, throughput, and consumer lag before cutover. Confirm IAM/IAM-equivalent access on DIS (tenant isolation, HTTPS, IAM policies) and migrate observability from CloudWatch GetRecords.IteratorAge to DIS Cloud Eye metrics and dump failure logs.
Expect a different cost model and operational scope. AWS bills per shard-hour plus PUT payload units and optional extended retention; DIS is pay-per-use (documented from $0.0117/hour) on partition runtime with storage/transfer for dump targets. Recalculate TCO including peak shard/partition count, retention beyond 7 days (requiring OBS dump), cross-region traffic, and DLI/DWS/CDM runtime where the composed Huawei pattern is used. Treat any claimed performance or savings figures as unverified until validated on the target region.
Huawei Cloud
Huawei equivalent service
Shortname: DIS
General function: Data Integration
Ingest streaming and batch data into analytics platforms.
Keywords: ingestion, stream, data pipeline