# DynamoDB Data Connector Deployment Guide

Operating guide for the DynamoDB data connector in production: IAM, streams, checkpointing, lag behavior, and observability.

Production operating guide for the DynamoDB data connector covering IAM, DynamoDB Streams CDC, checkpointing, and lag handling.

## Authentication & Secrets

DynamoDB authentication uses the standard AWS credential chain. Configure via the same parameters as the [S3 connector](https://spice.ai/docs/building-blocks/data-connectors/s3-deployment#authentication--secrets):

| Parameter | Description |
| - | - |
| `dynamodb_aws_region` | AWS region of the DynamoDB table. |
| `dynamodb_aws_access_key_id` | Explicit access key (optional; falls back to the credential chain when unset). |
| `dynamodb_aws_secret_access_key` | Explicit secret key (optional). |
| `dynamodb_aws_session_token` | Session token for temporary credentials (optional). |

For production on EKS/ECS, leave access-key parameters unset and rely on instance-profile, IRSA, or ECS task-role credentials. Grant the role `dynamodb:Scan`, `dynamodb:Query`, and `dynamodb:DescribeTable` on the table; for streams, additionally grant `dynamodb:DescribeStream`, `dynamodb:GetShardIterator`, and `dynamodb:GetRecords`.

Secrets should be sourced from a [secret store](https://spiceai.org/docs/components/secret-stores) when not using IAM role auth.

## Resilience Controls

### Streams and Checkpointing

The DynamoDB connector supports CDC via DynamoDB Streams with an accelerated dataset as the sink. Stream state is persisted as a checkpoint alongside the accelerator, allowing resumption after a restart.

| Parameter | Default | Description |
| - | - | - |
| `scan_interval` | `0s` | Interval between polls for new records in a DynamoDB stream. |
| `dynamodb_replication_ready_lag` | `2s` | Once replication lag falls below this threshold, the dataset is reported as `Ready`. (Previously `ready_lag`, still accepted as a deprecated alias.) |
| `dynamodb_replication_initial_snapshot` | `auto` | When `refresh_mode: changes` first loads existing items: `auto` scans only when no resumable checkpoint exists, `disabled` streams changes only, `always` scans on every start. |
| `dynamodb_replication_invalid_checkpoint_behavior` | `error` | Behavior when the persisted checkpoint can no longer be honored (past \~24h shard retention): `error` or `restart`. (Previously `lag_exceeds_shard_retention_behavior`, still accepted as a deprecated alias.) |

### Shard Retention and Lag

DynamoDB Streams retain records for 24 hours. If Spice is offline longer than the retention window, the checkpoint becomes stale and the next stream open returns `ShardNotFound`. Behavior is controlled by `dynamodb_replication_invalid_checkpoint_behavior`:

- **`error`** (default): Mark the dataset `Error`. Requires operator intervention to re-bootstrap.
- **`restart`**: Drop the stale checkpoint and re-bootstrap the accelerated dataset from a fresh scan.

A checkpoint older than **18 hours** is treated as near-expired and triggers the same recovery path even if the shard has not yet been dropped by DynamoDB.

The deprecated `lag_exceeds_shard_retention_behavior` alias is still accepted: `ready_after_load` maps to `restart`, and `ready_before_load` (which briefly served stale data before reloading) has been removed and also maps to `restart`.

## Capacity & Sizing

- **Read capacity**: Full-table scans consume provisioned read capacity. Use on-demand billing or reserve sufficient RCU for refresh windows to avoid throttling.
- **Stream throughput**: DynamoDB Streams shards cap at 1000 records/sec and 2 MB/sec each. Wide or high-write tables automatically partition into more shards.
- **Checkpoint storage**: Checkpoint records live in the acceleration engine and add roughly one row per stream shard. Negligible for sizing.

## Metrics

The DynamoDB connector registers the following metrics:

| Metric | Type | Description |
| - | - | - |
| `shards_active` | Gauge | Number of active DynamoDB Streams shards being consumed. |
| `records_consumed_total` | Counter | Total stream records consumed. |
| `lag_ms` | Gauge | Current lag behind the stream head in milliseconds (approximate, summed across shards). |
| `errors_transient_total` | Counter | Transient stream read errors (retried automatically). |
| `reinitializations_on_lag_exceeds_shard_retention_total` | Counter | Number of times the stream was reinitialized due to lag exceeding shard retention. |

Metrics are exposed with the `dataset_dynamodb_` prefix. Monitor `lag_ms` together with `errors_transient_total`: a climbing lag with rising transient errors indicates the connector is falling behind retention.

See [Component Metrics](https://spiceai.org/docs/features/observability/component_metrics) for enabling and exporting metrics.

## Task History

Stream polling and bootstrap operations emit spans that participate in [task history](https://spiceai.org/docs/reference/task_history) under the enclosing `acceleration_refresh` and changes-stream tasks.

## Known Limitations

- **Global Secondary Indexes**: Not exposed as separate datasets. Query the base table and let DataFusion filter.
- **Conditional writes**: DynamoDB conditional expressions (e.g., `attribute_exists`) are not supported in DML operations.
- **Cross-region streams**: Must configure `dynamodb_aws_region` to match the region of the source table; cross-region access requires resource policies and is not recommended.
- **Table with `StreamSpecification` disabled**: CDC mode is unavailable; fall back to full-table refresh.

## Troubleshooting

| Symptom | Likely cause | Resolution |
| - | - | - |
| Dataset stuck in `Error` after restart with stream enabled | Checkpoint older than 18h or exceeded 24h retention. | Set `dynamodb_replication_invalid_checkpoint_behavior: restart` to auto-recover, or trigger a manual refresh. |
| `ProvisionedThroughputExceededException` | RCU exhausted during initial scan. | Switch to on-demand billing, raise RCU for the refresh window, or slow the refresh via acceleration settings. |
| `TrimmedDataAccessException` | Records trimmed from the stream before they could be processed. | Same recovery path as `ShardNotFound`: re-bootstrap. Reduce bootstrap duration via parallel segments if supported. |
| `AccessDeniedException` on `DescribeStream` | IAM role lacks stream permissions. | Add `dynamodb:DescribeStream`, `GetShardIterator`, `GetRecords` to the role. |
| `ResourceNotFoundException` on stream start | Stream not enabled on the table. | Enable streams on the DynamoDB table (`NEW_AND_OLD_IMAGES` recommended). |