Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 96 additions & 23 deletions docs/sdk-reference/state/serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,13 @@ Each SDK uses a default SerDes when you do not provide one.

=== "C#"

There is no per-operation default SerDes. Register a single `ILambdaSerializer` at the
host boundary and the SDK uses it for every durable operation result.
The default is the single `ILambdaSerializer` registered at the host boundary — the SDK
uses it for every durable operation result and for the handler's return value. An
optional per-operation override is available (see
[Custom SerDes on durable operations](#custom-serdes-on-durable-operations)): set
`Serializer` on `StepConfig`, `CallbackConfig`, `InvokeConfig`,
`WaitForConditionConfig<TState>`, or `ChildContextConfig` to use a different
`ILambdaSerializer` for that one operation.

Use `DefaultLambdaJsonSerializer` (from `Amazon.Lambda.Serialization.SystemTextJson`)
for reflection-based serialization. For AOT or trim-friendly functions, use
Expand Down Expand Up @@ -163,8 +168,10 @@ Each SDK uses a default SerDes when you do not provide one.

=== "C#"

.NET has no per-operation SerDes interface. Serialization is controlled by the single
`ILambdaSerializer` registered on `ILambdaContext.Serializer`.
.NET has no separate per-operation SerDes interface — it reuses the standard
`ILambdaSerializer` contract. Serialization defaults to the single `ILambdaSerializer`
registered on `ILambdaContext.Serializer`, and an individual operation can override it
by setting `Serializer` (an `ILambdaSerializer`) on that operation's config.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/SerdesInterface.cs"
Expand Down Expand Up @@ -240,9 +247,10 @@ same handler continue to use the default.

=== "C#"

`StepConfig` does not expose a serializer. The step result is serialized with the
`ILambdaSerializer` registered at the host boundary. Register a custom
`ILambdaSerializer` there to change how step results are serialized.
Set `StepConfig.Serializer` to serialize this step's result with a specific
`ILambdaSerializer`. When it is `null` (default), the step result is serialized with the
`ILambdaSerializer` registered at the host boundary. Only this step is affected; other
operations and the handler's return value continue to use the registered serializer.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/StepConfigExample.cs"
Expand Down Expand Up @@ -276,9 +284,10 @@ system sends when it completes the callback.

=== "C#"

`CallbackConfig` does not expose a serializer. The payload the external system delivers
is deserialized with the `ILambdaSerializer` registered at the host boundary. Register a
custom `ILambdaSerializer` there to change how the callback payload is deserialized.
Set `CallbackConfig.Serializer` to deserialize the callback payload with a specific
`ILambdaSerializer`. When it is `null` (default), the payload the external system
delivers is deserialized with the `ILambdaSerializer` registered at the host boundary.
Only the deserialize path is used for callbacks.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/CallbackConfigExample.cs"
Expand Down Expand Up @@ -320,9 +329,12 @@ Map and parallel perations support two SerDes fields that apply at different lev

=== "C#"

`MapConfig` does not expose a serializer, and there is no separate item-level
serializer. Each item result is serialized with the `ILambdaSerializer` registered at
the host boundary. `ParallelConfig` likewise has no serializer field.
Set `MapConfig<TItem>.ItemSerializer` (and `ParallelConfig.ItemSerializer`) to serialize
each item / branch **result** with a specific `ILambdaSerializer`. When `null` (default),
item results use the `ILambdaSerializer` registered at the host boundary. There is no
separate whole-result serializer: the aggregated batch envelope (per-item statuses and
completion reason) is an SDK-internal, source-generated structure and is not
user-serialized — only the per-item results are.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/MapConfigExample.cs"
Expand Down Expand Up @@ -422,9 +434,15 @@ that every Lambda execution environment can read. In AWS Lambda, this means:

=== "C#"

Not available. The .NET SDK has no FileSystem serdes. To keep large payloads out of the
checkpoint, write them to a persistent store (such as Amazon S3) inside a step and
checkpoint a pointer instead of the payload.
`FileSystemSerializer` wraps an inner `ILambdaSerializer` and writes each result to the
mounted filesystem, keeping only a file pointer in the checkpoint. Pass it to a single
operation through `StepConfig`, `ChildContextConfig`, or `WaitForConditionConfig`, or as the
`ItemSerializer` on `MapConfig`/`ParallelConfig`. Other operations in the same handler
continue to use the serializer registered at the host boundary.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/FileSystemSerdesWalkthrough.cs"
```

### Create a FileSystem serdes

Expand Down Expand Up @@ -470,7 +488,39 @@ Create a FileSystem serdes and pass it to an operation's config.

=== "C#"

Not available.
`FileSystemSerializer` has two constructors, both returning a serializer that implements
both `ILambdaSerializer` and `IDurableResultSerializer`:

- `new FileSystemSerializer(inner, basePath, storageMode?, pathEncoding?)` — you supply
the inner serializer explicitly.
- `new FileSystemSerializer(basePath, storageMode?, pathEncoding?)` — no inner serializer.
The durable runtime binds the serializer registered at the host boundary (the
`[assembly: LambdaSerializer(...)]` serializer, or the one passed to
`LambdaBootstrapBuilder.Create(handler, serializer)`) as the inner when the instance
runs through a per-operation serializer slot. Use this when the on-the-wire format is
just the function's normal serializer, so you do not have to thread it in yourself.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/FileSystemSerdesSignature.cs"
```

**Parameters:**

- `inner` The `ILambdaSerializer` that converts values to and from bytes (for example
`DefaultLambdaJsonSerializer`, or a wrapper that compresses). This is where the
on-the-wire format is controlled. Omit it to reuse the serializer registered at the
host boundary. An explicitly-supplied inner always wins over the global one.
- `basePath` Directory where the SDK writes data files. Set this to your filesystem
mount point (for example `/mnt/efs`).
- `storageMode` (optional) A `FileSystemStorageMode` value. Default: `Always`.
- `pathEncoding` (optional) A `FileSystemPathEncoding` value. Default: `Uri`.

**Returns:** A serializer that reads and writes files under `basePath`.

A `FileSystemSerializer` built without an inner serializer only resolves the global
serializer when it is used through a per-operation slot (for example
`StepConfig.Serializer`). Using it as a plain `ILambdaSerializer` with no bound inner —
for instance as the host-boundary serializer — throws `InvalidOperationException`.

### FileSystemSerdesConfig

Expand Down Expand Up @@ -516,7 +566,13 @@ Create a FileSystem serdes and pass it to an operation's config.

=== "C#"

Not available.
The .NET SDK has no separate config object — pass `storageMode` and `pathEncoding` as
constructor arguments (see above). The **inner serializer** is where you control the
on-the-wire format; wrap it to compress results before they are written to disk.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/FileSystemSerdesCompression.cs"
```

### Storage modes

Expand Down Expand Up @@ -548,7 +604,13 @@ execution checkpoint size limit. See

=== "C#"

Not available.
`FileSystemStorageMode.Always` (default) writes every result to a file; the checkpoint
holds only the pointer. `FileSystemStorageMode.Overflow` keeps the result inline until it
would exceed the durable execution checkpoint size limit, then spills to a file.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/FileSystemSerdesOverflow.cs"
```

### Path encoding

Expand Down Expand Up @@ -585,7 +647,14 @@ enough to exceed the name-length limit.

=== "C#"

Not available.
`FileSystemPathEncoding.Uri` (default) builds human-navigable paths from the durable
execution ARN, with the entity id as the file name. `FileSystemPathEncoding.Hash` replaces
the ARN (directory) and entity id (file name) with their SHA-256 hex digest — fixed length
and always filesystem-safe.

```csharp
--8<-- "examples/csharp/sdk-reference/serialization/FileSystemSerdesPathEncoding.cs"
```

### Preview and PII masking

Expand Down Expand Up @@ -659,7 +728,9 @@ default.

=== "C#"

Not available.
Not yet supported in the .NET SDK. `FileSystemSerializer` writes the full serialized value
to the filesystem and stores only a file pointer in the checkpoint; inline previews and
field masking are planned as a follow-up.

### Set as the default for the handler

Expand Down Expand Up @@ -689,8 +760,10 @@ once with `configureSerdes`.

=== "C#"

Not available. The single `ILambdaSerializer` you register at the host boundary is
already applied to every durable operation result in the handler.
Not yet supported in the .NET SDK. Set `FileSystemSerializer` per operation via
`StepConfig.Serializer` (or `ItemSerializer` for Map/Parallel). An execution-wide default
serializer hook is planned as a follow-up; until then, the single `ILambdaSerializer` you
register at the host boundary applies to any operation that does not set its own.

## Serialization errors

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

public class CallbackConfigExample
{
Expand All @@ -9,13 +10,14 @@ public Task<DurableExecutionInvocationOutput> Handler(

private async Task<ApprovalResult> Workflow(object input, IDurableContext ctx)
{
// CallbackConfig has no serializer slot. The callback payload delivered by
// the external system is deserialized with the ILambdaSerializer registered
// on ILambdaContext.Serializer. To customize deserialization, register a
// custom ILambdaSerializer at the host boundary.
// Set CallbackConfig.Serializer to deserialize the callback payload with a specific
// ILambdaSerializer. When null (default), the payload the external system delivers is
// deserialized with the ILambdaSerializer registered on ILambdaContext.Serializer.
// Only the deserialize path is used for callbacks.
var config = new CallbackConfig
{
Timeout = TimeSpan.FromHours(1),
Serializer = new DefaultLambdaJsonSerializer(),
};

ICallback<ApprovalResult> callback =
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
using System.IO.Compression;
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

// The inner serializer controls the on-the-wire format. Wrap one to compress results before
// they are written to the filesystem — FileSystemSerializer just handles storage.
public sealed class GzipJsonSerializer : ILambdaSerializer
{
private readonly ILambdaSerializer _inner = new DefaultLambdaJsonSerializer();

public void Serialize<T>(T response, Stream responseStream)
{
using var gzip = new GZipStream(responseStream, CompressionLevel.Optimal, leaveOpen: true);
_inner.Serialize(response, gzip);
}

public T Deserialize<T>(Stream requestStream)
{
using var gzip = new GZipStream(requestStream, CompressionMode.Decompress, leaveOpen: true);
return _inner.Deserialize<T>(gzip);
}
}

public static class FileSystemSerdesCompression
{
// Results are JSON-encoded, gzip-compressed, then written to EFS.
public static ILambdaSerializer Create() =>
new FileSystemSerializer(inner: new GzipJsonSerializer(), basePath: "/mnt/efs");
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

public static class FileSystemSerdesOverflow
{
// ALWAYS (default): every result is written to a file; the checkpoint holds only a pointer.
public static ILambdaSerializer Always() =>
new FileSystemSerializer(
new DefaultLambdaJsonSerializer(), "/mnt/efs", FileSystemStorageMode.Always);

// OVERFLOW: the result stays inline in the checkpoint until it would exceed the durable
// execution checkpoint size limit (~256 KB), then it spills to a file.
public static ILambdaSerializer Overflow() =>
new FileSystemSerializer(
new DefaultLambdaJsonSerializer(), "/mnt/efs", FileSystemStorageMode.Overflow);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

public static class FileSystemSerdesPathEncoding
{
// URI (default): human-navigable paths built from the durable execution ARN
// (function / execution / invocation), with the entity id as the file name.
public static ILambdaSerializer Uri() =>
new FileSystemSerializer(
new DefaultLambdaJsonSerializer(), "/mnt/efs",
pathEncoding: FileSystemPathEncoding.Uri);

// HASH: the ARN (directory) and entity id (file name) are each replaced by their SHA-256
// hex digest — fixed length and always filesystem-safe, regardless of length or charset.
public static ILambdaSerializer Hash() =>
new FileSystemSerializer(
new DefaultLambdaJsonSerializer(), "/mnt/efs",
pathEncoding: FileSystemPathEncoding.Hash);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

// FileSystemSerializer implements both ILambdaSerializer and IDurableResultSerializer. It
// wraps an inner ILambdaSerializer that performs the actual value<->bytes conversion, so you
// control the on-the-wire format (JSON, compressed, custom).
public static class FileSystemSerdesSignature
{
public static ILambdaSerializer Create() =>
new FileSystemSerializer(
inner: new DefaultLambdaJsonSerializer(), // required: value <-> bytes
basePath: "/mnt/efs", // required: durable mount point
storageMode: FileSystemStorageMode.Always, // optional, default Always
pathEncoding: FileSystemPathEncoding.Uri); // optional, default Uri

// Omit the inner serializer to reuse the serializer registered at the host boundary
// (the [assembly: LambdaSerializer(...)] serializer, or the one passed to
// LambdaBootstrapBuilder.Create(handler, serializer)). The durable runtime binds it as
// the inner when this instance runs through a per-operation serializer slot, so you do
// not have to thread the global serializer in yourself.
public static ILambdaSerializer CreateUsingGlobalSerializer() =>
new FileSystemSerializer(
basePath: "/mnt/efs", // required: durable mount point
storageMode: FileSystemStorageMode.Always, // optional, default Always
pathEncoding: FileSystemPathEncoding.Uri); // optional, default Uri
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

public class FileSystemSerdesWalkthrough
{
public Task<DurableExecutionInvocationOutput> Handler(
DurableExecutionInvocationInput input, ILambdaContext context)
=> DurableFunction.WrapAsync<object, Report>(Workflow, input, context);

private async Task<Report> Workflow(object input, IDurableContext ctx)
{
// FileSystemSerializer wraps an inner ILambdaSerializer (here the default JSON
// serializer) and writes each result to the mounted filesystem, keeping only a small
// file pointer in the checkpoint. Pass it to a single operation's config; other
// operations continue to use the serializer registered at the host boundary.
var fileSystem = new FileSystemSerializer(
inner: new DefaultLambdaJsonSerializer(),
basePath: "/mnt/efs");

Report report = await ctx.StepAsync(
async (_, _) => await BuildLargeReportAsync(),
name: "build-report",
config: new StepConfig { Serializer = fileSystem });

return report;
}

private static Task<Report> BuildLargeReportAsync() => Task.FromResult(new Report("…large body…"));
}

public record Report(string Body);
10 changes: 7 additions & 3 deletions examples/csharp/sdk-reference/serialization/MapConfigExample.cs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.DurableExecution;
using Amazon.Lambda.Serialization.SystemTextJson;

public class MapConfigExample
{
Expand All @@ -9,12 +10,15 @@ public Task<DurableExecutionInvocationOutput> Handler(

private async Task<IReadOnlyList<ProcessedItem>> Workflow(object input, IDurableContext ctx)
{
// MapConfig has no serializer slot. Each item result is serialized with the
// ILambdaSerializer registered on ILambdaContext.Serializer. To customize
// serialization, register a custom ILambdaSerializer at the host boundary.
// Set MapConfig.ItemSerializer to serialize each item's RESULT with a specific
// ILambdaSerializer. When null (default), item results are serialized with the
// ILambdaSerializer registered on ILambdaContext.Serializer. This controls only the
// per-item result — not the aggregated batch envelope (statuses / completion
// reason), which is SDK-internal. ParallelConfig has the same ItemSerializer field.
var config = new MapConfig<string>
{
MaxConcurrency = 3,
ItemSerializer = new CamelCaseLambdaJsonSerializer(),
};

var items = new[] { "a", "b", "c" };
Expand Down
Loading
Loading