Workflows / pods taking a long time to terminate

[jrbe228]

Using this environment:
Rancher Desktop: 1.16.0
Argo Workflows: 3.5.11
WASM plugin: 0.3.1

Base OS WSL:
WSL version: 2.3.24.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.65
MSRDC version: 1.2.5620
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.19045.5011

WASM workflows are taking up to 20 minutes to terminate:

Workflows are created using the suggested example in Releases.
kubectl create -f https://raw.githubusercontent.com/Shark/wasm-workflows-plugin/67ae607ec65e1ac93bc1749c5f7a9bde4b673cfd/wasm-modules/examples/ferris-says/workflow.yaml

Other workflows such as the standard hello-world are terminating a few seconds after pod completion, as expected.

[jrbe228]

This behavior may be environment specific. Testing on a production cluster (AKS v1.28.9), workflow completion time is reasonable:

However 30 seconds duration is still very high, relative to WASM framework expectations. From the plugin docs -

It takes a while for Kubernetes to spin up a container and run your code. The process has quite a few steps: pulling a container image, often 100s of megabytes in size, creating namespaces and virtual network interfaces. Starting the runtime for interpreted languages takes a while, too.

Wasm does not emulate a complete operating system as containers do. They are a much simpler abstraction. This means that a module executes in a matter of milliseconds.

What do other users see for their WASM workflows?

[jrbe228]

Surprisingly, performance greatly improved by reducing default times for Env Vars - ARGO_AGENT_PATCH_RATE and DEFAULT_REQUEUE_TIME.
argoproj/argo-workflows#6631 (reply in thread)

Maybe the ideal solution would be an option for client / server mode. Have a "server" pool of WASM server pods waiting. Send new "client" WASM modules to them for execution. Per the docs:

The plugin will run Wasm modules within the plugin process by default.

By default, we have a consistent container image for "server" pods.

Here is a related Workflow showing a similar concept.