Skip to content

Design — pinky-snowpark

Update date : 2026-05-31 20:23

Snowpark session utilities, telemetry, and stage I/O for Snowpark stored procedures. The layer that makes SPs observable, testable, and production-grade — lifecycle tracing, encrypted stage writes, Excel exports — without coupling business logic to infra concerns.

See ADR-0001 for the SP telemetry design decision.

Module inventory

Module Description Status
snowpark.py Session helpers, SP lifecycle telemetry (load_kwargs, sp_trace), stage I/O, PGP, rotating filenames, EnvPattern planned
excel.py create_workbook_from_df(), parse_worksheet(), 30+ CELL_FORMATS presets planned

SP pattern — load_kwargs + sp_trace

SPs are fully decoupled from Snowflake Tasks. Two functions form the contract:

  • load_kwargs(raw_args, required_keys, optional_keys) — normalises args whether called directly or via a Task DAG (_TASK_NAME$key → clean key). Emits sp_start telemetry automatically.
  • sp_trace(v_result, v_return_values, status) — called in the finally block. Emits sp_end with flat span attributes queryable in EVENT_TABLE without JSON parsing.

The SP does not know it is inside a DAG. The DAG does not know the business logic. Full DAG wiring design: snowflake-provider/provider_design.md.

Future directions

  • SP test harness — lightweight local Snowpark session mock for unit testing SPs without a live connection
  • Registry UI — Streamlit component reading the stage_write() registry table (stage inventory with metadata)
  • Async stage write — background thread upload for large DataFrames, returns a future
  • Conditional cell formatting — alternating row colors, threshold-based highlights in Excel workbooks
  • Snowsight deep links — extend get_current_snowsight_url() to direct links to query results / dashboards