---
title: "Duck on the Edge - Snowpack Data"
description: "A pattern for operational analytics in 1st party product"
url: https://snowpack-data.com/blog/duck-on-the-edge
---

[Blog](https://snowpack-data.com/blog)

# Duck on the Edge

A pattern for operational analytics in 1st party product

[Nate Robinson](https://www.linkedin.com/in/nate-robinson-71297383/) · October 27, 2026 · 11 min read

I’ve wanted to write about a data engineering pattern that I’ve fallen in love with in the past few years. I can’t claim to have invented this pattern, and while we’ve deployed a shape of it across multiple projects now, none have had a clear enough story that I can write a detailed case study. So I’m writing about the pattern itself (Duck on the Edge as I like to call it), describing how it works, what it can accomplish, and how I think that it will transform the role of data in the world of AI.

## What is Duck on the Edge?

Duck on the Edge is a pattern for operationalizing large sets of data directly as a first-class product. Duck refers to [DuckDB](https://duckdb.org/), the in-process analytical database, and *edge* refers to the concept of locating the data directly with the user, rather than on a server.

Before I had even seen a formal implementation elsewhere, I had worked on deploying a similar version of the pattern. In 2024 Snowpack Data worked with one of our clients to overhaul the business operations process around their financial ledger. Our solution was a swarm of per-user DuckDB nodes that were spun-up with a subset of ledger data they would need for detailed financial operations work. What ultimately inspired me, however, was a [presentation](https://www.smalldatasf.com/2025/talks/explore23-web-application-for-exploration-of-a-large-genomic-research-cohort) at the 2025 Small Data Conference by Teague Sterling (23andMe Research Institute) on how they had facilitated analysis of massive amounts of genomic data using a similar pattern. We compared notes after his presentation, and we subsequently honed this pattern at Snowpack Data using [DuckDB-WASM](https://duckdb.org/docs/stable/clients/wasm/overview) (DuckDB compiled to WebAssembly) where we have deployed it to phenomenal effect.

## What problems are we trying to solve?

I’ll present you with a use case we solved earlier this year. Suppose you’re a fortune 100 manufacturer, and want to “slice and dice” a [forecast of all of your SKUs](https://snowpack-data.com/services/ml-forecasting) in an interactive UI. You want to be able to compare part forecasts, errors, trends, and then dive into specific details on a part by part basis to update your procurement forecasts. This is a different, operational use case than traditional BI and requires a different recipe. Historically there have been two undesirable options:

1. **Serve analytics directly from your transactional database.**
   
   1. This is often slow and unoptimized for analytical use cases (see OLTP vs OLAP engines).
   2. We need to send data “over the wire” from database to front-end which is slow for high-data requirements
   3. It can throttle our transactional database load and result in a poor customer experience
2. **Traditional BI**
   
   1. This requires a data stack to move and transform data from warehouse, to semantic layer, to BI tool.
   2. This BI tool does not live as part of your product, and is difficult to add operational functionality.
   3. It moves at the speed of your warehouse rather than your transactional database, and round trips can be slow with cold starts and high query traffic.

Both dashboards get the same 20 filter changes.

Figure 1. Example of applying 20 filters against a traditional warehouse vs locally with duck on the edge

This pattern grants us the ability to merge data analysis and operations into the [first-party product](https://snowpack-data.com/services/embedded-product-analytics) and make it actionable instead of supporting only reporting. Our goal is to use data to solve problems rather than to inform solutions. This poses a tradeoff because we introduce limitations, we don’t have the full data warehouse at our fingertips, just a chosen subset of important data. The interactability and supporting semantic model is also very prescriptive, similar to any other product where we want to place tight guardrails around how users can interact with our data. This won’t replace a BI layer, and it won’t replace a data warehouse, but it does present an alternative path for execution on data driven design.

## Embedded Example

A classic problem in the NFL is whether to punt, kick a field goal, or go for it on 4th down, a situation in which coaches notoriously eschew the opinions of us analytics nerds and instead trust their gut. It’s a wonderful problem to detail because it has the same shape as many operational problems as it requires a decision to be made quickly, based on analysis of years of historical data.

Suppose you’re an analyst for an NFL team. You may want to see what teams have historically done on 4th & 2 from their own 45, or how that changes when they’re down 10 in the 4th quarter. You may want to know what each choice is actually worth in expected points, or which teams around the league are the most aggressive in that spot. What if there is a penalty and the yardage changes, you will need to decide in a split second what the choice is.

In the example below you can load the entire corpus of [NFL play-by-play data](https://github.com/nflverse/nflverse-data) for the past 10 years (376K plays, ~10 MB) and set the yards to go, field position, score, and quarter to see what teams did in similar situations and the expected points added (EPA) for each choice. Every adjustment re-queries the full 10 years of history, and the results populate nearly instantly, vs a minimum 2-3 second roundtrip to a warehouse you might expect in a reporting focused tool.

Punt, kick or go for it?

Data from all 376K NFL plays from 2016-2025

Yards to go Ball on Score

Quarter

1st 2nd 3rd 4th

4th-Down Choices in Similar Plays

Go-for-It Rate by Offense in Similar Plays

EPA: expected points added. Data: [nflverse](https://github.com/nflverse/nflverse-data)

## How we build it

Despite the myriad use cases, a consistent data engineering flow facilitates this duck on the edge pattern.

1. Page load requests data
2. Browser loads data in DuckDB and sets up a semantic layer
3. User explores data in UI with data requests served by browser

The technical details are what make or break this system, like how to handle the cold vs warm start. On page load, the application checks with the server whether the latest data watermark is up to date (warm). If it is, the server responds with the latest compressed file location, and the front-end makes an authorized request for the data from a signed URL sitting in bucket storage. This loads a compressed [parquet](https://parquet.apache.org/) file of the required data into the browser and instantiates a web-assembly DuckDB engine running in the browser with a validated set of views (semantic layer) to power the application. This entire process takes less than three seconds.

If the watermark is stale, we enter the cold-start path, prompting the server to publish a new watermarked export which may slow things down, but we find that it still takes <10 seconds to startup on ~1GB of data. There are limitations here, WebAssembly limits each tab to 4GB of memory, but since the views can be defined directly over the already compressed parquet file, DuckDB never needs to unpack the whole thing. Each query only decompresses the columns/rows that it requires, meaning that you can fit a whole hell of a lot of data (tens of millions of rows) using this pattern. It’s also worth noting that anything in the file is readable by the user, so the export needs to be scoped to what that user is allowed to see.

Once the data is loaded, the user can explore the UI seamlessly. Any data query that is requested on the front-end is instantly executed using the in-browser DuckDB engine. And since we have a defined semantic layer, this can extend beyond a simple UI to an embedded SQL interface for unlimited functionality by an agent. [Embedded agent analytics](https://snowpack-data.com/blog/context-your-analytics-agent-is-missing) are an amazing extension on this pattern, where a user can ask open ended questions to an agent, which converts that to a query, issues it locally, and loads the UI nearly instantly.

Another interesting flavor we brought to this UI is the submission of updates. Imagine our procurement planner from earlier wants to adjust the forecast for a dozen parts and see them reflected immediately. We don’t want to issue the requests to the server individually and go down the cold-start path, so we stage them locally. The changes are tracked as atomic edits, the same way a traditional database uses change-data-capture in a write-ahead log, but these updates are immediately applied to the local DuckDB version. When the user is content with their changes or leaves the page, we post a single update of all the staged changes which gets applied as an atomic operation to the production database and a validation confirms that DuckDB has not drifted from prod as a result.

1. The app server exports the rows past the last watermark from its OLTP database.
2. The export goes to cloud storage as Parquet files, about 87% smaller than the same data as CSV.
3. The browser downloads the files. DuckDB-WASM loads the raw export and builds a typed view layer on top of it.
4. The application reads only from the view layer. Every filter runs in the tab.
5. Edits collect in the browser and go to the server as one batch. The batch commits or rolls back as a whole.

Figure 2. Architecture diagram of a duck on the edge product

## The grand vision

If we zoom out on the role of data we can observe a consolidation of a disparate number of subroles that make up the data practice (data engineer, analytics engineer, bi analyst, data scientist), with practitioners ultimately becoming just “data experts”. And the role of a data expert is not to transform data, or build dashboards, or even provide insights; it’s to solve functional problems using data. This duck on the edge pattern is emblematic of that vision and carries us to use cases far beyond traversing a general ledger or NFL 4th down data. As data practitioners, we should be looking for ways to bring data directly to the consumers who need it to solve problems rather than tossing dashboards over the fence. This is a vision that views data as a product rather than a service function. And, as AI pushes the boundaries of what we can do, we’re no longer hampered by the constraints of BI tools, and building a native frontend that can consume our modeled data is trivial once the right APIs are in place.

## What’s next

This model of bringing full end-to-end solutions to data problems using modern engineering techniques is not going away. In the past year we have already landed multiple similar implementations of this pattern and I expect there are more on the way. Extending this pattern into the world of AI through agents embedded in the frontend has already yielded amazing results for the problems we’ve seen. Some of the nuances will change, but the shape is always the same: “how do I enable my consumers to take action with data?”. While this pattern isn’t an end-all-be-all solution for everyone, it represents a frontier of the interaction of data and product and I can’t wait to see what solutions it brings next.

[Nate Robinson](https://www.linkedin.com/in/nate-robinson-71297383/)

A co-founder of Snowpack Data, Nate thrives in technical challenges by bridging data science with application development. A lifelong entrepreneur, he takes pride in constantly applying new learnings to the next challenge.

## More from the blog

[September 3, 2026 **Two Signals That Decide When to Build MLOps**  
\
A framework for investing in MLOps before it becomes an operational burden](https://snowpack-data.com/blog/two-signals-that-decide-when-to-build-mlops)

[May 20, 2026 **The Context Your Analytics Agent is Missing**  
\
Why agentic analytics makes the case for a metrics layer stronger than ever.](https://snowpack-data.com/blog/context-your-analytics-agent-is-missing)

[March 23, 2026 **Data standardization across networked entities**  
\
How PE rollups can reduce the cost and complexity of consolidating portco data by standardizing at the source.](https://snowpack-data.com/blog/pe-rollup-data-standardization)

## References