I studied data structures and algorithms for the same reason most people do: to get a job.

Grind Leetcode. Pass the screen. Look sharp on paper. That was the plan. What I didn't expect was that the knowledge would reach back nearly a decade and rewrite history on a problem I thought I had put to bed.

The problem I was solving

Almost a decade ago, I was working as an analytics engineer in oil and gas. An engineer on the team had been struggling with a problem in Tableau for weeks. He couldn't crack it. It landed on my desk.

The job was to detect aggressive cycling in a live pipeline. Products moved under pressure, and the engineers needed to know when that pressure was swinging violently between highs and lows. They had a formal framework for this: rainflow counting, a fatigue analysis method developed by Japanese researchers Tatsuo Endo and M. Matsuishi in 1968. The core idea was to convert a chaotic pressure history into identifiable stress cycles, then flag the ones violent enough to cause damage. A stream of sensor readings arriving in real time looked something like this:

Live pressure readings (PSI)
847 ↗ 1,203 ↗ 1,891 ↗ 2,744 ↘ 1,456 ↘ 892 ↗ 1,634 ↗ 2,901 ↘ 1,203 ↘ 567
Peak Valley

Pressure climbs to 2,744 PSI, then collapses to 892. Climbs again to 2,901, then craters to 567. Two cycles. Both violent. Both worth flagging. My job was to catch those turning points as the numbers arrived, calculate the swing between each peak and valley, and fire an alert when the amplitude crossed the engineers' threshold. Simple enough to describe. A nightmare to build.

Why the first solution was messy

Tableau couldn't loop, so I pushed the logic down to the data layer. The team had Alteryx; a drag-and-drop workflow tool, powerful for what it was designed for, and not designed for this. What I built worked. But I can only describe it as a creature stitched together in real time: nodes feeding into nodes, branches compensating for gaps, logic layered on top of logic until it stood upright through sheer stubbornness. I had built something real. I just had no idea what I was standing on.

Alteryx workflow
  • Many nodes and branches
  • Logic layered on logic
  • Difficult to change
  • Hidden complexity
Algorithm pattern
  • Single pass over data
  • Clear mathematical logic
  • Reusable thinking
  • Easier maintenance

Every change request sent a spike of dread through me, and it had nothing to do with the work. I didn't trust what I had built. I was soldering wires while the machine was running, praying nothing shorted.

The deeper problem wasn't Alteryx. It was how I had approached the problem. I started building before I truly understood what I was solving. I never stopped to ask whether the problem had a name, whether someone had already solved it, whether there was a body of knowledge I was supposed to be standing on.

There was. I just didn't know it yet.

The pattern I missed

Years later, on a different job, I started studying data structures and algorithms. Still mostly to pass interviews. I was working through a pattern called the sliding window, a technique for tracking a moving subset of data as you process a sequence. It gave me a mental model I had never had before: how to reason about a stream of values as a living, moving thing rather than a static dataset to be queried.

That idea led me to another pattern: the monotonic stack.

The monotonic stack connection

A structure that maintains order as values arrive in a stream, holding them until the moment the trend breaks. Then it reveals something simple but powerful: a peak or a valley. In a single pass, it reduces chaos into structure.

How a monotonic stack finds turning points
↓
Incoming values
⇵
Maintain order
⚠
Trend breaks
✓
Reveal peak or valley

I was working through an example when something shifted. I put the book down.

That's the pipeline.

The sensor readings. The peaks. The valleys. The same logic I had once stitched together in Alteryx, except now expressed cleanly, mathematically, almost trivially.

Peak detected, flag it. Valley detected, flag it. Measure the swing. Compare it to the threshold. Trigger the alert.

The entire system I had spent weeks building was sitting there, reduced to its mathematical skeleton.

It was the same idea I had already been wrestling with in a more complicated form: tracking turning points in a stream of data as the trend shifts between rising and falling.

Here is what it looks like in Python in less than 10 lines. Clean, readable, fast. The kind of code you can hand to someone else without a thirty minute explanation and a prayer.

detect_cycles.py 
def detect_turning_points(readings, threshold):
    peaks, valleys = [], []
    for i in range(1, len(readings) - 1):
        if readings[i] > readings[i-1] and readings[i] > readings[i+1]:
            peaks.append((i, readings[i]))
        elif readings[i] < readings[i-1] and readings[i] < readings[i+1]:
            valleys.append((i, readings[i]))
    return [(p, v) for p in peaks for v in valleys
            if abs(p[1] - v[1]) >= threshold]

What changed in how I think

Three things became clear.

1
Your tools shape your thinking in ways you don't notice until it's too late.
I had Alteryx, I knew Alteryx, so I reached for Alteryx. I never questioned whether it was the right instrument. To someone who only knows how to use a hammer, every problem looks like something to be hammered.
2
Speed is a trap in the early stages of a problem.
The time I saved by jumping straight to building, I paid back with interest every time someone asked for a change. A problem understood slowly is a problem half solved. The most valuable thing I could have done in that first week wasn't to open Alteryx. It was to sit with the problem, decompose it, and ask what it actually was.
3
Almost every problem you will ever face has already been solved.
Rainflow counting was formalized in 1968. The monotonic stack predates my career by decades. The library exists. The literature exists. The elegant solution exists. My only job is to find it, understand it, and apply it.

I call this standing on the shoulders of others. It sounds humble. It is. But it is also the most efficient thing a problem solver can do.

DSA sharpened my resume. It sharpened my interviews. What I didn't expect was that it would sharpen how I think.

The problem was already solved. It usually is. You just have to recognize it when you find it.
Originally published on Medium. Read the original article →