NelworksNelworks

Finance / Season 2 / EP09

Why Pandas Was Built by a Hedge Fund

Shez

I love Pandas. It's so cute and friendly!

Shez

df.sort_values(by='cuteness'). It organizes my life!

Kurumi

You think it's a toy.

Shez

It is a toy! It's the 'Hello World' of Data Science! Everyone uses it.

Shez

Biologists use it. Marketers use it. It's the universal tool for lists!

Kurumi

Shez. Do you know why Pandas exists?

Shez

Because Python needed spreadsheets?

Kurumi

It exists because in 2008, a guy at a massive Hedge Fund needed to calculate the correlation between 5,000 stocks during a financial meltdown.

Shez

Wait... Pandas is from a Hedge Fund?

Kurumi

Pandas was built by Wes McKinney from AQR Capital Management.

Kurumi

It wasn't built for cat videos. It was built for Quantitative Finance.

Shez

B-But... it's open source! It's free! Wall Street is greedy. It doesn't give away free stuff!

Kurumi

Who knows? If everyone uses your tool, maybe you can hire people easier. But look at the syntax. Really look at it.

Kurumi

pct_change(). Why is that a built-in function?

Shez

To... see how much things changed?

Kurumi

In biology, things grow linearly.

Kurumi

In physics, we measure absolute values.

Kurumi

Only in Finance do we obsess over 'Daily Returns.'

Kurumi

That function exists solely to convert Stock Prices into Returns for volatility modeling.

Shez

What about df.shift(1)? I use that to move rows down.

Kurumi

That is for Lag Generation.

Kurumi

In trading you compare P(t) against P(t-1), constantly.

Kurumi

shift(1) is the time machine required to build a Momentum Strategy.

Shez

They don't line up! Row 2 on Apple is 10:02. Row 2 on Adobe is 10:03.

Shez

If I subtract them, I'm subtracting different times!

Kurumi

That is the time series Alignment Problem.

Kurumi

Pandas aligns data by the Label (The Index), not the integer location.

Kurumi

This prevents you from calculating a 'Fake Correlation' between mismatched times.

Shez

That... is actually really useful for stocks.

Kurumi

It is essential. Without it, your backtest is garbage.

Kurumi

And the name. Pandas.

Shez

Cute Chinese bears! Bamboo!

Kurumi

No.

Shez

Panel Data?

Kurumi

It's an Econometrics term.

Kurumi

Financial data is 3D. You track Many Stocks over Many Days with Many Metrics.

Kurumi

That is a Panel.

Shez

So there is no Chinese bear?

Kurumi

There is only a multidimensional matrix optimized for C-contiguous memory access and bear market predictions.

Shez

Fine. It's a finance tool.

Shez

But why is it so slow? If Quants use it, shouldn't it be instant?

Kurumi

Slow? What do you mean?

Kurumi

Stop! Stop! You are killing it!

Kurumi

Python loops are slow. They check type safety at every step.

Kurumi

Pandas is built on NumPy (C code). It wants to go fast.

Kurumi

When you do this, you aren't adding numbers one by one.

Kurumi

You are sending a command to the CPU: 'Add these two arrays using SIMD instructions.'

Kurumi

Quants need speed. Waiting in a for loop means your profits is going to next door's hedge fund.

Shez

So... df.rolling(20).mean()?

Kurumi

That calculates the 20-Day Moving Average. A classic trading signal.

Shez

df.corr()

Kurumi

Correlation Matrix. Find out which stocks move together or in opposite directions. That's the backbone of portfolio construction.

Shez

This whole time I've been using a Hedge Fund weapon... to sort cats.

Kurumi

You could be making billions instead of kittens.