Physics-Informed Neural Networks and Objective Function Mismatch

It learned to wiggle.

The MSE is low. The R-squared is high. The numbers are happy.

Now, show me the energy consumption plot.

'Weird?' Shez, your robot just invented a perpetual motion machine.

According to your model, to perform that elegant little flip, the arm needs to *generate* 50 Watts of power out of thin air.

It violated the First Law of Thermodynamics.

Anyone who doesn't want their robot to tear a hole in spacetime.

The optimizer is a genie. It will grant your wish *literally*.

You wished for a low MSE. You got a low MSE. You didn't wish for a model that respected reality.

This is the genie's contract.

All it says is 'Make the red line touch the blue dots.' It says *nothing* about Newton, Einstein, or the laws of energy.

A calculator has no common sense. It just calculates.

Your neural network is just a very, very complicated calculator. It will happily divide by zero if you let it.

Let's ask your model to predict the arm's position if it moves 10% faster.

That was **Extrapolation**. Your model is a genius inside the training data, and a complete lunatic outside of it.

Yes. Because you taught it imitation, not comprehension.

To get a smart genie, you need to write a smarter contract.

With more math.

We give the model a **Conscience**.

The sheep wants to eat the grass (Minimize Data Loss).

But we are going to build an electric fence around the field.

This fence is a **Physical Law**. If the sheep tries to cross it, it gets a shock.

The shock is a **Penalty**. We add it to the loss function.

We have two terms now.

**Data Loss:** 'Don't be wrong about what you see.'

**Physics Loss:** 'Don't be stupid about what you don't see.'

That's the voltage on the fence. How much we punish stupidity.

We use **Automatic Differentiation**.

Because every operation in this network is differentiable, we can analytically compute the derivative of the output with respect to the input (time).

We backpropagate not just the data error, but the *physics error* too.

Yes. It finds a solution that is both close to the grass and far from the fence. It learns a physically plausible interpolation.

Of course. We constrained it. It can't wiggle perfectly anymore.

It generalizes. Because it learned the rule, not just the data points.

A model is a tool. You told the first model to be perfect. It lied to you how reality works.

You told the second model to be a good engineer. It designed a movement that is constrained by physics.

That's the way of a scientist.