Calculs

├── Calculus
│   ├── Functions
│   ├── Limits
│   ├── Derivatives
│   │   ├── Partial Derivatives
│   │   ├── Gradients
│   │   ├── Jacobian
│   │   └── Directional Derivative                  
│   ├── Chain Rule
│   ├── Taylor Expansion
│   ├── Hessian Matrix
│   ├── Integrals                                   
│   │   ├── Definite Integrals                      
│   │   ├── Multiple Integrals                      
│   │   └── Integration by Parts                    
│   ├── Multivariable Calculus                      
│   │   ├── Divergence                              
│   │   └── Laplacian                               
│   └── Automatic Differentiation                   
│       ├── Forward Mode                            
│       └── Reverse Mode (Backprop)                 

Slope

Mathematically, slope means “steepness of a line.”

$$(x_1,y_1),(x_2,y_2)$$

the slope is:

$$m=\frac{y_2-y_1}{x_2-x_1}$$

This formula is basically:

$$\text{slope}=\frac{\text{change in vertical direction}}{\text{change in horizontal direction}}$$

which is often called rise over run.

Slope isn’t just math scribbles it describes how fast one thing changes compared to another.

• Economics: The slope of a cost graph tells you how much cost increases per unit of product.

• Physics / Motion: If time is on the x-axis and distance is on the y-axis, the slope tells you speed. Example: slope = 60 means 60 km per hour.

• Geography: Slope of a hill = how steep the hill is. A slope of 0.5 means every 1 meter forward, you climb 0.5 meters up.

• Data Science / Trends: If sales are graphed vs. time, the slope tells you how quickly sales are increasing or decreasing.

So slope = rate of change. It answers: “For every unit I move in x, how much does y change?”

Limits

Slope of a line (constant slope) For a straight line, slope is easy:

$$m=\frac{y_2-y_1}{x_2-x_1}$$

That number never changes no matter which two points you pick. Example: line $y=2x+3$ always has slope 2.

The problem: curves (slope not constant)

Real life is not straight.

• Speed changes when you press the accelerator.
• Business profit doesn’t grow in a straight line.
• A hill is steeper in some places, flatter in others.

So asking “what’s the slope of a curve?” is tricky. If you pick two points, you only get the average slope between them, not the instant slope at one exact point.

Example: $y=x^2$

• Between $x=1$ and $x=2$, slope = $(4-1)/(2-1)=3$.
• Between $x=2$ and $x=3$, slope = $(9-4)/(3-2)=5$.

See? It keeps changing.

The idea: zoom in infinitely

The genius of calculus is this thought:

If slope changes, what if we zoom in on the curve so much that it looks like a straight line?

At any tiny point, a curve looks like a line. That line is called the tangent line.

The slope of that tangent = slope of the curve at that exact point.

How do we calculate it? (Limits)

We take two points very close to each other and compute slope:

$$\text{Slope near }x=a=\frac{f(a+h)-f(a)}{h}$$

where $h$ is a tiny step.

Now, we shrink $h$ closer and closer to zero (but never actually zero — otherwise we’d divide by zero).

That shrinking process is called a limit:

$$f^{\prime }(a)=\underset{h\to 0}{\lim }\frac{f(a+h)-f(a)}{h}$$

This is the derivative.

NOTE: Derivative is just a equation we find slope at any point just plug in vaule

What is $\frac{dy}{dx}$?

The notation $\frac{dy}{dx}$ is called a derivative. It represents the rate of change of a function $y$ with respect to the variable $x$.

• $y$ is some function of $x$, i.e., $y=f(x)$.
• $x$ is the independent variable, and $y$ is the dependent variable.

So, $\frac{dy}{dx}$ tells you how much $y$ changes when $x$ changes by a small amount.

What Does It Mean Practically?

Imagine you’re driving a car, and $x$ is the time and $y$ is the distance you’ve traveled. The derivative $\frac{dy}{dx}$ represents the speed of the car how fast the distance $y$ changes as time $x$ changes.

• If $\frac{dy}{dx}=10$, it means that for every 1 unit of time, the distance increases by 10 units (i.e., your speed is 10 units per time).

• If $\frac{dy}{dx}=0$, it means you’re not moving — the distance isn’t changing as time goes on.

Differentiation Rules

1. Power Rule

$$f(x)=x^n(n\text{ is any real number}),$$

then

$$f^{\prime }(x)=n\cdot x^{n-1}.$$

Examples:

• $\frac{d}{dx}(x^2)=2x$

• $\frac{d}{dx}(x^5)=5x^4$

2. Constant Rule

$$\frac{d}{dx}(c)=0$$

(any constant has zero slope).

Example: $\frac{d}{dx}(7)=0$.

3. Constant Multiple Rule

$$\frac{d}{dx}[c\cdot f(x)]=c\cdot f^{\prime }(x)$$

Example:

$\frac{d}{dx}[5x^3]=5\cdot (3x^2)=15x^2$.

4. Sum and Difference Rule

$$\frac{d}{dx}[f(x)\pm g(x)]=f^{\prime }(x)\pm g^{\prime }(x)$$

Example:

$\frac{d}{dx}(x^3+4x^2)=3x^2+8x$.

5. Product Rule

When multiplying two functions:

$$\frac{d}{dx}[f(x)g(x)]=f^{\prime }(x)g(x)+f(x)g^{\prime }(x)$$

Example:

$\frac{d}{dx}[x^2\cdot \sin x]=(2x)(\sin x)+(x^2)(\cos x)$.

Why: slope of a product involves change in both factors.

6. Quotient Rule

For division:

$$\frac{d}{dx}\left(\frac{f(x)}{g(x)}\right)=\frac{f^{\prime }(x)g(x)-f(x)g^{\prime }(x)}{[g(x){]}^2}$$

Example:

$\frac{d}{dx}\left(\frac{x^2}{x+1}\right)=\frac{(2x)(x+1)-(x^2)(1)}{(x+1{)}^2}$.

7. Chain Rule (very important!)

If one function is inside another:

$$\frac{d}{dx}[f(g(x))]=f^{\prime }(g(x))\cdot g^{\prime }(x)$$

Example:

Let’s say we making a cup of coffee, and there’s a machine that automatically heats up water for you. The speed at which the temperature of the water changes depends on how much you’ve turned the knob on the coffee machine. The knob controls the heating power.

Step 1: Breaking Down the Process

1. The knob setting (denote it as $x$) affects the heating power (denote this as $H(x)$), i.e., how strong the heat is.
2. The heating power $H(x)$ affects the rate of temperature change in the water (denote the rate of temperature change as $\frac{dT}{dt}$, where $T$ is the temperature and $t$ is time).
3. Finally, the rate of temperature change $\frac{dT}{dt}$ affects how quickly the water’s temperature increases, which is what you’re interested in.

Step 2: Putting It Together

we can think of this situation as a chain of functions. The heating power depends on the knob setting, and the temperature change depends on the heating power. This is why it’s a composite function (one function inside another).

So we have:

• The rate of temperature change $\frac{dT}{dt}$ depends on the heating power, which in turn depends on the knob setting.

Now, we apply the chain rule to express this relationship. Here’s how it looks:

$$\frac{dT}{dt}=\frac{dT}{dH}\cdot \frac{dH}{dx}$$

• $\frac{dT}{dH}$: This is how the temperature changes with heating power. It tells you how fast the temperature increases for a given amount of heating power.

• $\frac{dH}{dx}$: This is how the heating power changes with the knob setting. It tells you how much the heating power increases when you turn the knob.

8. Derivatives of common functions

• $\frac{d}{dx}(\sin x)=\cos x$

• $\frac{d}{dx}(\cos x)=-\sin x$

• $\frac{d}{dx}(\tan x)={\sec }^{2}x$

• $\frac{d}{dx}(e^x)=e^x$

• $\frac{d}{dx}(\ln x)=\frac{1}{x}$

Partial derviative

What is a Function of Two Variables?

So far, you’ve seen functions like

$$y=f(x)$$

• Input: one variable (x)
• Output: one variable (y)

Example: $f(x)=x^2$

Now, with two variables, the function looks like:

$$z=f(x,y)$$

• Input: two variables (x and y)
• Output: one variable (z)

Example:

$$f(x,y)=x^2+y^2$$

• Think of $x$ and $y$ as coordinates on a flat floor (2D plane).
• Then $z=f(x,y)$ tells you the height above that floor.

So the graph of a two-variable function is a surface in 3D space.

Example:

• $f(x,y)=x^2+y^2$ is a “paraboloid bowl” like a satellite dish.

In single-variable calculus: slope = “how much y changes when x changes.” But here we have two directions of change:

• Change in $x$ (keeping $y$ fixed)
• Change in $y$ (keeping $x$ fixed)

That’s why we use partial derivatives. Example: If

$$f(x,y)=x^{2}y+3y$$

• Partial derivative w.r.t. $x$:

$$\frac{\partial f}{\partial x}=2xy$$

• Partial derivative w.r.t. $y$:

$$\frac{\partial f}{\partial y}=x^2+3$$

These measure slope in each direction.

• $\frac{\partial f}{\partial x}$ = slope of the surface if you walk in the x-direction.

• $\frac{\partial f}{\partial y}$ = slope if you walk in the y-direction.

So calculus in two variables is like studying the shape of a mountain surface slopes in different directions, steepest ascent, valleys, saddles.

Preceptron

We have a model with preceptron of

$$\hat{y}=w_1{x}_1+w_2{x}_2+b$$

y hat is the output to find the loss we use loss mean square

which is

$$L(y,\hat{y})=\frac{1}{2}(y-\hat{y}{)}^2$$

Goal: Adjust $w_1,w_2,b$ so that predictions $\hat{y}$ get close to true values $y$.

For that we need to find how much output y cap is depends or change based on w1,w2, and bias so we need calclus here below things we need

How much Loss change based on bias etc

Problem: The loss $L$ is not a direct function of $w_1,w_2,b$.

It’s hidden inside layers:

$$w_1,w_2,b⟶\hat{y}⟶L$$

This is a composition of functions.

So we use the chain rule:

$$\frac{\partial L}{\partial w_1}=\frac{\partial L}{\partial \hat{y}}\cdot \frac{\partial \hat{y}}{\partial w_1}$$

To find dl/dy cap $L=f(u),u=g(y)$ Where u is $$ (y - \hat{y})

So **L** depends on **u**, and **u** depends on **y** That means: **L** depends on **y** **through** **u** Suppose we nudge $y$ by a tiny amount $\Delta y$. That changes $u$ by:

\Delta u \approx \frac{du}{dy} \cdot \Delta y

Then that change in $u$ changes $L$:

\Delta L \approx \frac{dL}{du} \cdot \Delta u

Now substitute the first relation ($\Delta u$):

\Delta L \approx \frac{dL}{du} \cdot \frac{du}{dy} \cdot \Delta y

Divide both sides by $\Delta y$:

\frac{\Delta L}{\Delta y} \approx \frac{dL}{du} \cdot \frac{du}{dy}

As $\Delta y \to 0$:

\frac{dL}{dy} = \frac{dL}{du} \cdot \frac{du}{dy}

$$Let’stestwith:$$

L = (u)^2, \quad u = 3y

$$\ast Firstway:directsubstitution.$$

L = (3y)^2 = 9y

$$Derivative:$$

\frac{dL}{dy} = 18y

$$\ast Chainruleway:$$

\frac{dL}{du} = 2u, \quad \frac{du}{dy} = 3

$$So:$$

\frac{dL}{dy} = (2u)(3) = 6u

Now plug $u=3y$:

\frac{dL}{dy} = 6(3y) = 18y

\frac{dL}{dy} = \frac{dL}{du}\cdot\frac{du}{dy}

$$![](../../../Images/Pasted![](../../../Images/Pasted$$