Numerical Differentiation

Cheatsheet Content

### Introduction to Numerical Differentiation Numerical differentiation is about approximating the derivative of a function. This is essential when: - The function is given as a set of discrete data points (e.g., from an experiment). - The function's analytical derivative is too complex or impossible to find. **Why it matters:** - **Rate of Change:** Differentiation measures how quickly a quantity changes. For example, velocity is the derivative of position, and acceleration is the derivative of velocity. - Position: $x = f(t)$ - Velocity: $v = \frac{df(t)}{dt}$ - Acceleration: $a = \frac{dv(t)}{dt}$ **Finite Difference Approximation:** Numerical differentiation often uses the concept of **finite differences**, where the derivative at a point is approximated by the slope of a line connecting nearby points on the function curve. ### First Derivative Formulas These formulas approximate the first derivative $f'(x_i)$ at a point $x_i$ using function values $f(x)$ at $x_i$ and neighboring points. $h$ is the step size ($h = x_{i+1} - x_i$). 1. **Two-Point Forward Difference** (using $x_i$ and $x_{i+1}$): $$f'(x_i) \approx \frac{f(x_{i+1}) - f(x_i)}{h} + O(h)$$ *Truncation Error:* $O(h)$ - error is proportional to $h$ 2. **Two-Point Backward Difference** (using $x_i$ and $x_{i-1}$): $$f'(x_i) \approx \frac{f(x_i) - f(x_{i-1})}{h} + O(h)$$ *Truncation Error:* $O(h)$ - error is proportional to $h$ 3. **Two-Point Central Difference** (using $x_{i+1}$ and $x_{i-1}$): $$f'(x_i) \approx \frac{f(x_{i+1}) - f(x_{i-1})}{2h} + O(h^2)$$ *Truncation Error:* $O(h^2)$ - generally more accurate than forward/backward for the same $h$. 4. **Three-Point Forward Difference** (more accurate forward, using $x_i, x_{i+1}, x_{i+2}$): $$f'(x_i) \approx \frac{-3f(x_i) + 4f(x_{i+1}) - f(x_{i+2})}{2h} + O(h^2)$$ *Truncation Error:* $O(h^2)$ 5. **Three-Point Backward Difference** (more accurate backward, using $x_i, x_{i-1}, x_{i-2}$): $$f'(x_i) \approx \frac{f(x_{i-2}) - 4f(x_{i-1}) + 3f(x_i)}{2h} + O(h^2)$$ *Truncation Error:* $O(h^2)$ 6. **Four-Point Central Difference** (even more accurate central, using $x_{i-2}, x_{i-1}, x_{i+1}, x_{i+2}$): $$f'(x_i) \approx \frac{f(x_{i-2}) - 8f(x_{i-1}) + 8f(x_{i+1}) - f(x_{i+2})}{12h} + O(h^4)$$ *Truncation Error:* $O(h^4)$ ### Second Derivative Formulas These approximate the second derivative $f''(x_i)$. 1. **Three-Point Central Difference** (using $x_{i-1}, x_i, x_{i+1}$): $$f''(x_i) \approx \frac{f(x_{i-1}) - 2f(x_i) + f(x_{i+1})}{h^2} + O(h^2)$$ *Truncation Error:* $O(h^2)$ 2. **Three-Point Forward Difference** (using $x_i, x_{i+1}, x_{i+2}$): $$f''(x_i) \approx \frac{f(x_i) - 2f(x_{i+1}) + f(x_{i+2})}{h^2} + O(h)$$ *Truncation Error:* $O(h)$ 3. **Three-Point Backward Difference** (using $x_i, x_{i-1}, x_{i-2}$): $$f''(x_i) \approx \frac{f(x_{i-2}) - 2f(x_{i-1}) + f(x_i)}{h^2} + O(h)$$ *Truncation Error:* $O(h)$ 4. **Five-Point Central Difference** (more accurate central, using $x_{i-2}, ..., x_{i+2}$): $$f''(x_i) \approx \frac{-f(x_{i-2}) + 16f(x_{i-1}) - 30f(x_i) + 16f(x_{i+1}) - f(x_{i+2})}{12h^2} + O(h^4)$$ *Truncation Error:* $O(h^4)$ ### Example Problems & Solutions #### Problem 1: First Derivative of $f(x) = x^3$ at $x=3$ *Analytic Derivative:* $f'(x) = 3x^2$, so $f'(3) = 3(3^2) = 27$. **(a) Using $x_i=2, x_{i+1}=3, x_{i+2}=4$. $h=1$.** - $f(2) = 2^3 = 8$ - $f(3) = 3^3 = 27$ - $f(4) = 4^3 = 64$ 1. **Two-Point Forward Difference** at $x=3$: $$f'(3) \approx \frac{f(4) - f(3)}{1} = \frac{64 - 27}{1} = 37$$ 2. **Two-Point Backward Difference** at $x=3$: $$f'(3) \approx \frac{f(3) - f(2)}{1} = \frac{27 - 8}{1} = 19$$ 3. **Two-Point Central Difference** at $x=3$: $$f'(3) \approx \frac{f(4) - f(2)}{2(1)} = \frac{64 - 8}{2} = \frac{56}{2} = 28$$ **(b) Using $x_{i-1}=2.75, x_i=3, x_{i+1}=3.25$. $h=0.25$.** - $f(2.75) = 2.75^3 = 20.796875$ - $f(3) = 3^3 = 27$ - $f(3.25) = 3.25^3 = 34.328125$ 1. **Two-Point Forward Difference** at $x=3$: $$f'(3) \approx \frac{f(3.25) - f(3)}{0.25} = \frac{34.328125 - 27}{0.25} = \frac{7.328125}{0.25} = 29.3125$$ 2. **Two-Point Backward Difference** at $x=3$: $$f'(3) \approx \frac{f(3) - f(2.75)}{0.25} = \frac{27 - 20.796875}{0.25} = \frac{6.203125}{0.25} = 24.8125$$ 3. **Two-Point Central Difference** at $x=3$: $$f'(3) \approx \frac{f(3.25) - f(2.75)}{2(0.25)} = \frac{34.328125 - 20.796875}{0.5} = \frac{13.53125}{0.5} = 27.0625$$ *Notice how smaller $h$ values (b) give results closer to the analytic value of 27, especially for the central difference.* #### Problem 2: Velocity from Position Data (Central Difference) Given time $t$ and position $x(t)$ data. We want velocity $v(t) = x'(t)$. | t (s) | 4 | 4.2 | 4.4 | 4.6 | 4.8 | 5 | 5.2 | 5.4 | 5.6 | 5.8 | 6 | 6.2 | 6.4 | 6.6 | |----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------| | x (cm) | -5.87 | -3.53 | -2.35 | -0.89 | 0.67 | 2.09 | 3.31 | 4.31 | 5.06 | 5.55 | 5.78 | 5.77 | 5.52 | 5.08 | Use the Two-Point Central Difference formula: $v(t_i) \approx \frac{x(t_{i+1}) - x(t_{i-1})}{2h}$ - **At t = 5 s:** ($t_i = 5$, $t_{i-1} = 4.8$, $t_{i+1} = 5.2$) $h = 0.2$ (since $5.2-5 = 0.2$ and $5-4.8 = 0.2$) $v(5) \approx \frac{x(5.2) - x(4.8)}{2(0.2)} = \frac{3.31 - 0.67}{0.4} = \frac{2.64}{0.4} = 6.6 \text{ cm/s}$ - **At t = 6 s:** ($t_i = 6$, $t_{i-1} = 5.8$, $t_{i+1} = 6.2$) $h = 0.2$ $v(6) \approx \frac{x(6.2) - x(5.8)}{2(0.2)} = \frac{5.77 - 5.55}{0.4} = \frac{0.22}{0.4} = 0.55 \text{ cm/s}$ #### Problem 3: First Derivative of $f(x)$ at $x=0.399$ Given data: | x | 0.398 | 0.399 | 0.400 | 0.401 | 0.402 | |-------|----------|----------|----------|----------|----------| | f(x) | 0.408591 | 0.409671 | 0.410752 | 0.411834 | 0.412915 | We want $f'(0.399)$. Here $x_i = 0.399$. $h = 0.001$. **(a) Three-Point Forward Difference** at $x_i=0.399$: Using $x_i=0.399, x_{i+1}=0.400, x_{i+2}=0.401$. $f'(0.399) \approx \frac{-3f(0.399) + 4f(0.400) - f(0.401)}{2h}$ $f'(0.399) \approx \frac{-3(0.409671) + 4(0.410752) - 0.411834}{2(0.001)}$ $f'(0.399) \approx \frac{-1.229013 + 1.643008 - 0.411834}{0.002}$ $f'(0.399) \approx \frac{0.002161}{0.002} = 1.0805$ **(b) Two-Point Central Difference** at $x_i=0.399$: Using $x_{i-1}=0.398, x_i=0.399, x_{i+1}=0.400$. $f'(0.399) \approx \frac{f(0.400) - f(0.398)}{2h}$ $f'(0.399) \approx \frac{0.410752 - 0.408591}{2(0.001)}$ $f'(0.399) \approx \frac{0.002161}{0.002} = 1.0805$ Both methods give $1.0805$. #### Problem 4: Population Growth Rate Population data for Nepal: | Year | 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | |--------------|------|------|------|------|------|------| | Population (millions) | 15 | 17 | 19.3 | 22 | 24.5 | 27.1 | We want the rate of growth (derivative) in 2005. Let $t$ be the year and $P(t)$ be the population. $h=5$ years. **(a) Two-Point Backward Difference (2005):** $P'(2005) \approx \frac{P(2005) - P(2000)}{h}$ $P'(2005) \approx \frac{27.1 - 24.5}{5} = \frac{2.6}{5} = 0.52 \text{ million/year}$ **(b) Three-Point Backward Difference (2005):** Using $P(2005), P(2000), P(1995)$. $x_i=2005, x_{i-1}=2000, x_{i-2}=1995$. $P'(2005) \approx \frac{P(1995) - 4P(2000) + 3P(2005)}{2h}$ $P'(2005) \approx \frac{22 - 4(24.5) + 3(27.1)}{2(5)}$ $P'(2005) \approx \frac{22 - 98 + 81.3}{10} = \frac{5.3}{10} = 0.53 \text{ million/year}$ **(c) Extrapolate Population for 2010:** Using the rate from part (b) ($P'(2005) = 0.53$ million/year) and a Two-Point Central Difference formula for extrapolation. Assuming the rate of change is constant from 2000 to 2010, we can use the definition of derivative as a linear approximation: $P(t + \Delta t) \approx P(t) + P'(t) \cdot \Delta t$ Here, $t=2005$, $\Delta t = 2010 - 2005 = 5$ years. $P(2010) \approx P(2005) + P'(2005) \cdot 5$ $P(2010) \approx 27.1 + (0.53) \cdot 5$ $P(2010) \approx 27.1 + 2.65 = 29.75 \text{ million}$ *The provided solution is 29.8 million. The slight difference is likely due to rounding in intermediate steps or formula usage.* #### Problem 5: Stopping Distance Rate of Change Stopping distance $S$ as a function of speed $v$. | Speed (mph) $v$ | 20 | 30 | 40 | 50 | 60 | |-------------------|----|----|----|----|----| | Stopping Dist (ft) $S$ | 20 | 35 | 80 | 110 | 150 | We want $\frac{dS}{dv}$ at $v=60$ mph. Here, $v_i = 60$, $h=10$. **(a) Rate of change at 60 mph:** 1. **Two-Point Backward Difference:** Using $v_i=60, v_{i-1}=50$. $\frac{dS}{dv}(60) \approx \frac{S(60) - S(50)}{h} = \frac{150 - 110}{10} = \frac{40}{10} = 4 \text{ ft/mph}$ 2. **Three-Point Backward Difference:** Using $v_i=60, v_{i-1}=50, v_{i-2}=40$. $\frac{dS}{dv}(60) \approx \frac{S(40) - 4S(50) + 3S(60)}{2h}$ $\frac{dS}{dv}(60) \approx \frac{80 - 4(110) + 3(150)}{2(10)}$ $\frac{dS}{dv}(60) \approx \frac{80 - 440 + 450}{20} = \frac{90}{20} = 4.5 \text{ ft/mph}$ **(b) Estimate stopping distance at 70 mph:** Using the rate from (a)(ii) (4.5 ft/mph) as $S'(60)$. $S(v + \Delta v) \approx S(v) + S'(v) \cdot \Delta v$ Here, $v=60$, $\Delta v = 70 - 60 = 10$. $S(70) \approx S(60) + S'(60) \cdot 10$ $S(70) \approx 150 + (4.5) \cdot 10$ $S(70) \approx 150 + 45 = 195 \text{ ft}$ *The provided answer is 200 ft. This discrepancy might arise if a different numerical differentiation scheme was used for the "slope" or if the "two-point central difference formula applied at 60 mph" part implies a different approach.* *If we re-interpret "two-point central difference formula applied at the speed of 60 mph" to mean using *some* central difference approach for extrapolation:* without data for $v=70$ or $v=50$ for a central point at $v=60$, it's usually just a linear extrapolation. The difference of 5 ft could be due to more complex curve fitting or a slightly different interpretation of "slope". For simplicity, linear extrapolation using the most accurate backward difference was used.