Explain why the local discretization error of the finite difference method is proportional to the square of the step size. Also explain why the global discretization error is proportional to the step size itself.

Local discretization error in the finite difference method is like a snapshot of the inaccuracies at a specific step in the numerical approximation process. It stems from the fact that we're using a finite step size, denoted as 'h', to approximate a derivative, which in reality is defined as the limit when this step size approaches zero.

When we try to estimate the derivative of a function at a point using finite differences, we're essentially cutting off the Taylor series expansion of the function after a few terms. For instance, using a forward difference formula, we neglect higher-order derivatives beyond the first. Mathematically speaking, the local error is derived from the remainder of the Taylor series after truncation and, for a first-order approximation, it's proportional to the value of the second derivative of the function times the square of the step size, or \(\frac{1}{2}f''(x_0)h^2\).

Imagine you're taking small steps along a curve trying to follow its shape. If your steps are large, each stride will land a bit farther from the curve because you're cutting corners. Similarly, in finite differences, large step sizes mean we're missing out more on the true shape of the function, causing more significant local errors. The smaller the steps, the closer we can stick to the real path of the curve, and hence, local errors due to discretization shrink proportionally to the square of the step size, \(h^2\).

While local discretization error is like a close-up on the inaccuracy at each step, global discretization error is the 'big-picture' error after we've completed all steps in our numerical estimation. It accumulates all the local errors together to provide a measure of total error across the entire interval we're studying.

Considering a series of steps from an initial point 'a' to an ending point 'b', if we divide this interval into 'N' steps of equal size, the global error is a sum of all the local errors. If each local error is roughly \(\frac{1}{2}f''(x_0)h^2\), and we have N such errors, the combined effect is \(\frac{1}{2}f''(x_0)N h^2\). But since \N = \frac{b-a}{h}\, the h^2 in the local error and the division by h in the expression for N combine to leave us with a single h. So, unlike the quadratic decrease seen in local error with reducing step size, the global error diminishes linearly as \(h\) is reduced.

Picture a marathon runner tracking the route on a map while taking shortcuts. Each shortcut represents a local error, and even though each is small, their total effect (global error) is significant enough to misjudge the distance covered. By making the steps (shortcuts) smaller, the runner's path becomes more accurate, just as the global error decreases with a smaller step size in the finite difference method.

The Taylor series expansion is a powerful mathematical tool that lets us express any differentiable function as an infinite sum of its derivatives at a certain point, each multiplied by a power of the displacement from that point.

Here's how the Taylor series expansion looks for a function \(f(x)\) around a point \(x_0\):
\[f(x) = f(x_0) + f'(x_0)(x - x_0) + \frac{1}{2}f''(x_0)(x - x_0)^2 + \frac{1}{3!}f'''(x_0)(x - x_0)^3 + ...\]
It's like unfolding the function into a linear combination of its slopes, curvatures, and higher-order shapes at \(x_0\), weighted by how far away \(x\) is from \(x_0\). For the finite difference method, this series allows us to approximate derivatives by considering only the first few terms and dropping the rest, which results in a discretization error.

Essentially, the finite difference method uses the Taylor series to create a formula that approximates derivatives by linear combinations of function values at adjacent points. It's like approximating a curved path with straight line segments—the more segments (higher-order terms) you use, the closer the approximation to the curve.

In the world of calculus, derivatives capture the rate at which a function changes. Numerically approximating derivatives is essential in computational science because we often require these rates of change but lack an analytical expression for the function or the derivative is too complex to work with directly.

The finite difference method provides a way to approximate derivatives by using function values at discrete points. For example, the forward difference formula \(f'(x_0) \approx \frac{f(x_0 + h) - f(x_0)}{h}\) approximates the slope of \(f\) at \(x_0\) by considering how \(f\) changes over a small 'h' interval.

However, approximating derivatives numerically comes with a trade-off. Smaller step sizes lead to more accurate results but can increase computational cost and are more prone to rounding errors. Bigger step sizes reduce the number of computations and rounding-related problems but can give a less accurate result. This trade-off is a key aspect of numerical analysis, requiring careful balancing based on the context and the desired accuracy of the problem at hand.