Skip to article frontmatterSkip to article content

A peek at numerical methods for diffusion models

University at Albany (SUNY)

This notebook is part of The Climate Laboratory by Brian E. J. Rose, University at Albany.


1. The one-dimensional diffusion equation


Suppose that a quantity u(x)u(x) is mixed down-gradient by a diffusive process.

The diffusive flux is

F=KuxF = - K \frac{\partial u}{\partial x}

There will be local changes in uu wherever this flux is convergent or divergent:

ut=Fx\frac{\partial u}{\partial t} = - \frac{\partial F}{\partial x}

Putting this together gives the classical diffusion equation in one dimension

ut=x(Kux)\frac{\partial u}{\partial t} = \frac{\partial}{\partial x} \left( K \frac{\partial u}{\partial x} \right)

For simplicity, we are going to limit ourselves to Cartesian geometry rather than meridional diffusion on a sphere.

We will also assume here that KK is a constant, so our governing equation is

ut=K2ux2\frac{\partial u}{\partial t} = K \frac{\partial^2 u}{\partial x^2}

This equation represents a time-dependent diffusion process. It is an initial-boundary value problem. We want to integrate the model forward in time to model the changes in the field u(x)u(x).


2. Discretizing the diffusion operator in space


Solving a differential equation on a computer always requires some approximation to represent the continuous function u(x,t)u(x,t) and its derivatives in terms of discrete quantities (arrays of numbers).

We have already dealt with simple discretization of the time derivative back in the first notes on solving the simplest EBM. We used the forward Euler method to step all our of radiation models forward in time so far.

Some notation for discretization of u(x,t)u(x,t)

We will discretize time and space on grids

xj,   tnx_j , ~~~ t^n

so that

ujn=u(xj, tn)u_j^n = u(x_j, ~t^n)

Discretizing the diffusive flux

The governing equation can be written in terms of the convergence of the diffusive flux:

ut=Fx\frac{\partial u}{\partial t} = - \frac{\partial F}{\partial x}

It is sensible to use a centered difference to approximate this derivative:

FxjFj+12Fj12xj+12xj12\frac{\partial F}{\partial x} \bigg|_j \approx \frac{F_{j+\frac{1}{2}} - F_{j-\frac{1}{2}}}{x_{j+\frac{1}{2}} - x_{j-\frac{1}{2}}}

The time tendency at point xjx_j can thus be written

utjFj+12Fj12xj+12xj12\frac{\partial u}{\partial t} \bigg|_j \approx - \frac{F_{j+\frac{1}{2}} - F_{j-\frac{1}{2}}}{x_{j+\frac{1}{2}} - x_{j-\frac{1}{2}}}

The flux itself depends on a spatial derivative of uu. We will apply the same centered difference approximation. At point xjx_j this would look like

uxuj+12uj12xj+12xj12\frac{\partial u}{\partial x} \approx \frac{u_{j+\frac{1}{2}} - u_{j-\frac{1}{2}}}{x_{j+\frac{1}{2}} - x_{j-\frac{1}{2}}}

But we actually want to approximate Fj+12F_{j+\frac{1}{2}} and Fj12F_{j-\frac{1}{2}}, so we apply the centered difference formula at these intermediate points to get

Fj+12Kuj+1ujxj+1xjF_{j+\frac{1}{2}} \approx -K \frac{u_{j+1} - u_{j}}{x_{j+1} - x_{j}}

and

Fj12Kujuj1xjxj1F_{j-\frac{1}{2}} \approx -K \frac{u_{j} - u_{j-1}}{x_{j} - x_{j-1}}

Putting this all together, we can write the time tendency at xjx_j as

utjKuj+1ujxj+1xjujuj1xjxj1xj+12xj12\frac{\partial u}{\partial t} \bigg|_j \approx K \frac{ \frac{u_{j+1} - u_{j}}{x_{j+1} - x_{j}} - \frac{u_{j} - u_{j-1}}{x_{j} - x_{j-1}}}{x_{j+\frac{1}{2}} - x_{j-\frac{1}{2}}}

We’ll make things easy on ourselves by using uniform grid spacing in xx, so

xj+1xj=xjxj1=xj+12xj12=Δxx_{j+1} - x_{j} = x_{j} - x_{j-1} = x_{j+\frac{1}{2}} - x_{j-\frac{1}{2}} = \Delta x

So our final formula for the diffusive flux convergence is

utjKuj+12uj+uj1Δx2\frac{\partial u}{\partial t} \bigg|_j \approx K \frac{ u_{j+1} - 2 u_{j} + u_{j-1}}{\Delta x^2}

No-flux boundary conditions

Suppose the domain is 0x10 \le x \le 1, with solid walls at x=0,1x=0, 1.

The physical boundary condition at the walls is that there can be no flux in or out of the walls:

F(0)=F(1)=0F(0) = F(1) = 0

So the boundary conditions on uu are

ux=0   at   x=0,1\frac{\partial u}{\partial x} = 0 ~~~ \text{at} ~~~ x=0,1

The staggered grid

Suppose we have a grid of J+1J+1 total points between x=0x=0 and x=1x=1, including the boundaries:

  • x0=0x^*_0 = 0
  • x1=Δxx^*_1 = \Delta x
  • x2=2 Δxx^*_2 = 2~\Delta x
  • ...
  • xj=j Δxx^*_j = j~\Delta x
  • ...
  • xJ1=(J1) Δx=1Δxx^*_{J-1} = (J-1)~\Delta x = 1 - \Delta x
  • xJ=J Δx=1x^*_J = J ~ \Delta x = 1

Clearly then the grid spacing must be Δx=1/J\Delta x = 1/J.

We’ll define the fluxes on this grid. The boundary conditions can thus be written

F0=FJ=0F_0 = F_J = 0

Since our centered difference discretization defines FF at points halfway between the uu points, it is sensible to locate uu on another grid that is offset by Δx/2\Delta x / 2.

The first grid point for uu is thus a distance Δx/2\Delta x / 2 from the wall, and there are a total of JJ points:

  • x0=Δx/2x_0 = \Delta x / 2
  • x1=Δx/2+Δxx_1 = \Delta x / 2 + \Delta x
  • x2=Δx/2+2 Δxx_2 = \Delta x / 2 + 2~\Delta x
  • ...
  • xj=Δx/2+j Δxx_j = \Delta x / 2 + j~\Delta x
  • ...
  • xJ1=Δx/2+(J1) Δx=1Δx/2x_{J-1} = \Delta x / 2 + (J-1)~\Delta x = 1 - \Delta x / 2

Implementing the boundary condition on the staggered grid

At x0x_0 we have

ut0F1F0Δx\frac{\partial u}{\partial t} \bigg|_0 \approx -\frac{ F_1 - F_0}{\Delta x}

Subbing in F0=0F_0 = 0 and the normal discretization for F1F_1 gives

ut0Ku1u0Δx2\frac{\partial u}{\partial t} \bigg|_0 \approx K \frac{ u_1 - u_0 }{\Delta x^2}

The same procedure at the other wall yields

utJ1KuJ1uJ2Δx2\frac{\partial u}{\partial t} \bigg|_{J-1} \approx - K \frac{ u_{J-1} - u_{J-2} }{\Delta x^2}

Pulling this all together we have a complete discretization of the diffusion operator including its boundary conditions:

ut0Ku1u0Δx2\frac{\partial u}{\partial t} \bigg|_0 \approx K \frac{ u_1 - u_0 }{\Delta x^2}
utjKuj+12uj+uj1Δx2,      j=1,...,J2\frac{\partial u}{\partial t} \bigg|_j \approx K \frac{ u_{j+1} - 2 u_{j} + u_{j-1}}{\Delta x^2}, ~~~~~~ j=1,...,J-2
utJ1KuJ1uJ2Δx2\frac{\partial u}{\partial t} \bigg|_{J-1} \approx - K \frac{ u_{J-1} - u_{J-2} }{\Delta x^2}

3. Coding the discretized diffusion operator in numpy


%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from IPython.display import display, Math, Latex

Here we will divide our domain up into 20 grid points.

J1 = 20
J = J1
deltax = 1./J
display(Math(r'J = %i' %J))
display(Math(r'\Delta x = %0.3f' %deltax))
Loading...

The fluxes will be solved on the staggered grid with 21 points.

uu will be solved on the 20 point grid.

xstag = np.linspace(0., 1., J+1)
x = xstag[:-1] + deltax/2
print( x)
[0.025 0.075 0.125 0.175 0.225 0.275 0.325 0.375 0.425 0.475 0.525 0.575
 0.625 0.675 0.725 0.775 0.825 0.875 0.925 0.975]
u = np.zeros_like(x)

Here’s one way to implement the finite difference, using array indexing.

dudx = (u[1:] - u[:-1]) / (x[1:] - x[:-1])
dudx.shape
(19,)

We can also use the function numpy.diff() to accomplish the same thing:

help(np.diff)
Help on _ArrayFunctionDispatcher in module numpy:

diff(a, n=1, axis=-1, prepend=<no value>, append=<no value>)
    Calculate the n-th discrete difference along the given axis.

    The first difference is given by ``out[i] = a[i+1] - a[i]`` along
    the given axis, higher differences are calculated by using `diff`
    recursively.

    Parameters
    ----------
    a : array_like
        Input array
    n : int, optional
        The number of times values are differenced. If zero, the input
        is returned as-is.
    axis : int, optional
        The axis along which the difference is taken, default is the
        last axis.
    prepend, append : array_like, optional
        Values to prepend or append to `a` along axis prior to
        performing the difference.  Scalar values are expanded to
        arrays with length 1 in the direction of axis and the shape
        of the input array in along all other axes.  Otherwise the
        dimension and shape must match `a` except along axis.

        .. versionadded:: 1.16.0

    Returns
    -------
    diff : ndarray
        The n-th differences. The shape of the output is the same as `a`
        except along `axis` where the dimension is smaller by `n`. The
        type of the output is the same as the type of the difference
        between any two elements of `a`. This is the same as the type of
        `a` in most cases. A notable exception is `datetime64`, which
        results in a `timedelta64` output array.

    See Also
    --------
    gradient, ediff1d, cumsum

    Notes
    -----
    Type is preserved for boolean arrays, so the result will contain
    `False` when consecutive elements are the same and `True` when they
    differ.

    For unsigned integer arrays, the results will also be unsigned. This
    should not be surprising, as the result is consistent with
    calculating the difference directly:

    >>> u8_arr = np.array([1, 0], dtype=np.uint8)
    >>> np.diff(u8_arr)
    array([255], dtype=uint8)
    >>> u8_arr[1,...] - u8_arr[0,...]
    255

    If this is not desirable, then the array should be cast to a larger
    integer type first:

    >>> i16_arr = u8_arr.astype(np.int16)
    >>> np.diff(i16_arr)
    array([-1], dtype=int16)

    Examples
    --------
    >>> import numpy as np
    >>> x = np.array([1, 2, 4, 7, 0])
    >>> np.diff(x)
    array([ 1,  2,  3, -7])
    >>> np.diff(x, n=2)
    array([  1,   1, -10])

    >>> x = np.array([[1, 3, 6, 10], [0, 5, 6, 8]])
    >>> np.diff(x)
    array([[2, 3, 4],
           [5, 1, 2]])
    >>> np.diff(x, axis=0)
    array([[-1,  2,  0, -2]])

    >>> x = np.arange('1066-10-13', '1066-10-16', dtype=np.datetime64)
    >>> np.diff(x)
    array([1, 1], dtype='timedelta64[D]')

np.diff(u).shape
(19,)

Here is a function that computes the diffusive flux FF on the staggered grid, including the boundaries.

def diffusive_flux(u, deltax, K=1):
    #  Take the finite difference
    F = np.diff(u)/deltax
    #  add a zero as the first element (no flux on boundary)
    F = np.insert(F, 0, 0.)
    #  add another zero as the last element (no flux on boundary)
    F = np.append(F, 0.)
    #  flux is DOWN gradient, proportional to D
    return -K*F
diffusive_flux(u,deltax).shape
(21,)

The time tendency of uu is just the convergence of this flux, which requires one more finite difference:

def diffusion(u, deltax, K=1):
    #  compute flux
    F = diffusive_flux(u, deltax, K)
    #  take convergence of flux
    return -np.diff(F) / deltax

A smooth example

Suppose we have an initial uu field that has a local maximum in the interior.

The gaussian (bell curve) function is a convenient way to create such a field.

def gaussian(x, mean, std):
    return np.exp(-(x-mean)**2/(2*std**2))/np.sqrt(2*np.pi*std**2)
K = 0.01
u = gaussian(x, 0.5, 0.08)
dudt = diffusion(u, deltax, K=K)
fig, ax = plt.subplots(1)
ax.plot(x, u, label='$u(x)$')
ax.plot(x, dudt, label='$du/dt$')
ax.legend()
<Figure size 640x480 with 1 Axes>

Hopefully this makes sense. The diffusion is acting to smooth out uu by reducing the peak and increasing uu on the flanks of the gaussian bump.

Some non-smooth examples

Use a random number generator to create some noisy initial conditions.

fig = plt.figure(figsize=(10,8))
for n in range(4):
    u = np.random.random(J)
    dudt = diffusion(u, deltax, K)
    ax = fig.add_subplot(2,2,n+1)
    ax.plot(x, u)
    ax.plot(x, dudt)
<Figure size 1000x800 with 4 Axes>

4. Discretizing the time derivative


The simplest way to discretize the time derivative is the forward Euler method:

dudtnun+1unΔt\frac{d u}{dt} \bigg|^n \approx \frac{u^{n+1} - u^n}{\Delta t}

We have already used this method to step our prognostic variables forward in time.

Solving the above for the future value of uu gives

un+1=un+Δtdudtnu^{n+1} = u^n + \Delta t \frac{d u}{dt} \bigg|^n

We apply our discretization of the diffusion operator to the current value of the field ujnu^n_j, to get our formula for the future values:

ujn+1=ujn+KΔtΔx2(uj+1n2ujn+uj1n)u_j^{n+1} = u_j^n + \frac{K \Delta t}{\Delta x^2} \left( u^n_{j+1} - 2 u^n_{j} + u^n_{j-1} \right)

(except at the boundaries, where the diffusion operator is slightly different -- see above).

Together, this scheme is known as Forward Time, Centered Space or FTCS.

It is very simple to implement in numpy code.

def step_forward(u, deltax, deltat, K=1):
    dudt = diffusion(u, deltax, K)
    return u + deltat * dudt
K = 0.01
deltat = 0.125
deltat1 = deltat

u0 = gaussian(x, 0.5, 0.08)
u1 = step_forward(u0, deltax, deltat1, K)
fig, ax = plt.subplots(1)
ax.plot(x, u0, label='initial')
ax.plot(x, u1, label='next')
ax.legend()
<Figure size 640x480 with 1 Axes>

Let’s loop through a number of timesteps.

#  regular resolution
J = 20
deltax = 1./J
xstag = np.linspace(0., 1., J+1)
x = xstag[:-1] + deltax/2
u = gaussian(x, 0.5, 0.08)
niter = 11
for n in range(niter):
    u = step_forward(u, deltax, deltat1, K)
    plt.plot(x, u, label=n)
plt.legend()
<Figure size 640x480 with 1 Axes>

The numerics were easy to implement, and the scheme seems to work very well! The results are physically sensible.

Now, suppose that you wanted to double the spatial resolution

Try setting J=40J=40 and repeat the above procedure.

What happens?

#  double the resolution
scaling_factor = 2
J = J1 * scaling_factor
deltax = 1./J
xstag = np.linspace(0., 1., J+1)
x = xstag[:-1] + deltax/2
u = gaussian(x, 0.5, 0.08)
for n in range(niter):
    u = step_forward(u, deltax, deltat1, K)
    plt.plot(x, u, label=n)
plt.legend()
<Figure size 640x480 with 1 Axes>

Suddenly our scheme is producing numerical noise that grows in time and overwhelms to smooth physical solution we are trying to model.

What went wrong, and what can we do about it?


5. Stability analysis of the FTCS scheme


Following Press et al. (1988), Numerical Recipes in C: The Art of Scientific Computing.

This is an example of the so-called von Neumann Stability Analysis. It is a form of normal mode analysis for a discrete system.

We look for normal mode solutions (i.e. wavy sines and cosines) of the finite difference equations of the form

ujn=ξnexp(i k j Δx)u_j^n = \xi^n \exp(i~k~j~ \Delta x)

where kk is some real number that represents a spatial wavenumber (which can have any value), and ξ=ξ(k)\xi = \xi(k) is a complex number that depends on kk.

The number ξ\xi is called the amplification factor at a given wavenumber kk.

The question is, under what conditions do wavy solutions grow with time? (This is bad, as it means small numerical noise will become large numerical noise and make our differencing scheme unusable)

Let’s substitute the normal mode solution into our finite difference equation

ujn+1ujnΔt=KΔx2(uj+1n2ujn+uj1n)\frac{u_j^{n+1} - u_j^n}{\Delta t} = \frac{K}{\Delta x^2} \left( u^n_{j+1} - 2 u^n_{j} + u^n_{j-1} \right)
ξn+1exp(i k j Δx)ξnexp(i k j Δx)Δt=KΔx2(ξnexp(i k (j+1) Δx)2ξnexp(i k j Δx)+ξnexp(i k (j1) Δx))\frac{\xi^{n+1} \exp(i~k~j~ \Delta x) - \xi^n \exp(i~k~j~ \Delta x)}{\Delta t} = \frac{K}{\Delta x^2} \left( \xi^n \exp(i~k~(j+1)~ \Delta x) - 2 \xi^n \exp(i~k~j~ \Delta x) + \xi^n \exp(i~k~(j-1)~ \Delta x) \right)

Divide through by ξnexp(i k j Δx)\xi^n \exp(i~k~j~\Delta x):

ξn+1ξn1=KΔtΔx2(exp(i k Δx)2+exp(i k Δx))\frac{\xi^{n+1}}{\xi^n} - 1 = \frac{K \Delta t}{\Delta x^2} \left(\exp(i~k~\Delta x) - 2 + \exp(-i~k~\Delta x) \right)

The exponentials simplify

ξn+1ξn=1+KΔtΔx2(2cos(k Δx)2)\frac{\xi^{n+1}}{\xi^n} = 1 + \frac{K \Delta t}{\Delta x^2} \left(2 \cos(k~\Delta x) - 2 \right)

Or using a double angle identity,

ξn+1ξn=14KΔtΔx2sin2(k Δx2)\frac{\xi^{n+1}}{\xi^n} = 1 - \frac{4 K \Delta t}{\Delta x^2} \sin^2 \left( \frac{k~\Delta x}{2} \right)

The wavy solution must not grow with time

We need to prevent growing normal modes. So successive amplitudes should be

ξn+1ξn1\bigg| \frac{\xi^{n+1}}{\xi^n} \bigg| \le 1

The stability condition is thus

14KΔtΔx2sin2(k Δx2)1\bigg| 1 - \frac{4 K \Delta t}{\Delta x^2} \sin^2 \left( \frac{k~\Delta x}{2} \right) \bigg| \le 1

and this condition must be met for EVERY possible wavenumber kk.

Because 0sin2(ϕ)10 \le \sin^2(\phi) \le 1 for any ϕ\phi, our condition can only be violated if

4KΔtΔx2>2\frac{4 K \Delta t}{\Delta x^2} > 2

We conclude the the FTCS scheme is stable so long as this stability condition is met:

ΔtΔx22K\Delta t \le \frac{\Delta x^2}{2 K}

We have just discovered an important constraint on the allowable timestep

The maximum timestep we can use with the FTCS scheme for the diffusion equation is proportional to Δx2\Delta x^2.

A doubling of the spatial resolution would require a 4x shorter timestep to preserve numerical stability.

Physically, the restriction is that the maximum allowable timestep is approximately the diffusion time across a grid cell of width Δx\Delta x.


6. Numerical tests with a shorter timestep


Going back to our Gaussian example, let’s double the resolution but shorten the timestep by a factor of 4.

#  double the resolution
J = J1 * scaling_factor
deltax = 1./J
xstag = np.linspace(0., 1., J+1)
x = xstag[:-1] + deltax/2
K = 0.01
#  The maximum stable timestep
deltat_max = deltax**2 / 2 / K
print( 'The maximum allowable timestep is %f' %deltat_max)

deltat = deltat1 / scaling_factor**2
print( '4x the previous timestep is %f' %deltat)
The maximum allowable timestep is 0.031250
4x the previous timestep is 0.031250
u = gaussian(x, 0.5, 0.08)
for n in range(niter):
    for t in range(scaling_factor**2):
        u = step_forward(u, deltax, deltat, K)
    plt.plot(x, u, label=n)
plt.legend()
<Figure size 640x480 with 1 Axes>

Success! The graph now looks like a smoother (higher resolution) version of our first integration with the coarser grid.

But at a big cost: our calculation required 4 times more timesteps to do the same integration.

The total increase in computational cost was actally a factor of 8 to get a factor of 2 increase in spatial resolution.


7. The need for a more efficient method


In practice the condition

ΔtΔx22K\Delta t \le \frac{\Delta x^2}{2 K}

is often too restrictive to be practical!

Consider our diffusive EBM. Suppose we want a spatial resolution of 1º latitude. Then we have 180 grid points from pole to pole, and our physical length scale is

Δx105m\Delta x \approx 10^5 \text{m}

We were using a diffusivity of D=0.6 W m2 K1D = 0.6 ~ \text{W m}^{-2}~\text{K}^{-1} and a heat capacity of C=4×107 J m2 K1C = 4 \times 10^7 ~ \text{J m}^{-2} ~\text{K}^{-1} (for 10 m of water, see Lecture 17).

Accounting for the spherical geometry in our EBM, this translates to

K=2πa2DC=2π (6.4×106 m)2 (0.6 W m2 K1)4×107 J m2 K14×106 m2 s1K = \frac{2 \pi a^2 D}{C} = \frac{2 \pi ~ (6.4 \times 10^6 ~\text{m})^2 ~(0.6 ~ \text{W m}^{-2}~\text{K}^{-1})}{4 \times 10^7 ~ \text{J m}^{-2} ~\text{K}^{-1}} \approx 4 \times 10^{6} ~ \text{m}^2 ~ \text{s}^{-1}

Recall that this is the diffusivity associated with the large-scale motion of the atmosphere (mostly). If we take a typical velocity scale for a mid-latitude eddy, V20 m s1V \approx 20~\text{m s}^{-1}, and a typical length scale for that eddy, L2000 kmL \approx 2000~\text{km}, the diffusivity then scales as

K=V L=4×106 m2 s1K = V~ L = 4 \times 10^{6} ~ \text{m}^2 ~ \text{s}^{-1}

Using these numbers the stability condition is roughly

Δt103 s\Delta t \le 10^3 ~\text{s}

which is less than one hour!

And if we wanted to double the resolution to 0.5º, we would need a timestep of just a few minutes.

This can be a very onerous requirement for a model that would like to integrate out for many years. We can do better, but we need a different time discretization!


8. Implicit time method


With numerical methods for partial differential equations, it often turns out that a small change in the discretization can make an enormous difference in the results.

The implicit time scheme applies exactly the same centered difference scheme to the spatial derivatives in the diffusion operator.

But instead of applying the operator to the field unu^n at time nn, we instead apply it to the field at the future time un+1u^{n+1}.

The scheme looks like

ujn+1ujnΔt=KΔx2(uj+1n+12ujn+1+uj1n+1)\frac{u_j^{n+1} - u_j^n}{\Delta t} = \frac{K}{\Delta x^2} \left( u^{n+1}_{j+1} - 2 u^{n+1}_{j} + u^{n+1}_{j-1} \right)

in the interior, and at the boundaries:

u0n+1u0nΔt=KΔx2(u1n+1u0n+1)\frac{u_0^{n+1} - u_0^n}{\Delta t} = \frac{K}{\Delta x^2} \left( u^{n+1}_1 - u^{n+1}_0 \right)

and

uJ1n+1uJ1nΔt=KΔx2(uJ1n+1uJ2n+1)\frac{u_{J-1}^{n+1} - u_{J-1}^n}{\Delta t} = - \frac{K}{\Delta x^2} \left( u_{J-1}^{n+1} - u_{J-2}^{n+1} \right)

This might seem like a strange way to write the system, since we don’t know the future state of the system at tn+1t^{n+1}. That’s what we’re trying to solve for!

Let’s move all terms evaluated at tn+1t^{n+1} to the left hand side:

ujn+1KΔtΔx2(uj+1n+12ujn+1+uj1n+1)=ujnu_j^{n+1} - \frac{K \Delta t}{\Delta x^2} \left( u^{n+1}_{j+1} - 2 u^{n+1}_{j} + u^{n+1}_{j-1} \right) = u_j^n

or

Kuj+1n+1+(1+2K)ujn+1Kuj1n+1=ujn-K^* u^{n+1}_{j+1} + \left(1+2K^* \right) u_j^{n+1} - K^* u_{j-1}^{n+1} = u_j^n

(in the interior)

where we have introduced a non-dimensional diffusivity

K=KΔtΔx2K^* = \frac{K \Delta t}{\Delta x^2}

The implicit scheme as a matrix problem

We can write this as a matrix equation

A Un+1=Un\mathbf{A} ~ \mathbf{U}^{n+1} = \mathbf{U}^n

where U\mathbf{U} is a J×1J\times1 column vector giving the field u(x)u(x) at a particular instant in time:

Un=[u0nu1nu2n...uJ2nuJ1n]\mathbf{U}^n = \left[ \begin{array}{c} u^n_0 \\ u^n_1 \\ u^n_2 \\ ... \\ u^n_{J-2} \\ u^n_{J-1} \\ \end{array} \right]

and Un+1\mathbf{U}^{n+1} is the same vector at tn+1t^{n+1}.

A\mathbf{A} is a J×JJ\times J tridiagonal matrix:

A=[1+KK00...000K1+2KK0...0000K1+2KK...000........................0000...K1+2KK0000...0K1+K]\mathbf{A} = \left[ \begin{array}{cccccccc} 1+K^* & -K^* & 0 & 0 & ... & 0 & 0 & 0 \\ -K^* & 1+2K^* & -K^* & 0 & ... & 0 & 0 & 0 \\ 0 & -K^* & 1+2K^* & -K^* &... & 0 & 0 & 0 \\ ... & ... & ... & ... & ... & ... & ... & ... \\ 0 & 0 & 0 & 0 & ... & -K^* & 1+2K^* & -K^* \\ 0 & 0 & 0 & 0 & ... & 0 & -K^* & 1+K^* \\ \end{array} \right]

Solving for the future state of the system Un+1\mathbf{U}^{n+1} is then just the solution of the linear system

Un+1=A1Un\mathbf{U}^{n+1} = \mathbf{A}^{-1} \mathbf{U}^{n}

Solving a tridiagonal matrix problem like this is a very common operation in computer science, and efficient numerical routines are available in many languages (including Python / numpy!)

Stability analysis of the implicit scheme

We’ll skip the details, but the amplification factor for this scheme is (see Numerical Recipes book or other text on numerical methods):

ξn+1ξn=11+4Ksin2(kΔx2)\frac{\xi^{n+1}}{\xi^n} = \frac{1}{1+4 K^* \sin^2 \left( \frac{k \Delta x}{2} \right) }

so the stability criterion of

ξn+1ξn1\bigg| \frac{\xi^{n+1}}{\xi^n} \bigg| \le 1

is met for any value of KK^* and thus for any timestep Δt\Delta t.

The implicit method (also called backward time) is unconditionally stable for any choice of timestep.


9. Your homework assignment


Write Python code to solve the diffusion equation using this implicit time method. Demonstrate that it is numerically stable for much larger timesteps than we were able to use with the forward-time method. One way to do this is to use a much higher spatial resolution.

Some final thoughts:

We have just scratched the surface of the wonders and sorrows of numerical methods here. The implicit method is very stable but is not the most accurate method for a diffusion problem, particularly when you are interested in some of the faster dynamics of the system (as opposed to just getting the system quickly to its equilibrium state).

There are always trade-offs in the choice of a numerical method.

The equations for most climate models are sufficiently complex that more than one numerical method is necessary. Even in the simple diffusive EBM, the radiation terms are handled by a forward-time method while the diffusion term is solved implicitly.

Once you have worked through the above problem (diffusion only), you might want to look in the climlab code to see how the diffusion solver is implemented there, and how it is used when you integrate the EBM.


Credits

This notebook is part of The Climate Laboratory, an open-source textbook developed and maintained by Brian E. J. Rose, University at Albany.

It is licensed for free and open consumption under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Development of these notes and the climlab software is partially supported by the National Science Foundation under award AGS-1455071 to Brian Rose. Any opinions, findings, conclusions or recommendations expressed here are mine and do not necessarily reflect the views of the National Science Foundation.


References
  1. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1988). Numerical Recipes in C. Cambridge University Press.