Our constrained iterative deconvolution procedure works by making a guess as to what the object looks like, blurring that guess in such a way as to simulate the blurring caused by the microscope's limited aperture, and then comparing the blurred guess to the actual image. The difference between them (or the ratio of blurred guess to image) is then used to modify the guess. The modified guess is then constrained to be non-negative, by setting any pixel with negative intensity to zero (this is analogous to solvent-flattening in crystallography). This procedure is repeated over and over and eventually the guess converges to a close approximation of the actual image.
The deconvolution package has a command line interface and a graphical front end. When using either, you'll need to select an appropriate model for how the microscope blurs the image (see Selecting the CTF or pupil function) and the method by which the deconvolution calculates the guess of the unblurred image (see Method). There are a number of parameters that can be used to tweak the calculation (Parameters has more information), but in practice the defaults are usually a good choice.
Certain operations, such as selecting a region to process and starting processing from the graphical interface, are described in BatchRegion.html since several applications use the same mechanism for those operations.
Overview | Selecting the CTF or pupil function | Measuring the CTF or pupil function | CTF and pupil function formats | Method | Parameters
Batch processing | Raw data correction | Priism
In order for deconvolution to correct blur, we must have a description of how the microscope blurs images. The point spread function is one such description. The deconvolution application does not use the point spread function directly; instead, it accepts a description of either the contrast transfer function (CTF) or the pupil function both of which can be derived from the point spread function.
The point spread function (PSF) is the image that a point source of light makes in the microscope. This is used to correct for blurring caused by the scope. Usually the PSF will not be symmetric in z, so you have an axially asymmetric PSF; however, it is possible to adjust the refractive index of the immersion oil to give an axially symmetric PSF.
The transfer function for the microscope is the Fourier transform of the point spread function. In this documentation, we frequently use the terms optical transfer function (OTF) and contrast transfer function as as synonyms for the transfer function of the microscope. Others are more accurate with their terminology: using optical transfer function to refer to the component of the net transfer function that arises from the optics of the microscope (excluding the contributions from the CCD, for instance) and using contrast transfer function to refer to the squared modulus of the intensity of an image of a square-wave pattern.
A pupil function describes the optics of the microscope. Using a process called phase retrieval and assumptions about the other contributions to the point spread function that are not part of the optics (namely the CCD), it is possible to calculate a pupil function from the point spread function.
From the graphical interface, the CTF or pupil function is selected by clicking on the OTFfile button, and then using the file browser that appears to select the file containing the CTF or pupil function. You can also enter the file name directly in the adjacent field. When you select an input data set (using the InFile button or entering the name directly), a CTF or pupil function will automatically be chosen based on the lens number recorded in the data set's header. The mapping between lens number and CTF is defined by the CTF/CTF.def file in the Priism distribution (or on a per-user basis by a .iveprefs/CTF.def file in the use's home directory); the default mapping is to one of two axially and radially symmetric CTFs for the om1 system in John Sedat's lab at UCSF.
From the command line, the name of the CTF or pupil function file is the third argument.
Several sample radially-symmetric CTFs are available in the CTF subdirectory of the Priism distribution. In the names for those files, lens12 refers to the 60x lens on John Sedat's om1, and lens13 is the 100x lens on that system. lens12.realctf and lens13.realctf represent axially- and radially-symmetric PSFs which assume that you have adjusted the immersion oil to give a symmetric PSF. If you have not adjusted the immersion oil, you should use an asymmetric PSF to obtain an optimal result from the deconvolution. All asymmetric CTFs have names of the form lens12_<refractive index>_asym.ctf.
To decide which asymmetric CTF to use, first examine the data to see if the out-of-focus blur is mostly thrown below (lower z) the object or above it (higher z). If most of the blur is thrown below, it means that the oil you used had too low of a refractive index. The asymmetric CTFs were recorded under conditions where the oil that gave the symmetric CTF had a refractive index of 1.5140 so you should use one of CTFs made with an immersion oil whose refractive index is less than 1.5140. It generally does not matter a whole lot which one you use, as long as it is a refractive index lower than the one used in the symmetric case. The same discussion applies if your oil had too high of a refractive index.
Overview | Selecting the CTF or pupil function | Measuring the CTF or pupil function | CTF or pupil function format | Method | Parameters
If you do not already have the CTF or pupil function for your imaging system, the recommended method for obtaining one is to measure it experimentally. I'll outline a procedure for measuring the CTF or pupil function with an emphasis on the computational aspects or those peculiar to Priism's deconvolution application. For a fuller consideration of the process, consult the literature; two references are:
To generate a 3D CTF for the deconvolution application,
The steps for generating a pupil function are similar, but instead of the 3D Fourier transform step and the optional radial averaging, do the following:
Overview | Selecting the CTF or pupil function | Measuring the CTF or pupil function | CTF or pupil function formats | Method | Parameters
The format for CTF files read by the deconvolution application is the standard Priism image file format (consult the IM library documentation for details). If the CTF is one where radial symmetry has been imposed, the file also has the following special attributes:
For a full 2D or 3D CTF (no assumption of radial symmetry), the file has the following special attributes:
For either radially-symmetric or non-radially symmetric CTF files, the emission wavelength, in nanometers, should be specified in the header so that the deconvolution application can modify the CTF as necessary if there is a wavelength mismatch with the data to be deconvolved.
The format for pupil function images is the standard Priism file format. The file also has the following special attributes:
Like CTF files, pupil function images should include the emission wavelength value or values, in nanometers, in the header so that the deconvolution application can modify the pupil function as necessary if there is a wavelength mismatch with the data to be deconvolved.
The files used to store the result of fitting Zernike polynomials to the pupil function image are plain text. The first line of the file must be
#parametric_pupil_format 0
After that there are zero or more components each of which is either a blank line or one of the following:
#attribute name [attribute values]
[coefficient values]
or
# label text
Valid attributes are:
As an example, here's what the file would look like for an ideal pupil (no variation in amplitude or phase out to the limit imposed by the numerical aperture) with numerical aperture of 1.3, immersion media refractive index of 1.5180, and a wavelength of 630 nanometers:
#parametric_pupil_format 0 #numerical_aperture 1.3 #immersion_refractive_index 1.5180 #lens_id 0 #zernike_set_for_amplitude 0 0 #zernike_set_for_phase -1 0 #polar_fourier_set_for_amplitude -1 -1 #polar_fourier_set_for_phase -1 -1 #wavelength_count 1 #wavelengths_um 0.63 #zernike_coeff_amplitude_0 0 0 1
Overview | Selecting the CTF or pupil function | Measuring the CTF or pupil function | CTF or pupil function formats | Method | Parameters
There are several different methods for doing the deconvolution. In the Additive method, the difference between the image and the blurred guess is added to the current guess to produce the new guess. In the Ratio method, the current guess is multiplied by the ratio of the image to the blurred guess to produce the new guess. The Richardson-Lucy method is similar to the Ratio method, but blurs the ratio with the abscissa-reversed blurring function and then multiplies that result with the current guess to get the new guess. For any of the methods, the new guess is forced to be non-negative before proceeding to the next iteration. Experience has indicated that the Ratio method converges faster than the Additive and Richardson-Lucy method (i.e., fewer cycles of deconvolution are needed). The Richardson-Lucy method can be useful for certain imaging conditions. If the imaging involves linear motion blur, the Additive or Ratio methods benefit if the data and CTF are reblurred before deconvolution; in contrast, the Richardson-Lucy method can be used without the reblurring step. The Richardson-Lucy method can also be useful for depth-dependent deconvolution, depending on how the object and image coordinates are handled. In either case, the Richardson-Lucy method has the drawback of more computation per iteration than the Additive or Ratio methods.
We have two improved methods that you can use, the Enhanced Additive and Enhanced Ratio methods. The Enhanced Additive method (which has also been called the SHAW method) is a modified form of the Additive method which, for the first three cycles (five cycles in the case of 2D deconvolution), inverse filters the difference (see the chapter in Meth. Cell Biol. vol 30B pg 364). The Enhanced Ratio method does the same thing as the Enhanced Additive method for the first three cycles (five cyles in the case of 2D deconvolution), and then switches over to the Ratio method (this method has also been called the DAA method). Both these enhanced methods are supposed to improve convergence allowing one to get away with fewer cycles of deconvolution.
On the command line, specify the method with
-method=type where type is one of the
following:
If you do not set the method on the command line, the default is to use the Enhanced Ratio method for depth-independent deconvolutions, the Ratio method for depth-dependent deconvolutions when the z coordinates have been remapped, and the Richardson-Lucy method for depth-dependent deconvolutions when the z coordinates have not been remapped.
Overview | Selecting the CTF or pupil function | Measuring the CTF or pupil function | CTF or pupil function formats | Method | Parameters
Once you have chosen an appropriate CTF or pupil function and selected a deconvolution method, there are a bunch of parameters that need to be set. If you wish to deconvolve data with an index of refraction mismatch between the sample mounting media and the expected environment (immersion oil, etc) for the lens, then you will need to adjust how the aberrations from the mismatch are modeled, the depth of the volume imaged, and the index of refraction for the mounting media. If you are doing a 2D deconvolution, you may need to adjust the 2D CTF parameter, depending on your chosen CTF. Two other parameters, the hints for the amounts of physical memory and temporary disk space to use, can affect the speed of the calculations but will not affect the results; modify these as necessary to get the best performance from your computer system.
For the remaining parameters, the defaults seem to work quite well, and unless you think that a slight improvement in the image processing will allow you to answer a biological question that you can not answer by using the defaults (in which case you should probably design a better experiment anyway!), there is almost certainly no point in fooling with these parameters.
Follow the links below for descriptions of the parameters.
Overview | Selecting the CTF or pupil function | Measuring the CTF or pupil function | CTF or pupil function formats | Method | Parameters
Number of cycles | Background | Smoothing | Force 2D | Depth effects | Depth | Mounting media n | Border rolloff | Detector fill factor | Remapping | CTF filtering | Z size for transforms | Smoothing interval | Prefilter resolution limits | Wiener filter enhancement and smoothing | Image scale factor | Memory size | Disk use | Autoterminate | Force floating-point | 2D CTF | 16-bit slices
This is how many cycles of deconvolution to carry out. Most
of the improvement happens in the first few cycles, after that
the improvement is more gradual. As a result, there is not
much point in doing more than 20 cycles (though this can vary depending
on the method chosen and the nature of the CTF). Anecdotal reports
suggest that doing extremely large numbers of cycles is actually
detrimental. On the command line, the number of cycles, n,
is specified with -ncycl=n.
To generate simulated blurred data for testing, use zero cycles of deconvolution and disable smoothing and prefiltering, the output will then be the blurred input.
For each wavelength, you can specify a background value that is
subtracted from each point in the input data before deconvolution
is done. On the command line, the background values for each
wavelength, x1, x2, x3, x4, and
x5 are specified by -sub=x1:x2:x3:x4:x5.
Every so many cycles (set by the
smoothing interval) the guess is smoothed by a
low pass filter to prevent noise from blowing up and dominating the solution.
The amount of smoothing increases with the value of the smoothing parameter;
a value of zero turns off smoothing. The default value for the smoothing
parameter, 0.15, is recommended. On the command line, specify the smoothing
parameter with the first parameter to the -smooth option
(i.e. -smooth=s:w). The second
parameter, w, controls the amount of
smoothing in the Wiener filter when an enhanced
method is used.
When the smoothing parameter, s, is greater than zero, the smoothing filter's analog representation as a function of the spatial frequency, k is Gaussian, e^(-0.5 * (k / (.05 + .3 * (1 - s)))^2), where k has units of cycles/pixel.
The deconvolution can be done as a 2D procedure, section by section.
This is a lot faster than 3D, and can also be done on projections. On
the command line, use -2D to force 2D processing.
The deconvolution application can simulate the aberrations due to a mismatch between immersion media and mounting media indices of refraction. If you have a pupil function describing the imaging system when the immersion media and mounting media have same indices of refraction, the deconvolution application can modify that pupil function with the simulated aberrations and use the result to deconvolve your images.
The "Depth effects" menu controls how the deconvolution application models the effects of the index of refraction mismatch. The available choices are:
To specify the modeling of depth-dependent effects from the command line,
use -deptheffect=type where type is one of
the following: none, mean, or full. By default, the deconvolution does not
attempt to model the effects due to a mismatch between the mounting media
and immersion media indices of refraction.
If you wish to deconvolve data with an index of refraction mismatch between
the mounting media and the immersion media for the lens (i.e. you've set
the Depth effects menu to something other than
"None"), you'll need to give the deconvolution application the depth of the
imaged volume from the cover slip. In the "Depth" field, enter the amount of
stage translation, in microns, necessary to shift the system from imaging the
bottom of the cover slip to where you started taking the images for the volume
(if you take a subset of the volume in z, the deconvolution application will
modify the depth you provide appropriately). To set the depth of the volume
on the command line, use -depth=d where d
has units of microns. By default, the deconvolution application assumes that
the most shallow z section is at the bottom of the cover slip (i.e. a depth of
zero microns).
If the first z section in the input file is closer to the cover slip than
the last z section, turn on the "z reversed" button in the dialog or specify
-reversez on the command line. If the first z section in the
input file is farther from the cover slip than the last z section, turn off
the "z reversed" button in the dialog and do not specify -reversez
on the command line.
If you wish to deconvolve data with an index of refraction mismatch between
the mounting media and the immersion media for the lens (i.e. you've set
the Depth effects menu to something other than
"None"), you'll need to give the deconvolution application the index of
refraction of the mounting media. Enter that index of refraction in
the "Mounting media n" field of the user interface or specify it on
the command line with -nmount=n, where n is
the index of refraction. By default, the deconvolution application assumes
that the mounting media has an index of refraction of 1.518.
Deconvolution works best when you have a bright object
surrounded by black background. when there are bright objects
on the edges, this can cause problems. Therefore, it can
sometimes be a good idea to roll off the edges to some average
value. The number given for this parameter gives the number
of pixels in from the edge over which the rolling-off proceeds.
This feature can be a mixed blessing because while it should in
theory give a nicer image it also reduces the information from
the edges. This is particularly a problem with 256 x 256 images.
On the command line, the number of pixels over which the rolloff
occurs, n, is specified with -apodize=n.
By default, the edges of the image are not apodized.
If you use a measured CTF to characterize the imaging system, the CTF will generally include the effects due to the non-zero size of the pixels; however if you use a pupil function to characterize the imaging system, it will not include those effects. The deconvolution application can model the effects of the non-zero pixel size and combine them with the input CTF or pupil function for use in the deconvolution. The model for the pixels is that they are all rectangles with the same dimensions and that they are uniformly sensitive across their surface. The values in the "Detector fill factor" set the x and y dimensions for the modeled pixels. The dimensions are set as multiples of the pixel spacing reported in the input volume's header. If you use a CTF to characterize the imaging system, the default values for the dimensions are both zero: no modeled effects of non-zero detector size will be included since a measured CTF normally includes those effects. If you use a pupil function to characterize the imaging system, the default values for the dimensions are both one: there are no insensitive areas on the CCD at the edges of pixels.
To set the modeled pixel size on the command line, use the option
-detfill=x:y where
x is the modeled x size of the pixel as multiple of the x pixel
spacing and y is the modeled y size of the pixel as a multiple of
the y pixel spacing.
For depth-dependent deconvolution and a mismatch between the indices of refraction for the immersion and mounting media, the point spread functions include a shift in z. The additive and ratio update algorithms are not designed to accommodate such a shift; to make those algorithms effective in the depth-dependent case, it is possible to generate the deconvolved result on a different set of grid points in z - i.e. to remap the image depths from the input to object depths in the output. The type of remapping done is controlled by the remapping menu in the graphical user interface. The available options are:
To set the type of remapping done from the command line, use
-remap=type where type is one of the
following: none, paraxial, zernike, or search.
The deconvolution application can apply a filter to your specified CTF or, if you supply a pupil function, to the PSFs computed from that pupil function. The primary purpose for the filter is as a fudge factor to bring PSFs computed from pupil functions into agreement with measurements. Use the "Form" menu in the special parameters dialog to select the profile for the filter. The available options are:
On the command line, use -ctffilt=f to set
the form of the CTF filter where f is one of the following:
off, ftsphere, j0, or gaussian. If you do not specify the form of the filter,
the default is "off" and no filter is applied. Use
-ctffiltparam=a:b:c
to set the form parameters. If you do not specify the form parameters, the
default values are specific to the form of the filter selected:
Use -ctffiltdim=sx:sy:sz
to set the frequency scale factors. If you do not specify the frequency
scale factors, the default parameters are specific to the form of the filter
selected:
When computing the 3D Fourier transforms, the data can be padded
in the z direction. This is important for handling edge effects in z
and can improve speed (some sizes are handled more efficiently, powers
of two for example). The default z size chosen for the data plus padding is
the smallest power of two at least as large as 1.3 times the original z size.
To alter the z size for the data plus padding, change the value in the
"Z size for transform" field of the user interface (in the special parameters
dialog). On the command line, use -nzpad=n to set
the z size with padding to be n sections.
After every n iterations of the deconvolution algorithm
a smoothing filter is applied to the guess. To change the value of
n from the graphical user interface, change the entry in
the "Smoothing interval" field which is within the special parameters dialog.
To change the value of n from the command line, use
-nsmod=n. The default value of n is five.
Prior to the first iteration of the deconvolution algorithm, the deconvolution application can filter the input data to reject high frequency components. The rationale for this is to remove contributions that are known to be noise: i.e. lie beyond the cutoff frequency for the imaging system.
The filter applied is radially symmetric. As a function of the spatial frequency, k (equal to (kx^2 + ky^2 + kz^2)^0.5 where kx, ky, and kz have units of cycles/pixel), the filter is one for k less than r1, zero for k greater than r2, and e^(s*(k-r1)^2) for k greater than or equal to r1 and less than r2. The prefilter input parameters, l1 and l2, control the values of r1, r2, and s. If the larger of l1 and l2 is greater than .00001, then r1 is the smaller of one and the x pixel spacing divided by the larger of l1 and l2; otherwise, r1 is equal to the larger of l1 and l2. If the smaller of l1 and l2 is greater than .00001, then r2 is the smaller of one and the x pixel spacing divided by the smaller of l1 and l2; otherwise, r1 is the smaller of l1 and l2. If 0.333 * (r2 - r1) is greater than .00001, then s is -0.5 / (0.333 * (r2 - r1))^2; otherwise, s is zero. The default value for l1 is 0.3, and the default value for l2 is 0.22.
The values of l1 and l2 are the values that appear
in the "Prefilter resolution limits" field of the graphical user interface.
You can specify l1 and l2 on the command line with
-prefilter=l1:l2.
l1 and l2 must be greater than or equal to zero.
Setting both to zero disables prefiltering of the input image data.
For the enhanced update algorithms, an inverse filter for the CTF is applied to the difference of the blurred guess and observation. The result of the inverse filter is then smoothed with another filter and the result is added to the current guess to form the guess for the next iteration. The inverse filter is approximated with a Wiener-like filter of the form: 1 / (CTF(kx,ky,kz) + s). The value of s, the noise-suppression term of the Wiener-like filter, is zero if the value, ew shown in the "Wiener filter enhancement" field is zero; otherwise, it is 1.001 minus ew. The smoothing filter applied to the result of the inverse filter has a Gaussian form ( e^(-0.5 * (k / (.05 + .3 * (1 - sw)))^2) where k is the spatial frequency in cyles/pixel) when the value, sw shown in the "Wiener filter smoothing" field is greater than zero; if sw is zero, no smoothing filter is applied to the result of the inverse filter. The default value for ew is 0.9; the default value for sw is 0.8.
On the command line, use -wiener=ew to set the
equivalent of the value in the "Wiener filter enhancement" field. The
equivalent of the "Wiener filter smoothing" field is set via the second
parameter to the -smooth option (i.e.
-smooth=s:sw where s
controls the smoothing filter periodically applied to the guess between
deconvolution iterations).
All the values in the image will be scaled by this value before deconvolution. Scaling is usually unnecessary: when integer data is output it is automatically scaled to reasonably fill the range representable by the output type. If you are deconvolving data whose maximum value is considerably different than typical image data from a CCD (i.e. with a maximum value of a hundred to tens of thousands), consider specifying a scale factor to put the maximum value in that range: certain tests for "small" observation values in the ratio and Richardson-Lucy update algorithms use a fixed threshold that was probably chosen as appropriate for typical 12- or 16- bit image data.
On the command line, specify the scaling factor, x, with
-scale=x.
Use the "Memory size (MB)" field to indicate how many megabytes of physical
memory are available for the deconvolution. When there is sufficient
memory and you are doing 3D depth-independent deconvolution, three arrays,
each nx by ny by nz, are kept in memory; otherwise, one or two of the arrays,
depending on the available memory, are stored on disk. For depth-dependent
3D deconvolution, the memory available will be used for deconvolution working
space and storage for the PSFs as a function of depth. If you want to force
the deconvolution to use the minimum amount of memory possible, turn on the
toggle labeled Use minimal amount. On the command line, the
available amount of memory, m, in megabytes is set by
-maxmem=m, and forcing the deconvolution to use a
minimum amount of memory is set with -nomem. By default,
the deconvolution application assumes that 480 megabytes of physical memory
are available.
If the deconvolution application can not hold all the PSFs as a function of depth in memory during depth-dependent deconvolution, it can use some combination of calculating PSFs on the fly or precomputing the PSFs and storing them on disk to make up the deficit. Which strategy performs best will depend on the relative speeds of the processor versus the disk. The amount of disk space available is also a limitation: the deconvolution application will put its temporary files in the directory named by the IVE_SHMDIR environment variable (this is normally set to /var/tmp in the Priism setup script; if the IVE_SHMDIR environment variable is not set, the deconvolution application will use the value of the HOME environment variable or, if the HOME environment variable is not set, /var/tmp as the location for the temporary files).
Use the "Disk use (MB)" field to indicate the maximum amount of disk space,
in megabytes, the deconvolution application should use for temporary storage
during depth-dependent deconvolution. If storing precomputed PSFs on disk is
slower than calculating them on the fly, use a value equal to either
zero, one times the size of one volume (nx * ny * nz * 4 bytes), or two
times the size of one volume; the latter two options allow the deconvolution
application to store the prefiltered observation or the current guess on
disk while the PSFs are still computed on the fly. To set the maximum
amount of disk space from the command line, use
-maxdisk=m where m is measured in
megabytes. By default, the deconvolution application will not use additional
disk space to store PSFs that did not fit into memory.
By default the deconvolution program automatically terminates
when the difference between the input data and the blurred
guess increases for two consecutive iterations. In some
cases, this may not indicate that the convolution process
won't converge and you may want to turn off the automatic
termination to see if that is the case. The command line option to
turn off autotermination is -noterm.
Forces the output file to be written in a floating-point format
(mode 2). The default is to write it in the same format as was
used for the input file. The command line option for this is
-mode2.
If you wish to use a 2D CTF for 2D deconvolution, you will need
to turn on the "2D CTF" button in the special parameters dialog of the
user interface. Otherwise, the deconvolution application will assume
that your CTF is a radially symmetric CTF and extract the kz=0 plane
for use as the CTF for 2D deconvolution. To force the CTF to be interpreted
as a 2D CTF from the command line, use -ctf2d.
With depth-dependent deconvolution, the 2D Fourier transforms of the
depth-dependent PSF slices can be stored as scaled 16-bit integers or
as 32-bit floating-point values. The former is the default because it
allows for more PSF slices to be held in memory or on disk and because the
loss of precision has not caused a degradation of the deconvolution results
from simulated test cases. If you are interested in using the higher
precision PSF slices, turn off the "16-bit slices" button in the special
parameters dialog of the user interface. The command-line equivalent to
turning off the "16-bit slices" button is the -noint16 option.