Profiles

The pipeline is configured through .toml configuration files called "profiles". These are placed in a folder with the data you would like to reduce and contain flags, settings, etc. to run the reduction.

In general, values are provided in either milliarcseconds or multiples of $λ/D$, rather than pixels.

Example

Before we begin, here is an example:

[telescope]
    inst="NIRC2"
    "λ" = 3.776e-6
    D = 10.0

[calibrate]
    dark = [
        "raw/cal/N2.2025.57814.fits.gz",
        "raw/cal/N2.2025.57815.fits.gz",
    ]
    dark_psf = [
        "raw/cal/N2.20250714.57967.fits.gz",
        ...
    ]
    flaton = "raw/flaton.fits.gz"
    flatoff = ["raw/flatoff.1.fits.gz","raw/flatoff.2.fits.gz"]
    sky = [
        "raw/sci/N2.20250714.47791.fits.gz",
        ...
    ]
    sky_psf = [
        "raw/sci/N2.20250714.47397.fits.gz",
        ...
    ]
    raw_path = [
        "raw/sci/N2.20250714.48606.fits.gz",
        ...
    ]
    raw_psf_path = [
        "raw/sci/N2.20250714.47277.fits.gz",
        ...
    ]
    badpix="bpm.fits"
    output_path = "cal/2025-07-14."
    output_psf_path = "cal/2025-07-14.psf.fits.gz"
    write_masters = true
    xcorr_r_inner = 500
    xcorr_r_outer = 700
    xcorr_to_stack = false
    calibrated_path = "cal/2025-07-14.0*.fits.gz"
    calibrated_psf_path = "cal/2025-07-14.psf.fits.gz"
[subtract]
    psf_radius_px = 20
    frame_crop_inwards = 0
    recentre_psf = true
    inject_planets = [
        [0.0, 700.0, 5e-4],
    ]
    inject_direction = 1
    inject_save = false
    presubtract_rprofs = true
    prehighpass_filter = 25
    mask_geometry = "annulus_segments_horseshoe_opt"
    outer_working_angle = 900.0
    inner_working_angle = 0.0
    inner_working_angle_optimization = 100.0 # Mas
    annulus_width = 2.0 #2.0
    num_segments=2
    buffer = 0.4
    optimization_dist = 3.2
    optimization_mode="global"
    optimzation_batches=1
    goal="SNROptConic"
    goal_alpha = 5e-3
    frame_selection = "CorrelationRanking"
    # frame_selection = "PairSNRRanking"
    correct_throughput=true
    averaging="median"
[report]
    candidate_sigma_thresh = 3.5
    candidate_max_sep = 4000.0
    report_tau = 3.0
    report_show_candidates = true

Telescope Section

The [telescope] section specifies which instrument-loader code should be used to load the files.

• inst can be NIRC2, SPHERE_IFS, or LBT
• "λ" must be the effective wavelength in meters of the sequence
• D is the diamter of the telescope in meters

The latter two keys are used to calculate the size of the keys in the later sections that are in terms of $\lambda/D$, and also help guide certain heuristics like Analyse.findplanets.

Calibrate Section

The [calibrate] section specifies what raw files to load, what calibration files to apply, how to calibrate and register them, and where to put them when done.

In general, all keys that accept a filename can also accept a list of filenames. In that case, they are stacked automatically where appropriate, e.g. multiple dark frames are medianed, raw_psf frames are registered and then medianed, and raw frames are each calibrated individually.

!! Note: unsatured PSF images are currently not flat field corrected. This might bias photometry.

• dark A list of dark frames to be averaged with a median and used for dark subtraction on the raw frames.
• dark_psf A list of dark frames to be averaged with a median and subtracted from the unsatured PSF images.
• flaton One or more evenly illuminated images to serve as a flat.
• flatoff Images to be subtracted from flaton to create the master flat.
• sky Images taken of the sky background. Currently, they are medianed and subtracted from the raw images after calibration.
• sky_psf Sky backgrounds to be subtracted from the unsaturated psf images.
• raw_path The list of paths to the raw images, relative to the profile.toml file.
• raw_psf_path Same for unsatured PSF images.
• badpix An optional argument that points to a manual bad pixel file. An automatic one is generated from the master dark, but you can additionally supply a manual one for tricky artifacts. Should be 1 (or high) on pad pixels or regions, and 0 (or low) on normal pixels.
• output_path This is where the output calibrated & reigstered data goes. This is a prefix, we'll add a frame number and .fits.gz to the end.
• output_psf_path Same for the unsatured PSF, except no number is appened to the end, so this should include .fits.gz.
• write_masters Should be true or false. If true, write the calibration masters.

The registration is first done against the unsatured PSF master.

• xcorr_to_stack After initial registration, should we use an annulus in each calibrated frame to register against the initial stack of calibrated frame?
• xcorr_r_inner The inner radius of an annulus in mas uesd for the above. e.g. 500.0.
• xcorr_r_outer The outer radius of an annulus in mas uesd for the above. e.g. 700.0.
• calibrated_path A glob pattern matching output_path etc above for loading images later. e.g. "cal/sat*.fits*"
• calibrated_psf_path Path to read calibrated PSF from later. e.g. "cal/unsat.psf.fits.gz"

Subtract Section

The [subtract] section specifies how the stellar PSF should be suppressed: what algorithm, what parameters, what to tune, etc.

• psf_radius_px = 20 The number of pixels across to crop the unsatured PSF template to. A good default is 20. The PSF must not contain NaN after cropping.

• frame_crop_inwards = 0 Nuber of pixels from each side of the calibrated images crop off, or 0. This is good for quick reductions on just the central region.

• recentre_psf = true Should we re-align the centre of the PSF by fitting a gaussian before forward modelling? The PSF should already be aligned but this does another pass with sub-pixel accuracy. If you're using data calibrated by someone else, should will correct any off-by-one error without you having to think about it.

• inject_planets = [ # [-1200.0, 1200.0, 1e-2] # [-750.0, 650.0, 1e7], # [-350.0, 0.0, 2e-4], # [-650.0, 0.0, 2e-4], # [-850.0, 0.0, 2e-4], # [200.0, 150.0, 2e-4], ] A list of locations, amplitudes of artificial companions to inject. Can be negative. Can be empty. This is useful for estimating true throughput/flux and manual forward-modelling.

• inject_direction = 1 Rotation direction for above ADI injection. 1 for forward (see next to any real planets), -1 for backwards (see in reverse rotated stack only).

• inject_save = false Should we save the frames after injecting planets, filtering, etc. but before subtraction?

• presubtract_rprofs = true Should we begin by subtracting a radial profile off of each frame? This will kill disks, but might help planets depending on the algorithm used.

• prehighpass_filter = 25 A value in pixels or false. Should we start by applying a high-pass filter to the images? This will kill disks, but will likely help planets. Either false, or the kernel size in pixels.

• frame_selection_ADI=true Allow frames with different wavelenths as references

• frame_selection_SDI=true Use correlation between frames near the central obscuration as a guide for which frames should be matched as references

• mask_geometry = "annulus_segments" or for example "annulus_segments_horseshoe_opt". This controls the region shape for the subtraction and optimization regions. For both, the subtraction region is an annulus segment. For annulus_segments the optimization region consists of an annulus segment below and above separated by buffer. For annulus_segments_horseshoe_opt, the area in the same annulus as the subtraction region is also included in the optimization regions, subject to buffer etc.

• outer_working_angle = 2200.0 What is the largest separation we should apply the subtraction to? A value in mas, or "max"

• inner_working_angle = 50.0 What is the minimum separation we should apply the subtraction too? (mas)

• inner_working_angle_optimization = 100.0 Exlude data from the optimization process inside this radius (but still apply subtraction below it)

• annulus_width = 2.0 Width of subtraction-region annulus segments (units of λ/D)

• num_segments=1 Number of segments per annulus. 1 gives a full annulus.

• buffer = 0.4 Buffer between the edge of the subtraction region and the start of the optimization region (units of λ/D).

• optimization_dist = 3.2 Size of the optimization region in all directions, starting at edge of the subtraction region (units of λ/D). Buffer is then subtracted from this. If you make buffer bigger than optimization_dist nothing will work.

• frame_selection = "CorrelationRanking" How should reference frames be ranked when matching them to target frame? Default is to sort based on their correlation with the target frame's subtraction region. PairSNRRanking is also an option to sort based on the optimal achievable SNR between the target and reference, but doesn't seem to work as well. Set frame_selection = false to always use all references. This will be deadly to your CPU if used in global optimization on large problems.

• correct_throughput=true Correct for algortihmic throughput across the subtraction region using forward modelling. This is almost computationally free and is on by default if unspecified. Even turned off, some algorithms like SNROpt correct for throughput at the centre of the subtraction regions. This is almost free computationally so there is little reason to turn it off besides debugging.

• averaging="median" How to average frames while stacking.

These keys select which targets will be considered silmultaneously for optimization.

optimization_mode="local" This will apply the optimization/subtraction to each frame individually.

optimization_mode="global" This will apply the optimization/subtraction to multiple frames at a time, subject to optimization_batches. optimzation_batches=1 For global optimization, this controls how many sub-sequences the reduction is divided into. 1 will give the whole sequence at once. 5 would break it into 5 chunks first, and then average like in local optimization.

Goal for subtraction. This selects the subtraction algorithm. goal="SNROptConic"

• SNROptConic: Regularized SNR optimization using Convex/Conic optimization. (Convex.jl + COSMO.jl). Supports local or global. Fastest option for global.
• SNROptNewton: Optimize the SNR of subtraction using Newton's method. Very fast for small problems (<100 references). A great option for local optimization. Set goal_g_tol = 1e-7 or another tolerance value to tell the optimizer when the gradient of the SNR is sufficiently converged. Not feasible for global optimization in general. Does not support regularization.
• SNROptLBFGS Same as above (and should also set goal_g_tol) but avoids creating and solving a large Hessian matrix at each iteration. Converges slower, but might be faster for larger problems where the Hessian becomes a bottleneck. A good choice for large (>300) reference local optimization problems. Might work okay for global optimization but does not support regularization.
• LOCI Traditional LOCI least squares. Global optimization not supported. Must also set loci_rejection_dist = 0.6, Minimum rotation distance to reject, in lambda/D.
• TLOCI.Same as above, but set tloci_max_contamination = 0.2 to instead reject frames that cause more then e.g. 0.2 relative flux contamination.