<commit_before>.TH LIGHTCURVES 1 "October 2014" CASU "User Commands"
.SH NAME
lightcurves \- combine list-driven photometry into light curves
.SH SYNOPSIS
\fBlightcurves\fR [\fIoptions\fR] \fIreffile\fR \fIfile\fR [\fI...\fR]
.PP
\fBupdate\fR [\fIoptions\fR] \fIlcfile\fR \fIfile\fR [\fI...\fR]
.SH DESCRIPTION
.B lightcurves
combines list-driven photometry files into light curves, optionally
performing differential photometry.
.PP
.B update
is used to append additional list-driven photometry files to a
set of existing light curves.  This is predominantly intended for use
in the realtime system, and uses the existing comparison star
selection and magnitudes read out of the light curve file.
.PP
These programs expect input files containing FITS binary tables in the
32-column or 80-column CASU catalogue formats.  Such files are
produced by the CASU pipeline toolkit or CASUTOOLS.  The extension
structure (number of extensions, and ordering) of the input files is
preserved exactly in the outputs.  These often correspond to particular
detectors in mosaic instruments.
.PP
Inputs are generated by first choosing a reference image, which should
usually be the best one or a stack of several.  The photometric
zero-points in the light curves are tied to this image, so it should
be properly calibrated if these need to be correct.  A
.I master catalogue
is generated from this image using
.B imcore_conf
in the pipeline toolkit, or
.B imcore
in CASUTOOLS.  The astrometry may be refined, and the classifier
should usually be run on it before proceeding.
.PP
For each input image in the time-series,
.B imcore_list
is then run using the master catalogue produced above.  For normal
imaging, usually the classifier is then run on the list-driven
photometry files, but this is not strictly required.  For difference
imaging, the classifier should not be run on the list-driven files.

In the
.B SYNOPSIS
above,
.I file
is a list-driven photometry file,
.I reffile
is the master catalogue file, and
.I lcfile
is an existing light curve file from
.B lightcurves
or
.B update
to which the new points should be appended.  Multiple list-driven
photometry files can be given on the command-line, or filenames
beginning with an 
.I @
character are treated as ASCII lists of input filenames, one filename
per line.
.PP
The default behaviour if no further options are to given is just to
build light curves and display diagnostic plots.  No differential
photometry is performed, and no outputs are written.  Most of the
time, this is not very useful behaviour, so the user is likely to want
to specify at least the
.B -f
and
.B -o
options to turn on the differential photometry and write an output
light curve file, respectively.
.PP
For most data-sets, I recommend starting with only
.B -f
specified, and examining the first plot, an RMS versus magnitude
diagram, to decide what range of magnitudes are usable for comparison
stars.  The
.B -u
flag can then be used to set the brightest usable comparison star
magnitude to just below saturation (in most cases).  By default, a
range from this magnitude to 4 magnitudes fainter is used to select
comparison stars.  This usually works well, but the lower limit can
also be overridden using this flag if needed.
.PP
The output file contains FITS binary tables with one output row for
each input row from the master catalogue, unless overridden using
command-line options.  The light curves themselves are encoded as a
set of vectors within the row.  Some quantities of interest in the
output are not specific to each object, such as the exposure time, and
are written to the FITS header as numbered header keywords.
.PP
A slightly extended version of the standard CASU classification flag
system is used.  The special value of
.I 9
is used to indicate target stars.  These are treated specially during
processing: they are not considered in the comparison star selection.
In plots, these objects are circled in red.

.SH OPTIONS
The following command-line options are supported by both programs:
.TP
.BI "-c " "class"
Save only objects used as comparison stars and objects with
classification flags equal to
.I class
in the output file.  The most common use of this option is
.BI "-c " "9"
to save only targets and comparison stars, which may be desirable to
reduce file size for observations where only a few objects in the
field are of interest.  It also speeds up light curve processing since
only those objects eligible to be comparison stars and those selected
with the
.B -c
option are processed.
.TP
.BI "-o " "file"
Write out light curves to
.I
file
in FITS format.  The output file will contain one output binary table
extension for each input extension.  The file format is described
below.
.TP
.BI "-p"
Disables plots and the prompt for a plot device.  If the program was
compiled without PGPLOT, it behaves as if this flag was always set.
.TP
.BI "-s " "level"
Manually override the program's idea of the count level at which
saturation occurs.  This is compared to the Peak_height plus local sky
level from the input files and is used to flag points as saturated if
they have values more than
.I level
x 0.95.  These points are not considered in the differential
photometry and are flagged in the output files.  See below, under
.B INPUT FILES
for the behaviour if the flag is not given.
.TP
.BI "-q"
Makes the program less verbose by one step.  Can be given multiple times.
.TP
.BI "-V " "file"
This option is specific to MEarth.  The argument
.I file
is assumed to be an ASCII file containing the table of dates of
instrument changes.  This file is usually called
.B instvers
in the master directory.
.TP
.BI "-v"
Makes the program more verbose by one step.  Can be given multiple times.
.PP
The following options are supported by
.B
lightcurves
only:
.TP
.BI "-a " "aper"
The default behaviour is to generate light curves for three apertures
of radius RCORE, sqrt(2)*RCORE and 2*RCORE, and then choose the one
giving the lowest RMS scatter for each object.  All three input
apertures and the chosen one are written to separate columns in the
output file.  This value of this flag can be
.I sel
to save only the chosen aperture, or an integer to select a particular
aperture.  The apertures are numbered from zero starting with RCORE
and following the list given above.  Forcing the choice of a
particular aperture causes the other apertures to not be processed.
.TP
.BI "-d"
Difference imaging mode.  Assumes the list-driven photometry files
were generated from difference images, and not regular images.  The
main effect of giving this flag is to cause the master catalogue flux
to be added to the measured flux from the list-driven file before
further use.
.TP
.BI "-f " "degree"
Perform differential photometry using a polynomial of degree
.I degree
to fit the magnitude zero-point in each frame.  Setting the degree to
.I 0
performs standard differential photometry where only a single
magnitude zero-point is adjusted.  Not passing the flag at all
disables differential photometry.
.TP
.BI "-g " "file"
Writes out a list of good quality input files to
.I file
after processing.  This was originally intended for removing
low-quality (highly non-photometric) frames from the processing.  It
is not often used any more.
.TP
.BI "-i " "file"
An experimental attempt at correcting for intra-pixel sensitivity
variation using a pixel map stored in an ASCII file.  Not likely to be
useful.
.TP
.BI "-m"
Allow for meridian flip (and more generally, ``segment'') offsets when
computing differential photometry.  Specifically, the program looks
for frames where the transformation matrix in the astrometric solution
shows both axes to be flipped, corresponding to meridian flips in a
German type mount. In such cases,  the magnitudes for every star could
measure differently on the two sides of the meridian due to flat
fielding error.  The program fits for two different magnitudes for
every star while solving differential photometry to allow for this.
Passing the option twice causes the light curves to also be corrected
for the offsets, as well as just allowing for them in the solutions.
For MEarth, this is generalised to also allow offsets at every
instrument change, in conjunction with the
.B -V
flag.  Use of this flag causes a ``segment table'' to be written to
the output file, as described below.
.TP
.BI "-n"
Normally, the magnitude system is tied to the master catalogue by
forcing the average offset between the median magnitudes of each star
and the master catalogue to be zero.  Some very old versions of the
program did not do this and simply allowed the magnitude system to
float.  This option restores the old behaviour.
.TP
.BI "-S"
Calculate the sky background noise assuming only Poisson noise on the
measured sky background counts from SKYLEVEL x GAIN, rather than using
the empirical estimate in SKYNOISE.  This is intended to be used in
cases where the empirical estimate is contaminated by large-scale sky
background variations rather than noise.
.TP
.BI "-u " "upper\fR[\fI,lower\fR]"
Manually set the bright limit (and optionally, the faint limit) for
comparison star selection.  These quantities are given as magnitudes
and are in the same units as the RMS vs magnitude plots produced by
the program.  If
.I lower
is not given, a value of
.I upper
+ 4 magnitudes is used.  If this option is not specified, the program
will attempt to guess based on the saturation level in the frame.
These guesses are not very good, so I recommend always setting this
option.
.PP
The following options are supported by
.B
update
only:
.TP
.BI "-u "
Performs an in-place update of the input file, rather than writing
output to a new file.  The implementation actually uses temporary
files to ensure that the original input file is not destroyed if the
program crashes, so the only difference is whether the output file is
renamed on top of the input file at the end of processing.
.SH INPUT FILES
The CASU catalogue formats are documented in detail elsewhere.  These
programs are most often used with 32-column inputs generated by the
CASU pipeline toolkit, but 80-column files and CASUTOOLS are 
supposed to work.  Please email bug reports to the author if these are
found to be broken.
.PP
The input files must have a FITS-WCS conforming to the standards for
FITS images.  The coordinate system is assumed to be ICRS, any RADESYS
values are silently ignored.  The only projection types correctly
understood by the program are currently ARC, SIN, TAN and ZPN.  SIP is
not supported.  These restrictions could be lifted by transitioning to
using one of the standard WCS libraries, however beware that most of
the rest of the CASU pipeline toolkit has the same restrictions.
While a FITS WCS is expected in the input files, one that is incorrect
(or not correctly understood) should merely render the
.B ra
/
.B dec
/
.B bjd
/
.B airmass
/
.B ha
column values incorrect without further ill effect.
.PP
A number of additional FITS headers are expected to be present in the
input files.  In the following list, these are grouped into sets of
alternatives, only one of which need be present.  The preferred
keyword name is given first in each set, and they are searched in the
order given below.  All keywords are optional, although the default
behaviour (stated below) may be undesirable.
.TP
AIRMASS
.TP
AMSTART
Air mass.  Used only to correct the magnitude zero-point of the frame
for extinction for consistency with the way the photometric
calibration program does it.  If not present, does not correct for
extinction.  Note: an internal calculation is used to produce the
.B airmass
column in the output files so this FITS header does not affect it.
.TP
EXPTIME
.TP
EXPOSED
.TP
EXP_TIME
Exposure time, in seconds.  If not present, emits warning and does not
correct for exposure time.  Output files will contain zero in TEXP.
.TP
EXTINCT
Extinction in magnitudes per airmass.  See notes for MAGZPT, below,
for how this value is used.  If not present, a default is supplied
(this is from a hard-coded table for the INT/WFC for historical
reasons, namely that the header did not exist in files processed by
very old versions of the pipeline toolkit).
.TP
FILTER
.TP
WFFBAND
.TP
HIERARCH ESO INS FILT1 NAME
.TP
FILTER2
.TP
INSFILTE
Filter name.  Optional, used only in plot axis labels and for looking
up the default extinction value.  The names of the keywords for this
quantity are wildly inconsistent between different telescopes /
instruments. 
.TP
GAIN
.TP
HIERARCH ESO DET OUT1 GAIN
.TP
EGAIN
Reciprocal gain, in electrons per data number.  If not present,
assume unity and emit warning.
.TP
HEIGHT
.TP
OBSALT
.TP
ALT-OBS
.TP
SITEALT
.TP
HIERARCH ESO TEL GEOELEV
Observing site height above the geoid, in metres.  Assumed to be zero
if not present (DANGER!).
.TP
LATITUDE
.TP
OBSLAT
.TP
LAT-OBS
.TP
SITELAT
.TP
HIERARCH ESO TEL GEOLAT
Observing site latitude, in degrees, following the standard North
positive convention.  Site coordinates are needed to calculate the
Barycentric position and velocity vectors of the observer for the
.B bjd
/
.B airmass
/
.B ha
calculations.  If LATITUDE and LONGITUD (or equivalents) are not both
available, the observer is assumed to be at the Geocentre in the
.B bjd
calculation, and
.B airmass
and
.B ha
are not computed.  The observing site location header keywords are
(sadly) yet another set that are wildly inconsistent between
different observatories, although they are at least usually in the
same units.
.TP
LONGITUDE
.TP
OBSLONG
.TP
LONG-OBS
.TP
SITELONG
.TP
<NAME>
Observing site longitude, in degrees, East positive.  See comments
under LATITUDE, above.
.TP
MAGZPT
.TP
ZMAG
Magnitude zero-point.  The CASU standard variant is MAGZPT and is for
an exposure time of one second and airmass of unity (and for mosaic
instruments, PERCORR=0).  ZMAG is a variant seen in UNSW APT data and
is for the exposure time of the frame and an airmass of unity.  If not
present, uses MAGZPT=25.0 (this is a somewhat randomly chosen value,
but happens to be a shockingly good match for a certain NASA exoplanet
hunting satellite) and emits a warning.
.TP
MJD-OBS
.TP
MJD
.TP
JD
Time stamp as Modified Julian Day or Julian Day, respectively.  The
program currently ignores the TIMESYS keyword and always assumes these
are in the UTC time-system.  If not present, uses 2000 January 1 at
12 hours UTC (aka 2000.0) and emits a warning.  The program will
run without time stamps available, but many outputs are meaningless
and no corrections for target star proper motion are performed.
.TP
PEDESTAL
Constant offset added to the counts in the image to prevent negative
numbers appearing and being clipped at zero.  Assumed to be zero if
not present.
.TP
PERCORR
Per-detector adjustment added in to the magnitude zero-point (the
MAGZPT values are for the whole file, not per extension).  Value in
magnitudes, default is zero for no correction.
.TP
SATLEV
.TP
SATURATE
Saturation level, assumed to be in counts.  This is compared to the
Peak_height to determine if a source is saturated.  SATURATE is
written by the classifier, so should be present in most input files;
the SATLEV form overrides this value and is used because the
classifier often cannot accurately estimate the saturation level,
particularly when there were no saturated sources on the frame.  In
MEarth, SATLEV originates from the FITS header rewriter, and is
written at the same time as GAIN and READNOIS.  The values were
derived by hand from the non-linearity data.  The
.B -s
command-line flag overrides the headers if given.  If not present and
the command line option was not given, uses 65535.
.PP
Due to the lack of standardisation, the header keyword names (and
contents) for these quantities can differ greatly between
observatories.  Some common variations are supported, but the amount
of changes required to support even just the limited number of
telescope and instrument combinations the program had been used on
eventually became excessive (for some quantities, there are almost as
many variations on the keyword name as there are telescopes), so the
preferred solution is now to translate the headers in the particular
flavour of input files in question into the preferred headers listed
above.  It is expected that this translation has already been
performed before the program is run.
.PP
The binary table columns needed are only those written by the standard
.B imcore_list
program and the classifier.  The Classification column is required to
be present in the master catalogue, but is not needed in the other
input files.  In addition to these columns, in the master catalogue,
the FITS header reader searches for a pair of optional columns named
.B PMRA
and
.B PMDec
which are assumed to be the proper motions of each source in rad/yr,
and are used to correct for proper motion when computing the sky
position of the source in each target frame for barycentering and the
airmass and hour angle columns.  They are assumed to be zero if not
present.
.SH OUTPUT FILES
Light curves are stored as FITS binary tables, with one row per
object.  The light curves themselves and several other time-dependent
quantities that are unique to each object are stored as vectors
(arrays) in cells of the table.  Quantities common to the entire frame
(e.g. observation times, exposure times, etc.) are stored into
numbered keywords in the FITS header (see BUGS, below).
.SS Header keywords
The
.B lightcurves
program adds a number of header keywords to the output that describe
the light curves, options used for processing, or are used to
communicate with the
.B update
program.  The following example shows these keywords:
.PP
.nf
NMEAS   =                  748 / Number of points in each lightcurve
NROWMAST=                 1246 / Number of rows in master catalogue
MJDBASE =        53326.0000000 / Base MJD for time axis
SATMAG  =              11.4264 / Approximate saturation magnitude
FLIM    =              22.0237 / Flux limit of reference catalogue
ZP      =              27.5753 / Zeropoint for magnitudes
UMLIM   =                12.00 / Upper mag limit for fit
LMLIM   =                15.00 / Lower mag limit for fit
THEOSKY =                    F / T theoretical sky noise, F empirical
POLYDEG =                    0 / Polynomial degree in fit
APSEL   =                    0 / Aperture used (0 = automatic)
APMODE  =                    3 / Aperture output mode
DOMERID =                    2 / Meridian flip removal?
REFFANG =             1.575403 / Reference file field angle
NSEGME  =                    1 / Number of segments
.fi
.PP
The ones most likely to be of interest for ``consumers'' of the file
format are NMEAS, MJDBASE and NSEGME.
.SS Header segment table
Information about the ``segments'' into which the light curve was
split during processing when using the
.B -m
flag are stored in into the header as numbered keywords for each
segment.  NSEGME gives the total number of segments.  These are
guaranteed to be populated and numbered contiguously.  In the common
case where there was only one segment and the
.B -m
flag was not used, these additional headers are the following:
.PP
.nf
SEGV1   =                   -1 / Segment 1 instrument version number
SEGD1   =                   -1 / Segment 1 instrument change date
SEGA1   =                    0 / Segment 1 angle
.fi
.PP
The use of ``segments'' for purposes other than meridian flips is
currently a MEarth-specific feature, as are the SEGV and SEGD numbered
headers.  SEGA gives a number in the same form as IANG, below.
.SS Header time-series vectors
A number of quantities that apply to all measurements in a frame are
stored into the header as numbered keywords, forming a vector.  These
are encoded as HHH\fIn\fR, where HHH is a header name and
.I n
is a number running from 1 to NMEAS.  This is similar to the scheme
used for the binary table headers (e.g. TTYP, TFORM, etc.)
themselves.
.PP
For example, the array of observation times is stored as follows:
.nf
TV1     =            1.1105781 / Time value for datapoint 1
TV2     =            1.1574578 / Time value for datapoint 2
...
TVn     =            1.9384262 / Time value for datapoint n
.fi
.PP
The following quantities are stored in this manner:
.TP
.B TV
Observation times.  These are in the same time-system as the input
files (presuming the headers are compliant, this is specified by
TIMESYS, but it is usually UTC), but with MJDBASE subtracted to get
the values to fit in less digits.  To recover the original MJD values,
add MJDBASE.
.TP
.B TEXP
Exposure times, in seconds, for each frame.  Copied from EXPTIME and
its variants in the input list-driven photometry files.
.TP
.B OFF
Median magnitude zero-point residual after applying differential
photometry.  It should be zero, but occasionally isn't when this
(robust) estimator returns a different result from the mean or
polynomial used to correct the frame zero-point.  The name is
misleading, this column is probably not what you'd think.  Not very
useful.
.TP
.B RMS
RMS scatter of the magnitude zero-point residuals after applying
differential photometry.  This quantity can be used to detect frames
where the zero-point correction didn't work very well, usually because
they were taken in non-photometric conditions.  As with most uses of
RMS relating to this software, it is actually computed using median
absolute deviation, scaled (by a factor of 1.48) to Gaussian RMS
equivalent.
.TP
.B EXTC
Differential photometry correction applied to the frame.  This is
expressed as the delta(magnitude) that was applied, so negative
numbers mean less light.  The name is misleading, it is -extinction.
This is probably what you wanted when you looked at OFF above.
.TP
.B SEE
FWHM of the stellar images in the frame, in pixels.  Another
misleading name since seeing is only one of many things which can
influence FWHM.  Copied from SEEING in the input list-driven
photometry files.
.TP
.B ELL
Average ellipticity of stellar images in the frame.  Copied from
ELLIPTIC in the input list-driven photometry files.
.TP
.B SKY
.TP
.B NOIS
Global sky level and noise for the frame.  Copied from SKYLEVEL and
SKYNOISE in the input list-driven photometry files.
.TP
.B FANG
Position angle of the frame, in radians.  Zero means right ascension is
parallel to the CCD x coordinate.
.TP
.B IANG
Nearest integer modulo 2 of position angle of this frame minus the
reference, wrapped to [0,2*pi) and divided by pi.  This quantity is
used for detecting meridian flips, where it will change from 0 to 1.
.TP
.B ISEG
Segment number to which this frame belongs.  Can be used in
conjunction with
.B -m
to fit for separate magnitudes for the star in each segment in data
analysis, or with
.B -mm
to allow for the effects of having already done so in the light curve
generation.
.TP
.B IUPD
Update number when the data point was added, numbering from 1 if the
point was added by the
.B update
program.  The value increments by one each time the
.B update
program is used to append new points to the light curve.  0 if the
point was in the original set and was written out by
.B lightcurves
meaning the frame was included in the original set of differential
photometry solutions used to select and characterise the comparison
stars.
.TP
.B LXX
.TP
.B LXY
.TP
.B LYX
.TP
.B LYY
.TP
.B LXD
.TP
.B LYD
These headers record a standard 6-coefficient linear transformation
from the pixel coordinates in the master frame to the pixel
coordinates in the target frame.  This transformation is derived
using all the stars detected in the frame, not just those considered
as comparison stars, so is written only if all of the stars in the
frame were processed (i.e. the
.B -c
option was not used).
.PP
A number of other, instrument-specific quantities may also be stored.
These are not yet documented.
.SS Table columns
The tables themselves contain the following columns:
.TP
.B x
X position of the object on the master frame, in pixels.
.TP
.B y
Y position of the object on the master frame, in pixels.
.TP
.B medflux
Median magnitude.
.TP
.B rms
Light curve RMS in magnitudes.  The value is actually computed using a
robust MAD estimator scaled to Gaussian RMS equivalent.
.TP
.B chisq
Chi squared of the light curve assuming a constant magnitude.
.TP
.B nchisq
Number of data points in
.B chisq
.TP
.B class
Source classification in the master frame.  This follows CASU
conventions, specifically: -1 = stellar; 0 = junk-like; 1 =
non-stellar; 9 = target.  Values of -2 and 2 are also used, these are
usually treated the same as -1 and 1 respectively.
.TP
.B bflag
Flag indicating the deblender was triggered in the master catalogue
for this object.  Indicates aperture flux
.I may
be contaminated by a close companion.
.TP
.B cflag
Count of the number of frames where bad or low-confidence pixels were
encountered while summing the aperture photometry.  I recommend using
the flags column instead.
.TP
.B sflag
Count of the number of frames where saturated pixels were encountered
while summing the aperture photometry.  I recommend using the flags
column instead.  See there for comments on the reliability of the
saturation flagging.
.TP
.B pointer
Row number in the input master catalogue.  This is invariant over
removal of rows from the output light curve file, for example by using
the
.B -c
option while processing the light curve, or using the CFITSIO extended
filename syntax, etc.  In the absence of other means to identify
specific objects, this column should be used.
.TP
.B offsets
.B Internal use only.
This column is used to pass state (specifically, the segment or
meridian flip offsets for each aperture) to the
.B update
program.
.TP
.B apnum
Number of the chosen aperture (numbering from zero in the same way as
the
.B -a
flag).
.TP
.B apradius
Radius of the chosen photometric aperture, in units of RCORE.
.TP
.B compok
Flag indicating whether the source was considered for use as a
comparison star.  To tell if the source was actually used as a
comparison star, see the
.B weight
column, below.
.TP
.B bjd
.B Vector.
Barycentric MJD in the TDB time-system.  This is stored as MJD to
reduce loss of accuracy.  Please don't forget the 0.5 in the
definition of MJD.
.TP
.B flux
.B Vector.
The light curve itself: magnitude as a function of time.  Can
be NULL (NaN in the FITS file) when no measurement was available.
.TP
.B fluxerr
.B Vector.
Uncertainties in the magnitudes.  Uses a standard CCD noise model
including Poisson error in the target, sky noise, scintillation, and
the error in the mean (or polynomial fit) of the magnitude zero-point
correction.  This model is known to (as usual) underestimate the true
uncertainties in most ground-based data.
.TP
.B xlc
.B Vector.
X pixel coordinate time-series.
.TP
.B ylc
.B Vector.
Y pixel coordinate time-series.
.TP
.B airmass
.B Vector.
Air mass time-series.  Can be used for detecting and correcting
residual (e.g. colour-dependent) extinction not removed by standard
differential photometry.
.TP
.B ha
.B Vector.
Hour angle time-series in radians.  Please do not attempt to use
for detecting meridian flips, it is not reliable for this purpose
because it is possible to take frames on the ``wrong'' side of the
meridian on some equipment.
.TP
.B weight
.B Vector.
Weight of this source in the comparison star solution.  Zero if not
included.
.TP
.B sky
.B Vector.
Local sky level in counts.
.TP
.B peak
.B Vector.
Peak counts in the source, including sky.
.TP
.B flags
.B Vector.
Flags for each data point.  This is encoded as a bitmask from the
following values: 1 = no data point (star was not on the detector); 2
= aperture contains bad pixels; 4 = saturated.  Note that the
reliability of saturation flagging depends critically on the accuracy
of the saturation level assumed, and mildly saturated data may be
still be usable for many purposes depending on the requirements.
To detect data points that are
.I guaranteed
to be bad, check if (flags & 3) is non-zero, where & is the bitwise
AND operator.
.TP
.B ra
ICRS right ascension of the source on the master frame, in radians.
.TP
.B dec
ICRS declination of the source on the master frame, in radians.
.TP
.B pmra
Assumed sky-projected proper motion in right ascension, in arcsec/yr.
This is copied from the input file.
.TP
.B pmdec
Assumed proper motion in declination, in arcsec/yr.  This is copied
from the input file.
.TP
.B refmag
Magnitude of the source on the master frame, used to normalise the
light curves.  Following CASU conventions this is normally computed in
aperture 0 (the smallest), however note that for MEarth the program is
compiled to use aperture 2 instead.
.PP
In the table above,
.B vector
indicates that the column is a time-series vector, of length NMEAS.
.PP
If multiple apertures were processed, the \fBmedflux\fR, \fBrms\fR and
\fBoffsets\fR columns are for the chosen aperture, and additional
columns named \fBmedflux1\fR, \fBmedflux2\fR, etc. are written out for
each original aperture.  This is also done for the \fBflux\fR,
\fBfluxerr\fR, and \fBweight\fR columns unless the
.BI "-a " "sel"
option is used to suppress the output.
.SH EXAMPLE
This example shows the full end-to-end process of generating light
curves from a set of reduced images of the same field.  It uses
programs found in CASUTOOLS.  The FITS headers are assumed to already
be massaged to a suitable form (see
.B INPUT FILES
above) and to contain sufficiently accurate WCS solutions in one of
the supported projections.
.PP
Suppose the image files are named
.BI "target" "0001" ".fit"
to
.BI "target" "nnnn" ".fit"
and reside in the current directory, along with a confidence map
.B conf.fits
which may have already been used to generate catalogues and obtain the
WCS solutions.  This naming convention has been in use for a while at
CASU and intentionally gives images a different filename extension
from everything else to avoid confusion and accidents.  However this
is purely optional.
.PP
First, we need to choose (and create, if necessary) a master
catalogue.  Usually a set of catalogues were used in making the WCS
solutions, and provided these used suitable
.B imcore
parameters one of them can simply be reused as the master for the
photometry.  However, here we'll assume we need to make a new one.
Another reason to do this might be severe background variations
(e.g. nebulosity) causing large numbers of false source detections, in
which case the nebuliser might be used to preprocess the input file
before running
.B imcore
on it (as an aside: in this case, I suggest then using
.B imcore_list
to make a list-driven file out of the
.B original
non-nebulised image, using the nebulised catalogue to place the
apertures, and then using this list-driven file as the master).
.PP
I usually symlink
.B master.fit
to the image I'm using as the (single) master, but some may prefer to
stack a few good images.  I tend to use the SEEING and ELLIPTIC
headers to find the best image.
.PP
We then generate the master catalogue and classify it:
.PP
.nf
  imcore --rcore=3 --cattype=1 master.fit conf.fits master_cat.fits 4 2
  classify master_cat.fits
.fi
.PP
the
.B imcore
parameters are purely illustrative and will need to be changed for the
specific application.  I usually suggest using a higher threshold (the
second number), than one would for ``standard'' survey applications,
because the faintest objects don't usually produce useful light curves
and serve only to slow down processing and waste space in the output
files.  Note that I recommend always specifying the catalogue type, it
has a had a habit of changing as new versions have been released.
.PP
In this example, I will use the WCS to align the input images and
place apertures (this is not always a good idea, e.g. if the WCS
solutions are a bit sketchy, or none are available).  To make the list
driven files (and classify them) for all the images automatically, the
following shell script is used:
.PP
.nf
  #!/bin/sh
  
  for imfile in target????.fit; do
      listfile=`echo "$imfile" | sed -e 's/\.fit$/_list.fits/g'`
      imcore_list --rcore=3 --cattype=1 "$imfile" conf.fits master_cat.fits "$listfile" 2
      classify "$listfile"
  done
.fi<commit_msg>Wording goof.<commit_after>.TH LIGHTCURVES 1 "October 2014" CASU "User Commands"
.SH NAME
lightcurves \- combine list-driven photometry into light curves
.SH SYNOPSIS
\fBlightcurves\fR [\fIoptions\fR] \fIreffile\fR \fIfile\fR [\fI...\fR]
.PP
\fBupdate\fR [\fIoptions\fR] \fIlcfile\fR \fIfile\fR [\fI...\fR]
.SH DESCRIPTION
.B lightcurves
combines list-driven photometry files into light curves, optionally
performing differential photometry.
.PP
.B update
is used to append additional list-driven photometry files to a
set of existing light curves.  This is predominantly intended for use
in the realtime system, and uses the existing comparison star
selection and magnitudes read out of the light curve file.
.PP
These programs expect input files containing FITS binary tables in the
32-column or 80-column CASU catalogue formats.  Such files are
produced by the CASU pipeline toolkit or CASUTOOLS.  The extension
structure (number of extensions, and ordering) of the input files is
preserved exactly in the outputs.  These often correspond to particular
detectors in mosaic instruments.
.PP
Inputs are generated by first choosing a reference image, which should
usually be the best one or a stack of several.  The photometric
zero-points in the light curves are tied to this image, so it should
be properly calibrated if these need to be correct.  A
.I master catalogue
is generated from this image using
.B imcore_conf
in the pipeline toolkit, or
.B imcore
in CASUTOOLS.  The astrometry may be refined, and the classifier
should usually be run on it before proceeding.
.PP
For each input image in the time-series,
.B imcore_list
is then run using the master catalogue produced above.  For normal
imaging, usually the classifier is then run on the list-driven
photometry files, but this is not strictly required.  For difference
imaging, the classifier should not be run on the list-driven files.

In the
.B SYNOPSIS
above,
.I file
is a list-driven photometry file,
.I reffile
is the master catalogue file, and
.I lcfile
is an existing light curve file from
.B lightcurves
or
.B update
to which the new points should be appended.  Multiple list-driven
photometry files can be given on the command-line, or filenames
beginning with an 
.I @
character are treated as ASCII lists of input filenames, one filename
per line.
.PP
The default behaviour if no further options are to given is just to
build light curves and display diagnostic plots.  No differential
photometry is performed, and no outputs are written.  Most of the
time, this is not very useful behaviour, so the user is likely to want
to specify at least the
.B -f
and
.B -o
options to turn on the differential photometry and write an output
light curve file, respectively.
.PP
For most data-sets, I recommend starting with only
.B -f
specified, and examining the first plot, an RMS versus magnitude
diagram, to decide what range of magnitudes are usable for comparison
stars.  The
.B -u
flag can then be used to set the brightest usable comparison star
magnitude to just below saturation (in most cases).  By default, a
range from this magnitude to 4 magnitudes fainter is used to select
comparison stars.  This usually works well, but the lower limit can
also be overridden using this flag if needed.
.PP
The output file contains FITS binary tables with one output row for
each input row from the master catalogue, unless overridden using
command-line options.  The light curves themselves are encoded as a
set of vectors within the row.  Some quantities of interest in the
output are not specific to each object, such as the exposure time, and
are written to the FITS header as numbered header keywords.
.PP
A slightly extended version of the standard CASU classification flag
system is used.  The special value of
.I 9
is used to indicate target stars.  These are treated specially during
processing: they are not considered in the comparison star selection.
In plots, these objects are circled in red.

.SH OPTIONS
The following command-line options are supported by both programs:
.TP
.BI "-c " "class"
Save only objects used as comparison stars and objects with
classification flags equal to
.I class
in the output file.  The most common use of this option is
.BI "-c " "9"
to save only targets and comparison stars, which may be desirable to
reduce file size for observations where only a few objects in the
field are of interest.  It also speeds up light curve processing since
only those objects eligible to be comparison stars and those selected
with the
.B -c
option are processed.
.TP
.BI "-o " "file"
Write out light curves to
.I
file
in FITS format.  The output file will contain one output binary table
extension for each input extension.  The file format is described
below.
.TP
.BI "-p"
Disables plots and the prompt for a plot device.  If the program was
compiled without PGPLOT, it behaves as if this flag was always set.
.TP
.BI "-s " "level"
Manually override the program's idea of the count level at which
saturation occurs.  This is compared to the Peak_height plus local sky
level from the input files and is used to flag points as saturated if
they have values more than
.I level
x 0.95.  These points are not considered in the differential
photometry and are flagged in the output files.  See below, under
.B INPUT FILES
for the behaviour if the flag is not given.
.TP
.BI "-q"
Makes the program less verbose by one step.  Can be given multiple times.
.TP
.BI "-V " "file"
This option is specific to MEarth.  The argument
.I file
is assumed to be an ASCII file containing the table of dates of
instrument changes.  This file is usually called
.B instvers
in the master directory.
.TP
.BI "-v"
Makes the program more verbose by one step.  Can be given multiple times.
.PP
The following options are supported by
.B
lightcurves
only:
.TP
.BI "-a " "aper"
The default behaviour is to generate light curves for three apertures
of radius RCORE, sqrt(2)*RCORE and 2*RCORE, and then choose the one
giving the lowest RMS scatter for each object.  All three input
apertures and the chosen one are written to separate columns in the
output file.  This value of this flag can be
.I sel
to save only the chosen aperture, or an integer to select a particular
aperture.  The apertures are numbered from zero starting with RCORE
and following the list given above.  Forcing the choice of a
particular aperture causes the other apertures to not be processed.
.TP
.BI "-d"
Difference imaging mode.  Assumes the list-driven photometry files
were generated from difference images, and not regular images.  The
main effect of giving this flag is to cause the master catalogue flux
to be added to the measured flux from the list-driven file before
further use.
.TP
.BI "-f " "degree"
Perform differential photometry using a polynomial of degree
.I degree
to fit the magnitude zero-point in each frame.  Setting the degree to
.I 0
performs standard differential photometry where only a single
magnitude zero-point is adjusted.  Not passing the flag at all
disables differential photometry.
.TP
.BI "-g " "file"
Writes out a list of good quality input files to
.I file
after processing.  This was originally intended for removing
low-quality (highly non-photometric) frames from the processing.  It
is not often used any more.
.TP
.BI "-i " "file"
An experimental attempt at correcting for intra-pixel sensitivity
variation using a pixel map stored in an ASCII file.  Not likely to be
useful.
.TP
.BI "-m"
Allow for meridian flip (and more generally, ``segment'') offsets when
computing differential photometry.  Specifically, the program looks
for frames where the transformation matrix in the astrometric solution
shows both axes to be flipped, corresponding to meridian flips in a
German type mount. In such cases,  the magnitudes for every star could
measure differently on the two sides of the meridian due to flat
fielding error.  The program fits for two different magnitudes for
every star while solving differential photometry to allow for this.
Passing the option twice causes the light curves to also be corrected
for the offsets, as well as just allowing for them in the solutions.
For MEarth, this is generalised to also allow offsets at every
instrument change, in conjunction with the
.B -V
flag.  Use of this flag causes a ``segment table'' to be written to
the output file, as described below.
.TP
.BI "-n"
Normally, the magnitude system is tied to the master catalogue by
forcing the average offset between the median magnitudes of each star
and the master catalogue to be zero.  Some very old versions of the
program did not do this and simply allowed the magnitude system to
float.  This option restores the old behaviour.
.TP
.BI "-S"
Calculate the sky background noise assuming only Poisson noise on the
measured sky background counts from SKYLEVEL x GAIN, rather than using
the empirical estimate in SKYNOISE.  This is intended to be used in
cases where the empirical estimate is contaminated by large-scale sky
background variations rather than noise.
.TP
.BI "-u " "upper\fR[\fI,lower\fR]"
Manually set the bright limit (and optionally, the faint limit) for
comparison star selection.  These quantities are given as magnitudes
and are in the same units as the RMS vs magnitude plots produced by
the program.  If
.I lower
is not given, a value of
.I upper
+ 4 magnitudes is used.  If this option is not specified, the program
will attempt to guess based on the saturation level in the frame.
These guesses are not very good, so I recommend always setting this
option.
.PP
The following options are supported by
.B
update
only:
.TP
.BI "-u "
Performs an in-place update of the input file, rather than writing
output to a new file.  The implementation actually uses temporary
files to ensure that the original input file is not destroyed if the
program crashes, so the only difference is whether the output file is
renamed on top of the input file at the end of processing.
.SH INPUT FILES
The CASU catalogue formats are documented in detail elsewhere.  These
programs are most often used with 32-column inputs generated by the
CASU pipeline toolkit, but 80-column files and CASUTOOLS are 
supposed to work.  Please email bug reports to the author if these are
found to be broken.
.PP
The input files must have a FITS-WCS conforming to the standards for
FITS images.  The coordinate system is assumed to be ICRS, any RADESYS
values are silently ignored.  The only projection types correctly
understood by the program are currently ARC, SIN, TAN and ZPN.  SIP is
not supported.  These restrictions could be lifted by transitioning to
using one of the standard WCS libraries, however beware that most of
the rest of the CASU pipeline toolkit has the same restrictions.
While a FITS WCS is expected in the input files, one that is incorrect
(or not correctly understood) should merely render the
.B ra
/
.B dec
/
.B bjd
/
.B airmass
/
.B ha
column values incorrect without further ill effect.
.PP
A number of additional FITS headers are expected to be present in the
input files.  In the following list, these are grouped into sets of
alternatives, only one of which need be present.  The preferred
keyword name is given first in each set, and they are searched in the
order given below.  All keywords are optional, although the default
behaviour (stated below) may be undesirable.
.TP
AIRMASS
.TP
AMSTART
Air mass.  Used only to correct the magnitude zero-point of the frame
for extinction for consistency with the way the photometric
calibration program does it.  If not present, does not correct for
extinction.  Note: an internal calculation is used to produce the
.B airmass
column in the output files so this FITS header does not affect it.
.TP
EXPTIME
.TP
EXPOSED
.TP
EXP_TIME
Exposure time, in seconds.  If not present, emits warning and does not
correct for exposure time.  Output files will contain zero in TEXP.
.TP
EXTINCT
Extinction in magnitudes per airmass.  See notes for MAGZPT, below,
for how this value is used.  If not present, a default is supplied
(this is from a hard-coded table for the INT/WFC for historical
reasons, namely that the header did not exist in files processed by
very old versions of the pipeline toolkit).
.TP
FILTER
.TP
WFFBAND
.TP
HIERARCH ESO INS FILT1 NAME
.TP
FILTER2
.TP
INSFILTE
Filter name.  Optional, used only in plot axis labels and for looking
up the default extinction value.  The names of the keywords for this
quantity are wildly inconsistent between different telescopes /
instruments. 
.TP
GAIN
.TP
HIERARCH ESO DET OUT1 GAIN
.TP
EGAIN
Reciprocal gain, in electrons per data number.  If not present,
assume unity and emit warning.
.TP
HEIGHT
.TP
OBSALT
.TP
ALT-OBS
.TP
SITEALT
.TP
HIERARCH ESO TEL GEOELEV
Observing site height above the geoid, in metres.  Assumed to be zero
if not present (DANGER!).
.TP
LATITUDE
.TP
OBSLAT
.TP
LAT-OBS
.TP
SITELAT
.TP
<NAME>
Observing site latitude, in degrees, following the standard North
positive convention.  Site coordinates are needed to calculate the
Barycentric position and velocity vectors of the observer for the
.B bjd
/
.B airmass
/
.B ha
calculations.  If LATITUDE and LONGITUD (or equivalents) are not both
available, the observer is assumed to be at the Geocentre in the
.B bjd
calculation, and
.B airmass
and
.B ha
are not computed.  The observing site location header keywords are
(sadly) yet another set that are wildly inconsistent between
different observatories, although they are at least usually in the
same units.
.TP
LONGITUDE
.TP
OBSLONG
.TP
LONG-OBS
.TP
SITELONG
.TP
<NAME>
Observing site longitude, in degrees, East positive.  See comments
under LATITUDE, above.
.TP
MAGZPT
.TP
ZMAG
Magnitude zero-point.  The CASU standard variant is MAGZPT and is for
an exposure time of one second and airmass of unity (and for mosaic
instruments, PERCORR=0).  ZMAG is a variant seen in UNSW APT data and
is for the exposure time of the frame and an airmass of unity.  If not
present, uses MAGZPT=25.0 (this is a somewhat randomly chosen value,
but happens to be a shockingly good match for a certain NASA exoplanet
hunting satellite) and emits a warning.
.TP
MJD-OBS
.TP
MJD
.TP
JD
Time stamp as Modified Julian Day or Julian Day, respectively.  The
program currently ignores the TIMESYS keyword and always assumes these
are in the UTC time-system.  If not present, uses 2000 January 1 at
12 hours UTC (aka 2000.0) and emits a warning.  The program will
run without time stamps available, but many outputs are meaningless
and no corrections for target star proper motion are performed.
.TP
PEDESTAL
Constant offset added to the counts in the image to prevent negative
numbers appearing and being clipped at zero.  Assumed to be zero if
not present.
.TP
PERCORR
Per-detector adjustment added in to the magnitude zero-point (the
MAGZPT values are for the whole file, not per extension).  Value in
magnitudes, default is zero for no correction.
.TP
SATLEV
.TP
SATURATE
Saturation level, assumed to be in counts.  This is compared to the
Peak_height to determine if a source is saturated.  SATURATE is
written by the classifier, so should be present in most input files;
the SATLEV form overrides this value and is used because the
classifier often cannot accurately estimate the saturation level,
particularly when there were no saturated sources on the frame.  In
MEarth, SATLEV originates from the FITS header rewriter, and is
written at the same time as GAIN and READNOIS.  The values were
derived by hand from the non-linearity data.  The
.B -s
command-line flag overrides the headers if given.  If not present and
the command line option was not given, uses 65535.
.PP
Due to the lack of standardisation, the header keyword names (and
contents) for these quantities can differ greatly between
observatories.  Some common variations are supported, but the amount
of changes required to support even just the limited number of
telescope and instrument combinations the program had been used on
eventually became excessive (for some quantities, there are almost as
many variations on the keyword name as there are telescopes), so the
preferred solution is now to translate the headers in the particular
flavour of input files in question into the preferred headers listed
above.  It is expected that this translation has already been
performed before the program is run.
.PP
The binary table columns needed are only those written by the standard
.B imcore_list
program and the classifier.  The Classification column is required to
be present in the master catalogue, but is not needed in the other
input files.  In addition to these columns, in the master catalogue,
the FITS header reader searches for a pair of optional columns named
.B PMRA
and
.B PMDec
which are assumed to be the proper motions of each source in rad/yr,
and are used to correct for proper motion when computing the sky
position of the source in each target frame for barycentering and the
airmass and hour angle columns.  They are assumed to be zero if not
present.
.SH OUTPUT FILES
Light curves are stored as FITS binary tables, with one row per
object.  The light curves themselves and several other time-dependent
quantities that are unique to each object are stored as vectors
(arrays) in cells of the table.  Quantities common to the entire frame
(e.g. observation times, exposure times, etc.) are stored into
numbered keywords in the FITS header (see BUGS, below).
.SS Header keywords
The
.B lightcurves
program adds a number of header keywords to the output that describe
the light curves, options used for processing, or are used to
communicate with the
.B update
program.  The following example shows these keywords:
.PP
.nf
NMEAS   =                  748 / Number of points in each lightcurve
NROWMAST=                 1246 / Number of rows in master catalogue
MJDBASE =        53326.0000000 / Base MJD for time axis
SATMAG  =              11.4264 / Approximate saturation magnitude
FLIM    =              22.0237 / Flux limit of reference catalogue
ZP      =              27.5753 / Zeropoint for magnitudes
UMLIM   =                12.00 / Upper mag limit for fit
LMLIM   =                15.00 / Lower mag limit for fit
THEOSKY =                    F / T theoretical sky noise, F empirical
POLYDEG =                    0 / Polynomial degree in fit
APSEL   =                    0 / Aperture used (0 = automatic)
APMODE  =                    3 / Aperture output mode
DOMERID =                    2 / Meridian flip removal?
REFFANG =             1.575403 / Reference file field angle
NSEGME  =                    1 / Number of segments
.fi
.PP
The ones most likely to be of interest for ``consumers'' of the file
format are NMEAS, MJDBASE and NSEGME.
.SS Header segment table
Information about the ``segments'' into which the light curve was
split during processing when using the
.B -m
flag are stored in into the header as numbered keywords for each
segment.  NSEGME gives the total number of segments.  These are
guaranteed to be populated and numbered contiguously.  In the common
case where there was only one segment and the
.B -m
flag was not used, these additional headers are the following:
.PP
.nf
SEGV1   =                   -1 / Segment 1 instrument version number
SEGD1   =                   -1 / Segment 1 instrument change date
SEGA1   =                    0 / Segment 1 angle
.fi
.PP
The use of ``segments'' for purposes other than meridian flips is
currently a MEarth-specific feature, as are the SEGV and SEGD numbered
headers.  SEGA gives a number in the same form as IANG, below.
.SS Header time-series vectors
A number of quantities that apply to all measurements in a frame are
stored into the header as numbered keywords, forming a vector.  These
are encoded as HHH\fIn\fR, where HHH is a header name and
.I n
is a number running from 1 to NMEAS.  This is similar to the scheme
used for the binary table headers (e.g. TTYP, TFORM, etc.)
themselves.
.PP
For example, the array of observation times is stored as follows:
.nf
TV1     =            1.1105781 / Time value for datapoint 1
TV2     =            1.1574578 / Time value for datapoint 2
...
TVn     =            1.9384262 / Time value for datapoint n
.fi
.PP
The following quantities are stored in this manner:
.TP
.B TV
Observation times.  These are in the same time-system as the input
files (presuming the headers are compliant, this is specified by
TIMESYS, but it is usually UTC), but with MJDBASE subtracted to get
the values to fit in less digits.  To recover the original MJD values,
add MJDBASE.
.TP
.B TEXP
Exposure times, in seconds, for each frame.  Copied from EXPTIME and
its variants in the input list-driven photometry files.
.TP
.B OFF
Median magnitude zero-point residual after applying differential
photometry.  It should be zero, but occasionally isn't when this
(robust) estimator returns a different result from the mean or
polynomial used to correct the frame zero-point.  The name is
misleading, this column is probably not what you'd think.  Not very
useful.
.TP
.B RMS
RMS scatter of the magnitude zero-point residuals after applying
differential photometry.  This quantity can be used to detect frames
where the zero-point correction didn't work very well, usually because
they were taken in non-photometric conditions.  As with most uses of
RMS relating to this software, it is actually computed using median
absolute deviation, scaled (by a factor of 1.48) to Gaussian RMS
equivalent.
.TP
.B EXTC
Differential photometry correction applied to the frame.  This is
expressed as the delta(magnitude) that was applied, so negative
numbers mean less light.  The name is misleading, it is -extinction.
This is probably what you wanted when you looked at OFF above.
.TP
.B SEE
FWHM of the stellar images in the frame, in pixels.  Another
misleading name since seeing is only one of many things which can
influence FWHM.  Copied from SEEING in the input list-driven
photometry files.
.TP
.B ELL
Average ellipticity of stellar images in the frame.  Copied from
ELLIPTIC in the input list-driven photometry files.
.TP
.B SKY
.TP
.B NOIS
Global sky level and noise for the frame.  Copied from SKYLEVEL and
SKYNOISE in the input list-driven photometry files.
.TP
.B FANG
Position angle of the frame, in radians.  Zero means right ascension is
parallel to the CCD x coordinate.
.TP
.B IANG
Nearest integer modulo 2 of position angle of this frame minus the
reference, wrapped to [0,2*pi) and divided by pi.  This quantity is
used for detecting meridian flips, where it will change from 0 to 1.
.TP
.B ISEG
Segment number to which this frame belongs.  Can be used in
conjunction with
.B -m
to fit for separate magnitudes for the star in each segment in data
analysis, or with
.B -mm
to allow for the effects of having already done so in the light curve
generation.
.TP
.B IUPD
Update number when the data point was added, numbering from 1 if the
point was added by the
.B update
program.  The value increments by one each time the
.B update
program is used to append new points to the light curve.  0 if the
point was in the original set and was written out by
.B lightcurves
meaning the frame was included in the original set of differential
photometry solutions used to select and characterise the comparison
stars.
.TP
.B LXX
.TP
.B LXY
.TP
.B LYX
.TP
.B LYY
.TP
.B LXD
.TP
.B LYD
These headers record a standard 6-coefficient linear transformation
from the pixel coordinates in the master frame to the pixel
coordinates in the target frame.  This transformation is derived
using all the stars detected in the frame, not just those considered
as comparison stars, so is written only if all of the stars in the
frame were processed (i.e. the
.B -c
option was not used).
.PP
A number of other, instrument-specific quantities may also be stored.
These are not yet documented.
.SS Table columns
The tables themselves contain the following columns:
.TP
.B x
X position of the object on the master frame, in pixels.
.TP
.B y
Y position of the object on the master frame, in pixels.
.TP
.B medflux
Median magnitude.
.TP
.B rms
Light curve RMS in magnitudes.  The value is actually computed using a
robust MAD estimator scaled to Gaussian RMS equivalent.
.TP
.B chisq
Chi squared of the light curve assuming a constant magnitude.
.TP
.B nchisq
Number of data points in
.B chisq
.TP
.B class
Source classification in the master frame.  This follows CASU
conventions, specifically: -1 = stellar; 0 = junk-like; 1 =
non-stellar; 9 = target.  Values of -2 and 2 are also used, these are
usually treated the same as -1 and 1 respectively.
.TP
.B bflag
Flag indicating the deblender was triggered in the master catalogue
for this object.  Indicates aperture flux
.I may
be contaminated by a close companion.
.TP
.B cflag
Count of the number of frames where bad or low-confidence pixels were
encountered while summing the aperture photometry.  I recommend using
the flags column instead.
.TP
.B sflag
Count of the number of frames where saturated pixels were encountered
while summing the aperture photometry.  I recommend using the flags
column instead.  See there for comments on the reliability of the
saturation flagging.
.TP
.B pointer
Row number in the input master catalogue.  This is invariant over
removal of rows from the output light curve file, for example by using
the
.B -c
option while processing the light curve, or using the CFITSIO extended
filename syntax, etc.  In the absence of other means to identify
specific objects, this column should be used.
.TP
.B offsets
.B Internal use only.
This column is used to pass state (specifically, the segment or
meridian flip offsets for each aperture) to the
.B update
program.
.TP
.B apnum
Number of the chosen aperture (numbering from zero in the same way as
the
.B -a
flag).
.TP
.B apradius
Radius of the chosen photometric aperture, in units of RCORE.
.TP
.B compok
Flag indicating whether the source was considered for use as a
comparison star.  To tell if the source was actually used as a
comparison star, see the
.B weight
column, below.
.TP
.B bjd
.B Vector.
Barycentric MJD in the TDB time-system.  This is stored as MJD to
reduce loss of accuracy.  Please don't forget the 0.5 in the
definition of MJD.
.TP
.B flux
.B Vector.
The light curve itself: magnitude as a function of time.  Can
be NULL (NaN in the FITS file) when no measurement was available.
.TP
.B fluxerr
.B Vector.
Uncertainties in the magnitudes.  Uses a standard CCD noise model
including Poisson error in the target, sky noise, scintillation, and
the error in the mean (or polynomial fit) of the magnitude zero-point
correction.  This model is known to (as usual) underestimate the true
uncertainties in most ground-based data.
.TP
.B xlc
.B Vector.
X pixel coordinate time-series.
.TP
.B ylc
.B Vector.
Y pixel coordinate time-series.
.TP
.B airmass
.B Vector.
Air mass time-series.  Can be used for detecting and correcting
residual (e.g. colour-dependent) extinction not removed by standard
differential photometry.
.TP
.B ha
.B Vector.
Hour angle time-series in radians.  Please do not attempt to use
for detecting meridian flips, it is not reliable for this purpose
because it is possible to take frames on the ``wrong'' side of the
meridian on some equipment.
.TP
.B weight
.B Vector.
Weight of this source in the comparison star solution.  Zero if not
included.
.TP
.B sky
.B Vector.
Local sky level in counts.
.TP
.B peak
.B Vector.
Peak counts in the source, including sky.
.TP
.B flags
.B Vector.
Flags for each data point.  This is encoded as a bitmask from the
following values: 1 = no data point (star was not on the detector); 2
= aperture contains bad pixels; 4 = saturated.  Note that the
reliability of saturation flagging depends critically on the accuracy
of the saturation level assumed, and mildly saturated data may be
still be usable for many purposes depending on the requirements.
To detect data points that are
.I guaranteed
to be bad, check if (flags & 3) is non-zero, where & is the bitwise
AND operator.
.TP
.B ra
ICRS right ascension of the source on the master frame, in radians.
.TP
.B dec
ICRS declination of the source on the master frame, in radians.
.TP
.B pmra
Assumed sky-projected proper motion in right ascension, in arcsec/yr.
This is copied from the input file.
.TP
.B pmdec
Assumed proper motion in declination, in arcsec/yr.  This is copied
from the input file.
.TP
.B refmag
Magnitude of the source on the master frame, used to normalise the
light curves.  Following CASU conventions this is normally computed in
aperture 0 (the smallest), however note that for MEarth the program is
compiled to use aperture 2 instead.
.PP
In the table above,
.B vector
indicates that the column is a time-series vector, of length NMEAS.
.PP
If multiple apertures were processed, the \fBmedflux\fR, \fBrms\fR and
\fBoffsets\fR columns are for the chosen aperture, and additional
columns named \fBmedflux1\fR, \fBmedflux2\fR, etc. are written out for
each original aperture.  This is also done for the \fBflux\fR,
\fBfluxerr\fR, and \fBweight\fR columns unless the
.BI "-a " "sel"
option is used to suppress the output.
.SH EXAMPLE
This example shows the full end-to-end process of generating light
curves from a set of reduced images of the same field.  It uses
programs found in CASUTOOLS.  The FITS headers are assumed to already
be massaged to a suitable form (see
.B INPUT FILES
above) and to contain sufficiently accurate WCS solutions in one of
the supported projections.
.PP
Suppose the image files are named
.BI "target" "0001" ".fit"
to
.BI "target" "nnnn" ".fit"
and reside in the current directory, along with a confidence map
.B conf.fits
which may have already been used to generate catalogues and obtain the
WCS solutions.  This naming convention has been in use for a while at
CASU and intentionally gives images a different filename extension
from everything else to avoid confusion and accidents.  However this
is purely optional.
.PP
First, we need to choose (and create, if necessary) a master
catalogue.  Usually a set of catalogues were used in making the WCS
solutions, and provided these used suitable
.B imcore
parameters one of them can simply be reused as the master for the
photometry.  However, here we'll assume we need to make a new one.
Another reason to do this might be severe background variations
(e.g. nebulosity) causing large numbers of false source detections, in
which case the nebuliser might be used to preprocess the input file
before running
.B imcore
on it (as an aside: in this case, I suggest then using
.B imcore_list
to make a list-driven file out of the
.B original
non-nebulised image, using the nebulised catalogue to place the
apertures, and then using this list-driven file as the master).
.PP
I usually symlink
.B master.fit
to the image I'm using as the (single) master, but some may prefer to
stack a few good images.  I tend to use the SEEING and ELLIPTIC
headers to find the best image.
.PP
We then generate the master catalogue and classify it:
.PP
.nf
  imcore --rcore=3 --cattype=1 master.fit conf.fits master_cat.fits 4 2
  classify master_cat.fits
.fi
.PP
the
.B imcore
parameters are purely illustrative and will need to be changed for the
specific application.  I usually suggest using a higher threshold (the
second number), than one would for ``standard'' survey applications,
because the faintest objects don't usually produce useful light curves
and serve only to slow down processing and waste space in the output
files.  Note that I recommend always specifying the catalogue type,
the default has a had a habit of changing as new versions have been
released.
.PP
In this example, I will use the WCS to align the input images and
place apertures (this is not always a good idea, e.g. if the WCS
solutions are a bit sketchy, or none are available).  To make the list
driven files (and classify them) for all the images automatically, the
following shell script is used:
.PP
.nf
  #!/bin/sh
  
  for imfile in target????.fit; do
      listfile=`echo "$imfile" | sed -e 's/\.fit$/_list.fits/g'`
      imcore_list --rcore=3 --cattype=1 "$imfile" conf.fits master_cat.fits "$listfile" 2
      classify "$listfile"
  done
.fi
{'fraction_non_alphanumeric': 0.04226029735853742, 'fraction_numerical': 0.0056802309339789555, 'mean_word_length': 3.7704893758791735, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1823417', 'n_tokens_mistral': 18351, 'n_tokens_neox': 17160, 'n_words': 10544}