P-band calibration and editing in <img src="cmsy10-41.png" alt="A" class="10x-x-41" /><img src="cmsy10-49.png" alt="I" class="10x-x-49" /><img src="cmsy10-50.png" alt="P" class="10x-x-50" /><img src="cmsy10-53.png" alt="S" class="10x-x-53" />

Download your data set from the archive found at
https://data.nrao.edu/
making sure that you select SDM-BDF data set (all files) as the download data format.

Before starting AIPS and while in the terminal window you intend to use for AIPS, enter

> cd my_data_directory C R

to change to the data directory containing the top-level directory of your data set.

> export MYAREA=‘pwd‘ C R

for bash shells, or

> setenv MYAREA ‘pwd‘ C R

for tcsh and c shells.

Note the back ticks around pwd which cause the output of pwd to appear in the typed line.

Start AIPS with, for example,

> aips tv=local pr=3 C R

to use Unix sockets for the TV and printer 3

and enter your user number when prompted.

Look at the SDM-BDF file contents with BDFLIST — make sure you do not have a quote at the end of the file name, or else the lower case characters get changed into upper case and the file won’t be found.

> default bdflist C R

initialize all adverbs

> dowait 1; docrt 1 C R

to get output on terminal as well as log file.

> asdmfile(1) = ’MYAREA: C R

two parts are concatenated.

> asdmfile(2) = ’TSUB0001.sb23643640.eb23666175.56454.14034020834 C R

> bdflist C R

to examine file contents.

Pay particular attention to the configuration numbers and number of spectral channels. These refer to each unique correlator and antenna setup, not just the size of the current VLA. Each configuration and number of channels that you want will need to be loaded individually.

Load data into

with BDF2AIPS — We put everything on disk d because that’s where there’s more room. Note, do not use DEFAULT if you wish to retain the adverbs from the previous step. Loading data may take a while depending on the size of your data.

> outn ’my-name’ C R

to select a meaningful file name.

> config c C R

to select configuration c.

> outdi d C R

to select the disk.

> bdf2aips C R

to translate the data into .

> nchan N C R

to limit to data with N spectral channels; some spectral-line configurations can contain more than one number of channels but AIPS can only load one number at a time.

Do this as often as needed to load all desired configurations. Spectral-line set-ups can have spectral windows with different numbers of channels within the same configuration. can only handle one at a time, so specify nchan N C R to load the data with N channels and repeat as needed.

Look at header of the

file written by BDF2AIPS

> indi d; getn 1 C R

to select disk and catalog number, assuming you had no files on disk d previously.

> imh C R

to see the details of the uv-data set header.

look at where the telescopes were during your observation with PRTAN

> default prtan C R

> indi d; getn 1 C R

> docrt = -1 C R

to make a printer listing to keep at your terminal.

> go prtan C R

Look at the observation summary with LISTR

> default listr C R

> indi d; getn 1 C R

> opty ’scan’ C R

to list scans and sources.

> docrt = -1 C R

to make a printer listing to keep at your terminal.

> go C R

At this stage, Minnie’s summary recommends separating P-band from 4-band data. At present, only P-band is recorded so that step is not needed. Then, Minnie recommends using POSSM to find dead antennas plus those with crossed polarization. Then she recommends UVFLG to flag the dead ones and FIXRL to uncross the mis-wired ones. These problems have been essentially eliminated. In addition, P-band data are now delivered by default in a number of relatively narrow spectral windows (called IFs in ) which helps minimize data lost to RFI and eliminates the NOIFS and MORIF steps from Minnie’s guide.

However, the requantizer gains are now allowed to vary and must be corrected.

> default tyapl C R

to select the needed task

> indi d; getn 1 C R

to select the input file.

> optype ’PGN’ C R

to select requantizer gain correction.

> inext ’SY’ ; in2ver = 1 C R

to apply the SysPower table, the version number is not defaulted.

> outdis d ; go C R

to stay on one disk.

Narrow-band RFI will normally cause ringing in P-band data. Use POSSM to examine some baselines from a calibration source to check this if you want to be careful. The default setup delivers data from outer IFs for which the sensitivity of the P-band receiver is poor. Use SPLAT to drop the insensitive IFs (those centered on 232, 248, 264, and 472 MHz) and to do Hanning smoothing to reduce the ringing.

> default splat C R

to select the needed task

> indi d; getn 2 C R

to select the (new) input file.

> flagver 1 C R

to apply the on-line flags.

> smooth = 1, 0 C R

to do Hanning smoothing once and for all.

> bif = 4; eif = 15 C R

to select the more sensitive IFs.

> outdis d ; go C R

to put the output on the same disk.

There is also a Navy satellite that makes a mess of data from 360 MHz to 380 MHz. Delete this with UVFLG in one of 2 ways. For the standard setup, you can specify the channels as

> default uvflg C R

> indi d; getn 3 C R

> bif 6 ; eif 6; bchan 65; echan 128; C R

> go C R

> bif 7 ; eif 7; bchan 1; echan 97; C R

> go C R

For any setup, you may make a text file containing the line
BFREQ = 360 EFREQ = 380 /
and then use it with UVFLG as

> default uvflg C R

> indi d; getn 3 C R

> intext = ’MYWORK:pband.flg C R

to give file name

> go C R

Edit UV data with EDITA and the SY table. The SY table P_sum parameter is sensitive to the occasional bursts of RFI which render the data useless. Flag all seriously discrepant points but do not flag really tightly since noise and even abnormalities in this parameter are “normal.”

> default edita C R

> indi d; getn 3 C R

> inext ’SY’ C R

> go C R

At this time, it is best to identify those spectral channels affected by RFI throughout the run and to flag them once and for all. FTFLG will display the data combining all included baselines in a single display (per polarization), while the very similar SPFLG allows much better discrimination (and more work) by displaying the baselines separately. Be sure to flag all sources if you flag channels over all times.

> default ftflg C R

> indi d; getn 3 C R

> calcode ’*’ C R

to examine calibrators only

> dparm(6) = x C R

to specify the basic integration time in the data in seconds.

> go C R

If you use SPFLG, you should limit the times, sources, and/or baselines examined. The flagging then is best done with UVFLG from notes made while you study the data with SPFLG.

> default uvflg C R

> indi d; getn 3 C R

> bif = bi ; eif = ei C R

to set range of spectral windows.

> bchan = bc ; echan = ec C R

to set range of spectral channels.

> antennas = a1,a2,a3 C R

to select antennas to flag.

> baseline = b1,b2,b3,b4 C R

to limit flags to the baselines from the pairs a1 through a3 with b1 through b4.

> reason ’bad channels’ C R

to set a reason in the flag table.

> go C R

and repeat as needed with different IFs, channels, antennas, etc.

This step is optional and should be considered only if you have several P-band data sets to reduce. There is a special capability in

developed for lowband data. The task FGTAB was written to read a flag table and write to a text file those flags that apply to all times, sources, antennas, subarrays, and frequency IDs. The channel specifications are written in frequency units which enables UVFLG to read the text file and apply the same flags to the channels to which they pertain in the second data set. Of course, if the two data sets have identical frequency structure, then one could simply copy the FG table from one data set to the other. Task TACOP would be used if you want the flag file by itself. TAPPE would be used if you want to append the flag table of the first file to one of the second. Note that these two tasks transfer all flags in the input table, not just those that flag the general RFI environment.

Calibrate for the delay errors with FRING. Choose a time range in the middle of the observation where you know the antennas have settled down. A short time range is all you need — look at LISTR — because you don’t want time variations to come into play. This is also why you should choose something bright — like 3C286, which is used here as the name for the generic fringe- and bandpass-fitting calibration source. It would be wise to check these data for RFI with POSSM before running FRING.

> default fring C R

> indi d; getn 3 C R

> calso ’3C286’ ’ C R

to select the primary calibration source.

> timerang 0 3 40 0 0 3 40 55 C R

to select < 1 minute of good data from the calibration source.

> solint 1 C R

to average over all included times.

> aparm(5) 1 ; aparm(6) 1 C R

to fit a single delay over all IFs and to print some extra detail in the fitting.

> dparm(9) 1 C R

to avoid any fitting for rates — important.

> refant 22 C R

to set a reference antenna, if one is known to be better than others.

> go C R

This print level will display the delays found, which should all be a few nano-seconds although larger (correct) delays sometimes arise.

Apply these delay solutions to the calibration (CL) table with CLCAL

> default clcal C R

> indi d; getn 3 C R

> calso ’3C286’,” C R

> snver 1 C R

to specify the SN table version written by FRING.

> go C R

to copy CL table 1 to version 2 adding the delay correction, taken to be constant in time.

You should flag out any bad data on your bandpass calibrator. Some users feel that a quick-and-dirty flagging is enough at this stage. Use CLIP with a carefully chosen, perhaps IF-dependent, cutoff level. A more detailed editing is done by other users. Here are options to use RFLAG and SPFLG. Maybe you’d like to RFLAG first and then eyeball with SPFLG. The options are endless! The first execution of RFLAG determines flagging level parameters and makes plots on the TV for you to be sure that they are reasonable, The second execution applies these levels and determines new levels in case you want to run RFLAG even more times.

> default rflag C R

> indi d; getn 3 C R

> sources(1) ’3C286’ C R

> docal 1 C R

to correct delays and a-priori gains.

> avgchan 11 C R

to set the width of the median-window filter run across the spectral channels in each IF

> fparm = 3, x, -1, -1 C R

to do a small rolling time buffer, set the sample interval, and to have NOISE and SCUTOFF set IF-dependent cutoff levels.

> fparm(13) = 1000 C R

to clip amplitudes at 1000 Jy.

> doplot 12 ; dotv 1 C R

to do the 2 most useful plots on the TV.

> go C R

RFLAG will return cutoff levels to the appropriate adverbs. Therefore, to apply the flags using the flagging levels found above, do not do a tget rflag which will reset the levels to zero. Instead, simply do

> doplot = -12 C R

to apply the cutoff levels and determine new ones, plotting on the TV.

> go C R

RFLAG will make a new flag table each time it determines flags (when DOPLOT≤ 0). RFLAG has numerous FPARM options you may choose. FPARM(9) and FPARM(10) scale the rms for NOISE and SCUTOFF that RFLAG computes. If the fraction of a channel that is flagged exceed FPARM(7), the entire channel is flagged. FPARM(6) is especially useful at P-band as it allows channels adjacent to an RFI peak to be flagged. Similarly, FPARM(8) allows channels between strong RFI peaks to be flagged. Read the help file for detailed information. Flagging calibrator scans aggressively is recommended especially for spectral-line observations.

Check the results with

> default spflg C R

> indi d; getn 3 C R

> sources(1) ’3C286’ C R

to examine only the bandpass calibrator.

> dparm(6) = x C R

to set sampling time in seconds

> docal 1 C R

to correct delays and a-priori gains.

> go C R

It is at this point that experienced users differ significantly in their approach. If you have only one bandpass-calibration scan and the ionosphere was reasonably behaved (phases nearly constant through that scan), then you should run BPASS on the calibration source in place. However, when one has multiple bandpass scans and/or a badly behaved ionosphere, it may be best to split the calibration scans out and self-calibrate them on a short time interval. The addition of multiple scans where the source has been phased up, but the remaining RFI has not, will reduce the effect of that RFI substantially. This scheme is described next.

Split off your calibrator for bandpass calibration — this is only necessary for self-calibrating the BP calibrator.

> default split C R

> indi d; getn 3 C R

> outdi indi ; source ’3C286’ ’ C R

to select calibrator and stay on the same disk.

> outcl ’bpspl’ C R

to indicate the use for this file.

> docalib 1 C R

to apply delay and a-priori calibration.

> go C R

To make a single-source file.

Self-cal on your bandpass cal with CALIB to solve for wiggles in phase as a function of time.

> default calib C R

> indi d; getn 4 C R

> refant 22 C R

to stay with the chosen reference antenna.

> solint y C R

to average n records in each solution interval, set y = nx.

> solty ’L1R’ ; solmode ’P’ C R

to do phase-only solutions.

> aparm(6) 1 C R

to get some diagnostic messages.

> go C R

to write a new SN table.

Eyeball this new table with SNPLT — should be flat, not all over the place. But EDITA lets you look at more things at once.

> default edita C R

> indi d; getn 4 C R

> inext ’sn’ ; dotwo 1 C R

to plot the phases in the new table.

> crowded 1 ; do3col 1 C R

to combine all IFs in each colorful plot.

> go C R

If these plots are too crowded, you can interactively select individual IFs and polarizations. This task will let you flag data if desired. If you do, then you should re-run CALIB.

Use BPASS to calibrate the bandpass shape — corrects for wiggles in phase and amplitude as a function of frequency

> default bpass C R

> indi d; getn 4 C R

> docalib 1 C R

to apply the new SN table phases.

> refant 22 C R

> solty ’l1r’ ; solint = -1 C R

to average the entire time range.

> bpassprm(5) = 1; bpassprm(10) = 3 C R

to normalize the solutions at the end rather than the data record-by-record.

> go C R

To generate a new BP table containing one record per antenna.

Eyeball the resulting bandpass (BP) table — APARM(8) = 2 plots the BP table in POSSM. If you see odd bandpass shapes or spikes in the bandpass, you may need to edit your data more extensively and repeat steps 17 through 20.

> tget possm C R

> indi d; getn 4 C R

> solint = -1 C R

> aparm(8) 2 ; aparm(9) 1 C R

to plot all IFs together for each bandpass.

> nplots 2 C R

2 per page is enough here.

> go C R

Alternatively, BPEDT lets you use the bandpass table to guide additional flagging of the bandpass calibrator(s). It is similar to EDITA in that it lets you look at multiple antennas at the same time for comparison.

> default bpedt C R

> indi d; getn 4 C R

> inext ’bp’ ; GO C R

Copy the bandpass table back to your main data with TACOP.

> default tacop C R

> indi d; getn 4 C R

> inext ’bp’ C R

> outdi indi ; geton 3 C R

to select output name parameters

> go C R

Since RFLAG probably made a large number of flags, let us copy the data file with UVCOP and apply the flags.

> default uvcop C R

> indi d; getn 3 C R

> outdi indi ; outn inna C R

to stay on d.

> outcl ’fixcp’ C R

with a meaningful class name

> flagver n C R

to apply the highest flag table (n); use IMHEAD if unsure. Version 0 means no flagging here.

> go C R

Use SETJY to compute the flux for the primary flux calibrator (3C286), storing it in the SU table.

> default setjy C R

> indi d; getn 5 C R

> source ’3C286’ ’ C R

specify all “known” calibrator sources.

> opty ’calc’ C R

to calculate and display fluxes from the frequencies plus tables of known sources.

> go C R

Go through all of your calibration sources, one at a time, to check for RFI with POSSM and, if present, to address it with SPFLG and/or RFLAG

> tget possm C R

> source ’calsrc_i’ ’ C R

to examine the i^th calibration source.

> indi d; getn 5 C R

> docal 1 ; doband 1 C R

apply our hard-won calibration.

> solint 0 C R

to look at the average over all time.

> aparm(9) 1 C R

to plot all IFs together.

> nplots 2 C R

2 per page is enough here.

> go C R

If RFI is present (it almost certainly will be), two rounds or more of RFLAG may be needed.

> default rflag C R

> indi d; getn 5 C R

> source ’calsrc_i’ ’ C R

to examine the i^th calibration source.

> docal 1 ; doband 1 C R

apply the calibration.

> avgchan 11 C R

to set the width of the median-window filter run across the spectral channels in each IF

> fparm = 3, x, -1, -1 C R

to do a small rolling time buffer, set the sample interval, and to have NOISE and SCUTOFF set IF-dependent cutoff levels.

> fparm(13) = 1000 C R

to clip amplitudes at 1000 Jy. There are many more FPARM options you may choose also.

> doplot 12 ; dotv 1 C R

to do the 2 most useful plots on the TV.

> go C R

RFLAG will return cutoff levels to the appropriate adverbs. Therefore, to apply the flags using the flagging levels found above, do not do a tget rflag which will reset the levels to zero. Instead, simply do

> doplot = -12 C R

to apply the cutoff levels and determine new ones, plotting on the TV.

> go C R

RFLAG will make a new flag table each time it determines flags (when DOPLOT≤ 0). It may be good to check what RFLAG has done with POSSM. If more edits are needed, use SPFLG

> default spflg C R

> indi d; getn 5 C R

> source ’calsrc_i’ ’ C R

to examine the i^th calibration source.

> dparm(6) x C R

> docal 1 ; doband 1 C R

apply the calibration.

> go C R

Since RFLAG probably made a large number of flags, let us copy the data file once again with UVCOP and apply the flags.

> default uvcop C R

> indi d; getn 5 C R

> outdi indi ; outn inna C R

to stay on d.

> outcl ’flaged’ C R

with a meaningful class name

> flagver n C R

to apply the highest flag table (n); use IMHEAD if unsure. Version 0 means no flagging here.

> go C R

Since all flag tables < n were already merged into table n which we have applied, delete all flag tables in the output file.

> default extdest C R

> indi d; getn 6 C R

> inext ’fg’ ; invers = -1 C R

to delete all remaining versions of flag table.

> extdest C R

Find the amplitude and phase calibration as a function of time using CALIB

> default calib C R

> indi d; getn 6 C R

> calcode ’*’ C R

to select all calibration sources

> docalib 1 ; doband 1 C R

to apply all existing calibration.

> refant 22 C R

to keep our reference antenna.

> solint 0 C R

One solution per calibrator scan is okay with good short-term phase stability. Otherwise, set to the longest time in minutes over which phase is stable.

> solty ’L1R’ ; solmode ’a&p’ C R

to solve for amplitude and phase with an L1 method iterated robustly.

> aparm(6) 1 C R

to print closure error statistics.

> go C R

Pay attention to the fraction of failed solutions. If the fraction is large, try adjusting parameters such as SOLINT, etc.

The solution table (SN) produced by the previous step will contain amplitude gains at multiple levels, correct ones for those sources with fluxes in the source table (3C286) and incorrect ones for those which CALIB was forced to call 1 Jy. To solve for the unknown fluxes of the secondary calibrators

> default getjy C R

> indi d; getn 6 C R

> source ’calsrc₁’, ’calsrc₂’, ’calsrc₃’, … C R

to list the secondary calibrators.

> calsour = ’3c286’ ’ C R

to list the known calibrator(s).

> snver = n C R

to select only the SN table just written by CALIB. Use IMHEADER to find the maximum version number n.

> go C R

to solve for the secondary fluxes as functions of IF.

Check the values displayed carefully to make sure that they are sensible. SOUSP offers tools to examine and adjust the fluxes and SN table n.

Look at both amplitude and phase solutions with EDITA or SNPLT.

> tvini C R

> default snplt C R

> indi d; getn 6 C R

> inext ’sn’ ; opty ’amp’ C R

to plot amplitudes.

> nplots 4 ; dotv 1 C R

to plot 4 panels on each TV page.

> opco ’alif’ ; do3col 1 C R

to plot all IFs in each panel using color to distinguish them.

> go C R

One can also plot IFs in separate panels or one at a time if these plots are too crowded. Then

> opty ’phas’; go C R

To check the phases.

To use EDITA to look at both amplitude and phase at the same time for multiple antennas

> default edita C R

> indi d; getn 6 C R

> inext ’sn’ ; dotwo 1 C R

to plot the phases in the new table.

> crowded 1 ; do3col 1 C R

to combine all IFs in each colorful plot.

> go C R

If these plots are too crowded, you can interactively select individual IFs and polarizations. This task will let you flag data if desired. If you do, you should then re-run CALIB.

Apply the solutions to the calibration table with CLCAL

> default clcal C R

> indi d; getn 6 C R

> calcode ’*’ C R

to select all calibration sources

> sampty ’box’ ; bparm 0.3, 0.3 C R

To smooth the SN table if you used SOLINT less than full scans. Leave this out if you used scan averages.

> refant 22 C R

> go C R

Use SNPLT to check that the solutions have been interpolated between calibrators.

> tget snplt C R

> inext ’cl’ C R

> go C R

Make an image of 3C286 as a sanity check using IMAGR

> default imagr C R

> indi d; getn 6 ; outdi indi C R

> docal 1 ; doband 1 C R

to apply the calibration.

> source = ’3c286’ ’ C R

to select only the primary calibrator.

> bchan 5; echan = n - 4 C R

to omit edge channels in each IF, where n is the number of channels in an IF, now usually 128.

> nchav = echan-bchan+1; chinc nchav C R

to average all spectral channels into the image.

> cellsize 1. C R

to set the image cell spacing — depends on configuration.

> imsi 1024 C R

to make a largish image to see sources around 3C286.

> outn ’3C286_quick’ C R

> niter 100 ; dotv 1 C R

to do some Cleaning and guide the progress with the TV

> go C R

If you wish to image Stokes Q and U, it is now necessary to calibrate the polarization. This is still somewhat of a research question in

, but the advice in §4.3.16 may work. You will need to run procedure VLATECR to correct for Faraday rotation and dispersion, task RLDLY to correct for the V minus H delay offset, task PCAL with SPECTRAL 1 to calibrate the polarization D terms, and then task VHDIF to calibrate the V minus H phase offset which is also a function of spectral channel. PCAL is best run with an unpolarized calibration source which is common at P band. However, VHDIF requires a calibration source with a useful flux in Stokes U. 3C345 has this, but, due to the rotation measure, not at all frequencies. Significant frequency smoothing may help. Task VHMOD models a point source with linear polarization which may help you model your source.

At this point, it may be simpler to SPLIT the data set and apply calibration to the target sources which we then can work with one at a time.

> defsult split C R

> indi d; getn 6 C R

> outdi indi C R

> docal 1 ; doband 1 C R

> dopol 1 C R

if you calibrated polarization

> sources = ’target₁’, ’target₂’, … C R

to select only target sources.

> go C R

To create a number of separate, single-source uv data sets each with separate target source.

Flag RFI and bad data out of the target data set with two or more rounds of RFLAG

> default rflag C R

> indi d; getn 9 C R

> avgchan 11 C R

Median window width on spectra.

> fparm = 3, x, -1, -1 C R

to do a small rolling time buffer, set the sample interval, and to have NOISE and SCUTOFF set IF-dependent cutoff levels.

> doplot 12 ; dotv 1 C R

To examine the two important plots on the TV.

> go C R

and then apply the flags levels and determine new ones

> doplot -12 ; go C R

and perhaps repeat this line for more flagging. NOTE: for flagging spectral-line target sources, you should set FPARM(4)=0 since you should not flag in the spectral domain. Check what RFLAG has done with SPFLG, which can be tedious.

> default spflg C R

> indi d; getn 9 C R

to select the first of the split target data sets.

> dparm(6) x C R

to set integration time.

> go C R

Because we used TYAPL only to apply the re-quantizer gains, your data weights simply reflect integration time. For better imaging, weights should reflect the actual data uncertainties. So re-weight your data with REWAY

> default reway C R

> indi d; getn 9 C R

> outdi indi C R

> aparm 31,0,500 C R

to find rmses from rolling buffers 31 time intervals long on a baseline basis and then smooth them over 500 seconds.

> go C R

Note that this applies the final SPFLG+RFLAG flag table. Delete the other flag table versions with

> default extdest C R

> indi d; getn 10 C R

> inext ’fg’ ; invers = -1 C R

to delete all remaining versions of flag table.

> extdest C R

[front] [up]

L.1 P-band calibration and editing in