Data Model

ccd.py

Data model for Euclid-VIS CCDs (ground testing at MSSL).

VIS CCDs have 4 quadrants (E,F,G & H). Each quadrant has 2048 (hor.) x 2066 (ver.) active pixels. To these we have to add 51 columns of prescan and 20 columns of overscan (29 in flight/FM). There are also 20 lines of parallel overscan. These pre and over scan pixels are virtual.

The main class in this module, CCD, allows to load, store, manipulate and save images generated by the instrument ROEs and saved in FITS format by the acquisition software, ELVIS.

History

Created on Fri Nov 13 17:42:36 2015

author

Ruyman Azzollini

class vison.datamodel.ccd.CCD(infits=None, extensions=None, getallextensions=False, withpover=True, overscan=20)

Class of CCD273 objects.

Input are Euclid Images as acquired by ELVIS software (Euclid LabView Imaging Software).

The class has been extended to handle multi-extension images. This is useful to also “host” calibration data-products, such as Flat-Fields.

A note on Coordinates Systems:

• ‘CCD’: referenced to the first pixel readout from channel H. All 4 quadrants

  are in a single array, their detection nodes in the 4 “corners” of the rectangle. Same system as images are displayed on DS9. In clock-wise sense, quadrants are H (bottom-left), E (top-left), F (top-right), and G (bottom-right).

• Physical: same as ‘CCD’ but takes into account virtual and non-active pixels. It is the closest to measuring real “distances” on the silicon.

• ‘Quadrant-canonical’: Quadrant coordinates system in which the first pixel is the first pixel read out (closest pixel to the readout node), and the last is the last readout. In this system, the serial pre-scan comes before the image area, and this before the serial overscan. Parallel overscan comes after image area in the parallel direction. In this system, coordinates of pixels across quadrants, for a single readout, correspond to the same point in time. Useful when doing cross-talk analysis, for example.

• ‘Quadrant-relative’: quadrant coordinates system with the same relative orientation as in the ‘CCD’ system, but referenced to the ‘lower-left’ pixel of the given quadrant in such system. In this system, the readout node is in a different corner for each quadrant: lower-left for H, top-left for E, top-right for F and bottom-right for G.

add_extension(data, header=None, label=None, headerdict=None)

Appends an extension to self (extensions are in a list).

add_to_hist(action, extension=- 1, vison='v1.1+19.g64b0cf3', params={})

Adds information to historial of operations applied on object.

cooconvert(x, y, insys, outsys, Q='U')

Coordinates conversion between different systems.

del_extension(ixextension)

Deletes an extension from self, by index.

divide_by_flatfield(FF, extension=- 1)

Divides CCD image by a Flat-field.

do_Vscan_Mask(VSTART, VEND)

Returns a vertical scan mask.

dummyrebin(arr, new_shape, stat='median')

extract_region(Q, area='img', vstart=0, vend=2086, Full=False, canonical=True, extension=- 1)

flip_tocanonical(array, Quad)

Reorients an array to canonical orientation, according to quadrant. Assuming the array is in “relative” orientation

get_1Dprofile(Q, orient='hor', area='img', stacker='mean', vstart=0, vend=2086, extension=- 1)

get_Q(x, y, w, h)

get_cutout(corners, Quadrant, canonical=False, extension=- 1)

Returns a cutout from the CCD image, either in canonical or non-canonical orientation.

Parameters

• corners (list (of int)) – [x0,x1,y0,y1]

• Quadrant (char) – Quadrant, one of ‘E’, ‘F’, ‘G’, ‘H’

• canonical (bool) – Canonical [True] = with readout-node at pixel index (0,0) regardless of quadrant. This is the orientation which corresponds to the data-readin order (useful for cross-talk measurements, for example). Non-Canonical [False] = with readout-node at corner matching placement of quadrant on the CCD. This is the orientation that would match the representation of the image on DS9.

• extension (int) – extension number. Default = -1 (last)

get_mask(mask)

Loads a mask into the extensions.

get_quad(Quadrant, canonical=False, extension=- 1)

Returns a quadrant in canonical or non-canonical orientation.

Parameters

• Quadrant (char) – Quadrant, one of ‘E’, ‘F’, ‘G’, ‘H’

• canonical –

Canonical [True] = with readout-node at pixel index (0,0) regardless of quadrant. This is the orientation which corresponds to the data-reading order (useful for cross-talk measurements, for example). Non-Canonical [False] = with readout-node at corner matching placement of quadrant on the CCD. This is the orientation that would match the representation of the image on DS9.

Parameters

extension (int) – extension number. Default = -1 (last)

get_region2Dmodel(Q, area='img', kind='spline', splinemethod='cubic', pdegree=2, doFilter=False, doBin=True, filtsize=1, binsize=1, filtertype='mean', recoveredges=False, vstart=0, vend=2086, canonical=True, extension=- 1)

Parameters

• ccdobj (object) – ccd object

• Q (char) – Quadrant

• kind (char) – type of interpolation

• doFilter (bool) – boolean to control whether to apply image filtering or not

• filtsize (int) – size of the filter to be applied (on a side)

• doBin (bool) – apply binning?

• binsize (int) – size of the binning window (on a side)

• filtertype (char) – type of the filter, if applied

• recoveredges (bool) – try to recover edges of array after binning / filtering?

• vstart (int) – windowing parameter, start line (from 0) to consider in image area

• vend (int) – windowing parameter, end line (+1) to consider in image area

• canonical (bool) – [quadrant] orientation of the resulting model

• extension (int) – image extension to be used from ccdobj

get_stats(Quadrant, sector='img', statkeys=None, trimscan=None, ignore_pover=True, extension=- 1, VSTART=0, VEND=2086, clip=None)

get_tile_coos(Quadrant, wpx, hpx, noedges=False)

Returns a dictionary with a tiling [coordinates of corners of tiles] of quadrant Q, with tiles of size wpx[width] x hpx[height].

CAUTION: Returned coordinates are Q-relative.

Parameters

• Quadrant – str, Quadrant, one of [‘E’,’F’,’G’,’H’]

• wpx – int, width [along NAXIS1] of tiles, in pixels.

• hpx – int, height [along NAXIS2] of tiles, in pixels.

Returns

tiles_dict = dict( wpx=’Width of tiles, integer’, hpx=’Height of tiles, integer’, llpix=’Lower left corner of tiles, list of tuples’, ccpix= ‘Central pixel of tiles, list of tuples’, Nsamps=’Number of tiles, integer’)

get_tiles(Quadrant, tile_coos, extension=- 1)

Returns cutouts from Quadrant using the coordinates in tile_coos.

Parameters

• Quadrant (str) – Quadrant where to take the cutouts from.

• tile_coos (dict()) – A dictionary with tiles coordinates, as output by get_tile_coos.

• extension (int) – extension to consider, by index. Last is -1.

Returns

A list with the cutouts.

get_tiles_stats(Quad, tile_coos, statkey=None, estimator=None, extension=- 1, clipsigma=- 1)

Returns statistics on a list of tiles.

Parameters

• Quad (str) – Quadrant where to take the cutouts from.

• tile_coos (dict()) – A dictionary with tiles coordinates, as output by get_tile_coos.

• statkey (str) – stat to retrieve (one of mean, median, std)

• estimator (obj) – function to retrieve stat (alternative to statkey)

• extension (int) – Extension index, last is -1

• clipsigma (float/int) – apply sigma clipping of value +-clipsigma if clipsigma>0

Returns

A 1D numpy array with the stat values for the tiles.

getsectioncollims(Q)

Returns limits of [HORIZONTAL] sections: prescan, image and overscan

getsectionrowlims(Q)

Returns limits of [VERTICAL] sections: image [and vertical overscan]

loadfromFITS(fitsfile, extensions=[- 1], getallextensions=False)

Loads contents of self from a FITS file.

or_mask(mask)

Adds (OR) a mask to self.extensions[*].data.mask

set_extension(data, header=None, label=None, headerdict=None, extension=- 1)

Sets extension ‘extension’ in self.

set_quad(inQdata, Quadrant, canonical=False, extension=- 1)

Sets the contents of a quadrant in an extension.

sim_window(vstart, vend, extension=- 1)

simadd_flatilum(levels=None, extension=- 1)

simadd_points(flux, fwhm, CCDID='CCD1', dx=0, dy=0, extension=- 1)

simadd_poisson(extension=- 1)

simadd_ron(extension=- 1)

sub_bias(superbias, extension=- 1)

Subtracts a superbias from CCD image.

sub_offset(Quad, method='row', scan='pre', trimscan=[3, 2], ignore_pover=True, extension=- 1)

Subtracts the offset from a quadrant.

writeto(fitsf, clobber=False, unsigned16bit=False)

Writes self to a FITS file.

class vison.datamodel.ccd.CCDPile(infitsList=None, ccdobjList=None, extension=- 1, withpover=True)

Class to hold and operate (e.g. stack) on a bunch of CCD images. Each image (a single extension picked from each) becomes an extension in the pile.

stack(method='median', dostd=False)

Stacking images with an stat.

class vison.datamodel.ccd.Extension(data, header=None, label=None, headerdict=None)

Extension Class. The images, as FITS files, have extensions, each with an extension, and optionally, a label, and data.

vison.datamodel.ccd.cooconv_arrays_decorate(func)

Decorator for conversion of coordinates on arrays.

vison.datamodel.ccd.test_create_from_scratch()

vison.datamodel.ccd.test_load_ELVIS_fits()

ccd_aux.py

Module auxiliary to ccd.py

History

Created on Mon Feb 19 13:14:02 2018

author

raf

class vison.datamodel.ccd_aux.Model2D(img, corners=None)

Class for 2D models of images and images sections.

bin_img(boxsize, stat='median')

Bins down image in self.

filter_img(filtsize=15, filtertype='median', Tests=False)

Returns filtered version of self.img.

fit2Dpol_xyz(xx, yy, zz, degree=1)

get_model_poly2D(sampling=1, pdegree=5, useBin=False)

get_model_splines(sampling=1, splinemethod='cubic', useBin=False, recoveredges=False, pdegree=5)

class vison.datamodel.ccd_aux.Profile1D(x, y)

Class for 1D profiles of images and images sections.

vison.datamodel.ccd_aux.extract_region(ccdobj, Q, area='img', vstart=0, vend=2086, Full=False, canonical=True, extension=- 1)

vison.datamodel.ccd_aux.get_1Dprofile(ccdobj, Q, orient='hor', area='img', stacker='mean', vstart=0, vend=2086, extension=- 1)

vison.datamodel.ccd_aux.get_region2Dmodel(ccdobj, Q, area='img', kind='spline', splinemethod='cubic', pdegree=2, doFilter=False, doBin=True, filtsize=1, binsize=1, filtertype='mean', recoveredges=False, vstart=0, vend=2086, canonical=True, extension=- 1)

Parameters

• ccdobj (object) – ccd object

• Q (char) – Quadrant

• kind (char) – type of interpolation

• doFilter (bool) – boolean to control whether to apply image filtering or not

• filtsize (int) – size of the filter to be applied (on a side)

• doBin (bool) – apply binning?

• binsize (int) – size of the binning window (on a side)

• filtertype (char) – type of the filter, if applied

• recoveredges (bool) – try to recover edges of array after binning / filtering?

• vstart (int) – windowing parameter, start line (from 0) to consider in image area

• vend (int) – windowing parameter, end line (+1) to consider in image area

• canonical (bool) – [quadrant] orientation of the resulting model

• extension (int) – image extension to be used from ccdobj

vison.datamodel.ccd_aux.rebin(arr, new_shape, stat='mean')

“Rebin 2D array arr to shape new_shape by averaging.

cdp.py

Classes to store Calibration Data Products.

History

Created on Tue Feb 27 10:58:42 2018

author

Ruyman Azzollini

class vison.datamodel.cdp.CCD_CDP(*args, **kwargs)

CCD Calibration Data Product

ingest_inputs(data, meta=None, header=None)

savehardcopy(filef='')

class vison.datamodel.cdp.CDP(*args, **kwargs)

Parent CDP Class.

loadfrompickle(pickf='')

savehardcopy(filef='')

savetopickle(pickf='')

class vison.datamodel.cdp.FitsTables_CDP(*args, **kwargs)

Fits Table CDP.

checkInputHardcopy(inhdulist)

To be filled up by a child-class, if so desired.

fill_Header()

fill_Meta()

fill_Table(sheet)

fill_allTables()

ingest_inputs(data, meta=None, header=None, figs=None)

init_HDUList()

loadhardcopy(filef)

savehardcopy(filef='')

class vison.datamodel.cdp.Json_CDP(*args, **kwargs)

Generic Json Object CDP.

ingest_inputs(data, meta=None, header=None)

loadhardcopy(filef='')

savehardcopy(filef='')

class vison.datamodel.cdp.LE1_CDP(*args, **kwargs)

LE1 FPA Image CDP. One extension per Quadrant.

ingest_inputs(data, header=None, inextension=- 1, fillval=0)

savehardcopy(filef='', clobber=True, uint16=False)

class vison.datamodel.cdp.Tables_CDP(*args, **kwargs)

Table CDP. Can export to excel.

fill_Header(title='')

fill_Meta()

fill_Sheet(sheet)

fill_allDataSheets()

get_textable(sheet, caption='', fitwidth=False, tiny=False, **kwargs)

ingest_inputs(data, meta=None, header=None, figs=None)

init_workbook()

savehardcopy(filef='')

vison.datamodel.cdp.loadCDPfromPickle(pickf)

Function to load a CDP from a pickle file.

vison.datamodel.cdp.wraptextable(tex, ncols=1, caption='', fitwidth=False, tiny=False, longtable=False)

Auxiliary function to Tables_CDP class

ccdsim.py

Methods to simulate data. Used by ccd.CCD class.

History

Created on Wed Apr 4 11:13:30 2018

author

Ruyman Azzollini

vison.datamodel.ccdsim.sim_window(ccdobj, vstart, vend, extension=- 1)

vison.datamodel.ccdsim.simadd_flatilum(ccdobj, levels=None, extension=- 1)

vison.datamodel.ccdsim.simadd_points(ccdobj, flux, fwhm, CCDID='CCD1', dx=0, dy=0, extension=- 1)

vison.datamodel.ccdsim.simadd_poisson(ccdobj, extension=- 1)

vison.datamodel.ccdsim.simadd_ron(ccdobj, extension=- 1)

compliance.py

Some functions to produce COMPLIANCE MATRICES.

History

Created on Mon Apr 9 17:32:03 2018

author

raf

class vison.datamodel.compliance.ComplianceMX(*args, **kwargs)

class vison.datamodel.compliance.ComplianceMX_CCD(CCDs=None, CCDlims=None)

check_stat(inparr)

class vison.datamodel.compliance.ComplianceMX_CCDCol(colnames, indexer, CCDs=None, lims=None)

check_stat(inparr)

class vison.datamodel.compliance.ComplianceMX_CCDQ(CCDs=None, Qs=None, CCDQlims=None)

check_stat(inparr)

class vison.datamodel.compliance.ComplianceMX_CCDQCol(colnames, indexer, CCDs=None, Qs=None, lims=None)

check_stat(inparr)

class vison.datamodel.compliance.ComplianceMX_CCDQColSpot(spotnames, colnames=None, indexer=None, CCDs=None, Qs=None, lims=None)

check_stat(inparr)

class vison.datamodel.compliance.TestComplianceClass(methodName='runTest')

setUp()

Hook method for setting up the test fixture before exercising it.

vison.datamodel.compliance.convert_compl_to_nesteditemlist(complidict)

vison.datamodel.compliance.gen_compliance_tex(indict, escape=True, caption='')

vison.datamodel.compliance.removescalars_from_dict(indict)

core.py

DataDict Class : holds data and results across sub-tasks of a “task” (Test).

This is the CORE data-structure used to do analysis and report results.

History

Created on Thu Sep 21 16:47:09 2017

author

Ruyman Azzollini

class vison.datamodel.core.DataDict(meta=None)

A Task object has associated a DataDict object where the input data for the Task/Test,

from the EXPLOG and HK files, and also results obtained through the Task.methods()

are stored.

So, DataDict is a data structure that usually grows as the Task execution progress.

A DataDict is basically a dictionary of arrays, but with some specific properties:

• All the arrays have a common dimension, equal to the number of frames / OBSIDs in the test.

• Other dimensions of the arrays may vary, depending on contents.

  • For example, the OBSID column only has this common dimension.

  • a column holding PSF FWHM ofspots, may have this common dimension, plus a dimension for the 3 CCDs, another for the 4 Quadrants in each CCD, and another for the 5 spots in each Quadrant.

• The DataDict is composed of Column arrays.

• The DataDict object has methods to save / reload from hard copies.

addColumn(array, name, indices, ix=- 1)

Adds a Column to self.

col_has_index(colname, indexname)

Verifies whether column colname has an index called indexname

dropColumn(colname)

Removes column colname from self.

flattentoTable()

Flattens the multidimensional contents of self to a 2D table.

Returns an astropy.table.Table object.

initColumn(name, indices, dtype='float32', valini=0.0)

Initialises a Column in self.

loadExpLog(explog)

Loads the contents of an EXPLOG.

name_indices()

Returns the names of the indices in self.

saveToFile(outfile, format='ascii.commented_header')

Saves self to a hardcopy.

uses the .write method of astropy.tables.Table.

class vison.datamodel.core.FpaDataDict(meta=None)

loadExpLog(explog)

vison.datamodel.core.useCases()

#TODO:

# create a DataDict object from an exposure log. # add a column indexed by ObsID, CCD and Quad # drop a column # create a column from an operation on several columns with different dimensions # save to a text / excel file # save to a pickle file

class vison.datamodel.core.vColumn(array, name, indices)

|Class for Column objects. | A column has contents (an array) and an Index/vMultiIndex object associated.

name_indices()

class vison.datamodel.core.vIndex(name, vals=None, N=0)

Class for indexes of a Column.

class vison.datamodel.core.vMultiIndex(IndexList=None)

Class for indices of a DataDict, which is made of Columns.

A MultiIndex is made up of individual Index objects.

append(*args)

Adds indices to self.

find(indexname)

finds the index an index name in self.

get_len(indexname)

Returns the length of index indexname in self.

get_names()

Returns the names of all indices in self.

get_shape()

Gets the dimensions of the indices in self.

get_vals(indexname)

Returns the values of the index indexname in self.

pop(*args)

Removes indices

update_names()

update_shape()

elvis.py

ELVIS variables dictionaries.

History

Created on Fri Sep 22 12:04:09 2017

author

Ruyman Azzollini

EXPLOGtools.py

class vison.datamodel.EXPLOGtools.ExpLogClass(elvis='7.5.X')

addRow(row)

iniExplog()

summary()

writeto(outfile)

vison.datamodel.EXPLOGtools.iniExplog(elvis)

vison.datamodel.EXPLOGtools.loadExpLog(expfile, elvis='7.5.X', safe=False)

Loads an Exposure Log from file.

vison.datamodel.EXPLOGtools.mergeExpLogs(explogList, addpedigree=False, verbose=False)

Merges explog objects in a list.

vison.datamodel.EXPLOGtools.test()

This Tests needs UPDATE (for data access and probably data format)

fpa_dm.py

FPA Data Model. LE1 FITS files.

Created on Thu Aug 1 17:05:12 2019

@author: raf

class vison.datamodel.fpa_dm.FPA_LE1(infits=None)

Class for LE1 fits files built from system-level image data (whole FPA).

This class relies on ccdobj instances to do analysis of the images (one CCD at a time).

add_extension(data, header, label=None, headerdict=None)

Adds an extension.

apply_function_to_ccds(ccdfunction, **kwargs)

Applies a function to each CCD in self.

del_extension(ixextension)

Deletes an extension.

get_CCDID_from_BLCCD(BLOCK, CCD)

Retrieves CCD ID (e.g. C_11) given BLOCK name and CCD in block.

param BLOCK

block nickname (e.g. ‘CURIE’)

param CCD

1, 2 or 3

get_as_FPAmosaic()

Returns a copy of self as an FPA mosaic

get_ccdobj(CCDID)

Returns a CCD Object given a CCDID

get_extid(CCDID, Q)

Retrieves extension ID given CCDID and quadrant.

param CCDID

e.g. ‘C_11’

param Q

‘E’, ‘F’, ‘G’ or ‘H’

initialise_as_blank(fillval=None)

Initialises object as a blank (filled with ‘fillval’) image of the FPA.

loadfromFITS(infits)

Loads contents of self from a FITS file.

savetoFITS(outfits, clobber=True, unsigned16bit=False)

Dumps self to a FITS file.

set_ccdobj(ccdobj, CCDID, inextension=- 1)

Sets the image contents of input ccdobj to the image in self, at CCDID.

set_extension(iext, data, header, label=None, headerdict=None)

Changes the contents of an extension.

simul(simputs=None, zerofirst=False)

Accessory function to simulate FPA images.

vison.datamodel.fpa_dm.test1()

vison.datamodel.fpa_dm.test2()

generator.py

Script to generate simulated data for pipeline testing purposes.

Created on Tue Aug 29 11:08:56 2017

author

Ruyman Azzollini

vison.datamodel.generator.IMG_bias_gen(ccdobj, ELdict, ogse=None)

vison.datamodel.generator.IMG_chinj_gen(ccdobj, ELdict, ogse=None)

vison.datamodel.generator.IMG_chinj_gen_v2(ccdobj, ELdict, ogse=None)

vison.datamodel.generator.IMG_flat_gen(ccdobj, ELdict, ogse=None)

vison.datamodel.generator.IMG_point_gen(ccdobj, ELdict, ogse=None)

vison.datamodel.generator.generate_Explog(scrdict, defaults, elvis='7.5.X', explog=None, OBSID0=1000, date=datetime.datetime(1980, 2, 21, 7, 0), CHAMBER=None)

Generates a fake ExposureLog from a test structure dictionary.

DEVELOPMENT NOTES:

To be generated: (EASY)

*ObsID, *File_name, *CCD, *ROE=ROE1, *DATE, *BUNIT=ADU, SPW_clk=0?,EGSE_ver=elvis,

Temporal:

SerRdDel

To be provided in defaults: (EASY)

Lab_ver,Con_file,CnvStart, Flsh-Rdout_e_time,C.Inj-Rdout_e_time, FPGA_ver,Chmb_pre,R1CCD[1,2,3]T[T,B]

To be read/parsed/processed from struct: (DIFFICULT)

SerRDel?,SumWell?, IniSweep?,+etc.

vison.datamodel.generator.generate_FITS(ELdict, funct, filename='', elvis='7.5.X', ogse=None)

vison.datamodel.generator.generate_FITS_fromExpLog(explog, datapath, elvis='7.5.X', CHAMBER=None)

vison.datamodel.generator.generate_HK(explog, vals, datapath='', elvis='7.5.X')

vison.datamodel.generator.merge_HKfiles(HKfilefs, masterHKf)

HKtools.py

House-Keeping inspection and handling tools.

History

Created on Thu Mar 10 12:11:58 2016

author

Ruyman Azzollini

vison.datamodel.HKtools.check_HK_abs(HKKeys, dd, limits='S', elvis='7.5.X')

Returns report on HK parameters, in DataDict (dd), compared to absolute limits.

HK Keys which have “relative” limits, always return False.

Parameters

• HKKeys – list of HK parameters, as named in HK files (without HK_ suffix)

• dd – DataDict object

• limits – type of limits to use, either “P” (Performance) or “S” (Safe)

• elvis – ELVIS version to find correspondence between HK key and Exposure Log input (commanded voltage).

Returns report

dictionary with pairs of HK-key : Bool. True = All values for given key are within limits. False = At least one value for given key is outside limits.

vison.datamodel.HKtools.check_HK_vs_command(HKKeys, dd, limits='P', elvis='7.5.X')

Returns report on HK parameters, in DataDict (dd), comparing inputs (commanded) vs. output (HK data).

HK Keys which do not correspond to commanded voltages always return ‘True’.

Parameters

• HKKeys – list of HK parameters, as named in HK files (without HK_ suffix)

• dd – DataDict object

• limits – type of limits to use, either “P” (Performance) or “S” (Safe)

• elvis – ELVIS version to find correspondence between HK key and Exposure Log input (commanded voltage).

Returns report

dictionary with pairs of HK-key : Bool. True = All values are within limits, referred to commanded value. False = At least one value is outside limits, referred to commanded value.

vison.datamodel.HKtools.doHKSinglePlot(dtobjs, HK, HKkey, ylabel='V', HKlims=[], filename='', fontsize=10)

Plots the values of a HK parameter as a function of time.

Parameters

• dtobjs – datetime objects time axis.

• HK – HK values (array)

• HKkey –

• ylabel –

• HKlims –

• filename – file-name to store plot [empty string not to save].

Returns

None!!

vison.datamodel.HKtools.filtervalues(values, key)

vison.datamodel.HKtools.iniHK_QFM(elvis='7.5.X', length=0)

vison.datamodel.HKtools.loadHK_QFM(filename, elvis='7.5.X', validate=False, safe=False)

Loads a HK file, or list of HK files.

Structure: astropy table. First column is a timestamp, and there may be a variable number of rows (readings).

Parameters

• filename – path to the file to be loaded, including the file itself, or list of paths to HK files.

• elvis – “ELVIS” version

Returns

astropy table with pairs parameter:[values]

vison.datamodel.HKtools.loadHK_QFMsingle(filename, elvis='7.5.X', validate=False, safe=False)

Loads a HK file

Structure: tab separated columns, one per Keyword. First column is a timestamp, and there may be a variable number of rows (readings).

Parameters

• filename – path to the file to be loaded, including the file itself

• elvis – “ELVIS” version

Returns

astropy table with pairs parameter:[values]

vison.datamodel.HKtools.loadHK_preQM(filename, elvis='5.7.07')

Loads a HK file

It only assumes a structure given by a HK keyword followed by a number of of tab-separated values (number not specified). Note that the length of the values arrays is variable (depends on length of exposure and HK sampling rate).

Parameters

filename – path to the file to be loaded, including the file itself

Returns

dictionary with pairs parameter:[values]

vison.datamodel.HKtools.mergeHK(HKList)

vison.datamodel.HKtools.parseDTstr(DTstr)

vison.datamodel.HKtools.parseHKfiles(HKlist, elvis='7.5.X')

Parameters

• HKlist – list of HK files (path+name).

• elvis – “ELVIS” version.

Returns

[obsids],[dtobjs],[tdeltasec],[HK_keys], [data(nfiles,nstats,nHKparams)]

vison.datamodel.HKtools.parseHKfname(HKfname)

Parses name of a HK file to retrieve OBSID, date and time, and ROE number.

Parameters

HKfname – name of HK file.

Returns

obsid,dtobj=datetime.datetime(yy,MM,dd,hh,mm,ss),ROE

vison.datamodel.HKtools.reportHK(HKs, key, reqstat='all')

Returns (mean, std, min, max) for each keyword in a list of HK dictionaries (output from loadHK).

Parameters

• HK – dictionary with HK data.

• key – HK key.

Reqstat

what statistic to retrieve.

vison.datamodel.HKtools.synthHK(HK)

Synthetizes the values for each parameter in a HK dictionary into [mean,std,min,max].

Parameters

HK – a dictionary as those output by loadHK.

Returns

dictionary with pairs parameter:[mean,std,min,max]

inputs.py

Inputs Handling Classes and utilities.

Created on Thu Jan 11 10:34:43 2018

author

Ruyman Azzollini

class vison.datamodel.inputs.ChildInputs(*args, **kwargs)

class vison.datamodel.inputs.Inputs(*args, **kwargs)

Class to hold, transfer and ‘document’ Task Inputs.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

QLAtools.py

Quick-Look-Analysis Tools.

History

Created on Wed Mar 16 11:31:58 2016

@author: Ruyman Azzollini

vison.datamodel.QLAtools.dissectFITS(FITSfile, path='')

vison.datamodel.QLAtools.getacrosscolscut(CCDobj)

vison.datamodel.QLAtools.getacrossrowscut(CCDobj)

vison.datamodel.QLAtools.getsectionstats(CCDobj, QUAD, section, xbuffer=0, 0, ybuffer=0, 0)

vison.datamodel.QLAtools.plotAcCOLcuts(dissection, filename=None, suptitle='')

vison.datamodel.QLAtools.plotAcROWcuts(dissection, filename=None, suptitle='')

vison.datamodel.QLAtools.plotQuads(CCDobj, filename=None, suptitle='')

vison.datamodel.QLAtools.reportFITS(FITSfile, outpath='')

scriptic.py

Classes and functions to generate ELVIS commanding scripts automatically.

Created on Wed May 24 15:31:54 2017

author

Ruyman Azzollini

class vison.datamodel.scriptic.Script(defaults=None, structure=None, elvis='7.5.X')

Core Class that provides automatic test script generation and validation.

build_cargo()

Updates ‘cargo’ attribute. ‘cargo’: list of lists, each corresponding to a column in the script.

Each element in the inner lists is a register value. The first column corresponds to the column with key names.

Note: the number of frames is accumuled across columns, as ELVIS expects.

get_struct_from_cargo()

load(*args, **kwargs)

alias method. Points to ‘load_to_cargo’.

load_to_cargo(scriptname, elvis='7.5.X')

Loads an script from an excel file.

Parameters

• scriptname – char, script to load

• elvis – char, ELVIS version of script to load

validate(defaults, structure, elvis='7.5.X')

Not sure ‘validation’ will work like as implemented… TODO: validate self.validate

write(scriptname)

Writes self.cargo (script) to an excel file.

Parameters

scriptname – char, name of file where to write script.

vison.datamodel.scriptic.test0()

vison.datamodel.scriptic.update_structdict(sdict, commvalues, diffvalues)

Updates an script structure with common values and differential values.

Parameters

• sdict – dict, dictionary with script structure. Takes precedence over commvalues.

• commvalues – dict, dictionary with common values to update sdict.

• diffvalues – dict, dictionaty with “differential” values to update “sdict”. Takes precedence over sdict and commvalues.