Scripts

HKmonitor.py

TODO

  • find HK files in a folder

  • parse HK files

  • plot HK parameters vs. time

  • assemble all plots into a pdf file

Script to produce HK reports out of HK files in a folder. Aimed at quick inspection of data from Characterization and Calibration Campaigns of Euclid-VIS.

History

Created on Tue Mar 15 10:35:43 2016

author

Ruyman Azzollini (MSSL)

QLA.py

QLA script. Aimed at quick inspection of data from Characterization and Calibration Campaigns of Euclid-VIS.

TODO

  • find FITS files in a folder

  • load FITS file into a CCD object

  • obtain metrics on the data:

    • image of 4 quadrants

    • across-rows and across-columns plots

    • statistics of image, pre and over scan regions: mea, med, std, p25, p75

  • parse header

  • do plots

  • assemble all plots into a pdf file per image

  • merge all pdfs into a single pdf file

History

Created on Wed Mar 16 11:33:21 2016

author

Ruyman Azzollini

quickds9.py

Wrap-up of SAO DS9 to quickly load a number of images, for inspection.

Core engine: pyds9 (http://hea-www.harvard.edu/RD/pyds9/)

History

Created on Thu Mar 17 13:18:10 2016

author

Ruyman Azzollini

vis_mkscripts.py

Automatically Generating Calibration Campaign Data Acquisition Scripts. Aimed at ELVIS.

History

Created on Fri Sep 08 12:03:00 2017

autor

Ruyman Azzollini

vison.scripts.vis_mkscripts.f_write_script(struct, filename, outpath, elvis)

Function that writes the test structure (as a dictionary) to an .xlsx file.

Calls vison.datamodel.scriptic.Script class.

vison.scripts.vis_mkscripts.scwriter(toWrite, test_generator, outpath, equipment, elvis='7.5.X', CHAMBER=None)

Parent function writing the test scripts.

  • calls test_generator to create a sequence of test objects, instantiated with their inputs.

  • the method build_scriptdict() of each test object is called to generate the dictionary-based structure of the test

  • test acquisition durations are estimated

  • the function f_write_script is called to write the script to an excel.

  • a txt file is written with a summary of the tests written.

  • a .xlsx file is written with the same summary.

  • the checksums of the scripts are written to a .txt file.

Parameters

  • toWrite (dict()) – a dictionary with pairs of test_name: flat, where flag is either 1 (create script), or 0 (do not create).

  • test_generator (function) – a function that instanstiates test objects for a given campaign type.

  • outpath (str) – outputs directory. Created on the fly if not already-existent.

  • equipment (dict()) – dictionary with equipment serials.

  • elvis (str) – elvis version to write scripts for.

  • CHAMBER (str) – chamber where the tests will be executed (affects exposure times, for exapmle).

Returns

None

vison.scripts.vis_mkscripts.write_summary_asexcel(summarydict, excelf, meta)

Writes a summary of the tests generated in .xlsx format. Uses vison.support.excel.ReportXL class and Pandas.

vison.scripts.vis_mkscripts.write_summary_astextfile(summarylines, inventoryfile, meta)

Writes a summary of the scripts generated, in .txt format.

vis_genDataSet.py

Development: Creating Calibration Campaign Fake Data-sets.

History

Created on Tue Sep 05 16:07:00 2017

autor

Ruyman Azzollini

vison.scripts.vis_genDataSet.datasetGenerator(TestsSelector, doGenExplog, doGenHK, doGenFITS, outpath, elvis, CHAMBER, Nrows=0)
vison.scripts.vis_genDataSet.genExpLog(toGen, explogf, equipment, elvis='7.5.X', CHAMBER=None)

vison_run

This is the main script that executes the pipeline. Because it does not have a .py extension, it is not correctly parsed into Sphynx.

vis_explogs_merger.py

Merges [concatenates] a list of [ELVIS] exposure logs.

History

Created on Fri Feb 9 15:20:01 2018

author

Ruyman Azzollini

vison.scripts.vis_explogs_merger.explog_merger(ELlist, output='EXP_LOG_merged.txt', elvis='7.5.X')

vis_run_xtalk.py

Master Script to measure and report cross-talk levels among 12 ROE channels. Takes as input a data-set composed of 3x12 CCD images, corresponding to injecting a “ladder” of signal on each of the 12 channels, using the ROE-TAB.

History

Created on Thu Mar 22 16:17:39 2018

author

Ruyman Azzollini

vison.scripts.vis_run_xtalk.run_xtalk(incat, inpath='', respath='', metafile='', doCompute=False)

vis_ROE_LinCalib.py

Non-Linearity Calibration of ROE (on bench).

History

Created on Thu Mar 15 15:32:11 2018

author

Ruyman Azzollini

vison.scripts.vis_ROE_LinCalib.find_adu_levels(qdata, Nlevels, debug=False)
vison.scripts.vis_ROE_LinCalib.run_ROE_LinCalib(inputsfile, incatfile, datapath='', respath='', doExtractFits=True, dopolyRT=False, debug=False)

vis_ROETAB_LinCalib.py

Linearity Calibration of ROE-TAB.

History

Created on Tue Mar 27 14:42:00 2018 Modified on Fri Sep 14 10:53:00 2018

author

Ruyman Azzollini

vison.scripts.vis_ROETAB_LinCalib.filter_Voltage_uni(rV, filt_kernel)
vison.scripts.vis_ROETAB_LinCalib.find_discrete_voltages_inwaveform(rV, levels, filtered=None, debug=False)
vison.scripts.vis_ROETAB_LinCalib.load_WF(WFf, chkNsamp=None, chkSampInter=None)
vison.scripts.vis_ROETAB_LinCalib.plot_waveform(WF, disc_voltages=[], figname='', chan='Unknown')
vison.scripts.vis_ROETAB_LinCalib.run_ROETAB_LinCalib(inputsfile, incatfile, datapath='', respath='', doBayes=False, debug=False)

vis_star_finder.py

Script to find point sources in VIS Ground Calibration Campaign. Used to ‘prime’ the position tables of point-source objects.

History

Created on Tue Jun 12 16:09:31 2018

author

Ruyman Azzollini

vison.scripts.vis_star_finder.write_ID_chart(filename, Quads, Starnames)

vis_load_DD.py

Loading a DataDict object for inspection.

History

Created on Wed Aug 01 10:00:00 2018

autor

Ruyman Azzollini

vis_cosmetics_masker.py

Script to create cosmetics masks in VIS Ground Calibration Campaign.

History

Created on Wed Aug 1 11:02:00 2018

author

Ruyman Azzollini

vison.scripts.vis_cosmetics_masker.do_Mask(inputs, masktype, subbgd=True, normbybgd=False, validrange=None, flagWholeColumns=False)
vison.scripts.vis_cosmetics_masker.pre_process(FITS_list, subOffset=False, validrange=None)
vison.scripts.vis_cosmetics_masker.read_OBSID_list(ff)
vison.scripts.vis_cosmetics_masker.run_maskmaker(inputs)

vis_clear_space.py

Script to clean-up processing files produced by the pipeline. Useful to free-up space.

USE WITH CAUTION.

History

Created on Thu Dec 6 10:19:24 2018

author

raf

vison.scripts.vis_clear_space.find_and_erase(path, keyword)

vis_reports_merger.py

Reports Merger - Tests

History

Created on Wed Dec 19 10:02:02 2018

author

raf

vison.scripts.vis_reports_merger.run_merger(infile)
vison.scripts.vis_reports_merger.run_merger_plus(infile, Issue=0.0)

vis_mksession.py

Script to build test SEQUENCES easily and reliably.

History

Created on Thu Mar 14 10:30:33 2019

author

raf

vison.scripts.vis_mksession.session_builder(inputs)

vis_dnl_extract.py

Exploring the histograms of counts in images. Do we have “missing codes” due to larger-than-expected DNL? DNL: Differential Non Linearity

This issue was brought to attention of VIS IDT by Ralf Kohley on Jan 10th 2019.

History

Created on Tue Feb 5 09:35:25 2019

author

raf

vison.scripts.vis_dnl_extract.batch_extract_dnls(explog, respath, interptype, multithread=1)
vison.scripts.vis_dnl_extract.batch_extract_histos(explog, respath, multithread=1)
vison.scripts.vis_dnl_extract.explog_selector(explog, testkeys)
vison.scripts.vis_dnl_extract.extract_histograms(infile_name, i, N)
vison.scripts.vis_dnl_extract.get_average_dnls(explog, respath, tag)
vison.scripts.vis_dnl_extract.interpol_histo_cubic(histo)

NOT WORKING PROPERLY, DO NOT USE

vison.scripts.vis_dnl_extract.interpol_histo_linear(histo)
vison.scripts.vis_dnl_extract.save_to_fits(array, fitsfile)