Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dqsegdb2 update #4442

Merged
merged 2 commits into from
Jul 26, 2023
Merged

dqsegdb2 update #4442

merged 2 commits into from
Jul 26, 2023

Conversation

tdent
Copy link
Contributor

@tdent tdent commented Jul 19, 2023

Need a version that supports scitokens

Co-authored-by: Duncan Macleod <[email protected]>
@tdent
Copy link
Contributor Author

tdent commented Jul 26, 2023

@spxiwh as I have heard Dunk is on holiday, could you approve this please

Copy link
Contributor

@spxiwh spxiwh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

... I know it's convenient, but remember that branches on origin spawn two copies of the CI code, and therefore slow things down. Please delete this after merging!

@spxiwh spxiwh merged commit 3ed210d into master Jul 26, 2023
72 checks passed
@spxiwh spxiwh deleted the tdent-dqsegdb2 branch July 26, 2023 09:58
@tdent
Copy link
Contributor Author

tdent commented Jul 26, 2023

It looks like you already did. I don't have much of an idea how the CI code is set up - do the two copies exist if the PR is not yet merged?

tdent added a commit that referenced this pull request Jul 26, 2023
* Try to fix CI tests (#4394)

* Switch FFTW/MKL order of preference

* Swap backend order and force gomp import

* FIx typo

* Do need openmpi in tox.ini

* Fix bug in MKL error function

* Force FFTW on this one.

* CC

* Requested Josh changes

* Add bank corner plot (#4339)

* Add corner plotting script for template banks

* Comment where premerger duration has been subtracted from template duration in bank_conversions

* add premerger_duration to bank_conversions.py

* Move premerger duration into bank_conversions

* Use bank's approximant

* Dont plot signular values as default

* CC complaints

* docstring

* TDC comments

* Keep approximant kwarg as default, TDC comments 2

* Added an OSError exception to MKL check (#4390)

* Add a possibility for defaultvalues sections in config files (#4385)

* Add a possibility for defaultvalues sections in config files

* do not allow [section] and [section-defaultvalues] options

* defaultvalues for top-level sections only

* underscore on add_ini_opts

* Remove default sections from get_subsections (#4399)

* Update setup-python action to v4 (#4400)

* Replace LOSC with GWOSC (#4401)

* Replace LOSC with GWOSC

* Fix leftover Python 2-ism

* Try to improve codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Exclude IFO sections when looping through the get_subsections (#4402)

* Exclude IFO sections when looping through the get_subsections for plots

* Add an example whihc would have previously failed into the search config

* Offline singles in workflow (#4386)

* Add single-detector options into workflow

* Fix a couple of isses which make results pages weird

* fix

* fix gen.sh to use correct workflow generator

* revert change made in error

* A couple of points which state coincident when it may not be

* Rejection samp (#4379)

* unexpected events

* added temp-volume in cl

* added rejection sampling

* added indexing of mchirp in fit_param

* added compensation factor

* added corrections

* added more corrections

* added random number generator seed

* added mchirp instead of log(mchirp) in rejection samp

* Use probabilities to compensate downsampled kde

* Try to deal with flaky failures when apt-getting (#4384)

* Implement pre-cuts in fit_over_multiparam for efficiency (#4374)

* Make some efficiency savings in pycbc_fit_sngls_over_multiparam

* Missing logging line in once case

* Add in progress reporting, in case of silent failure

* I thought we needed some leeway, but we don't

* TD comments

* Simplify comment

* Migrate to new gwdatafind API (#4383)

* Migrate to new gwdatafind API

* CC and one regression fix

* One more style change

* Don't need these options now

* Revert "Don't need these options now"

This reverts commit 6f2ed1b.

* Add a Fisher distribution for sky locations (#4209)

* Update transforms.py

Adding new_z_to_euler and rotate_euler from pylal.sphericalutils

* Update sky_location.py

Added new class FisherDist for drawing from sky locations (in ra, dec)

* Update sky_location.py

Included help message for fisherDist

* Update sky_location.py

* Code climate issues

* Code climate issues

* Code-climate changes

* Code climate issues

* Code climate issues

* Update transforms.py

changed lal.PI to numpy.pi in new_z_to_euler

* Update transforms.py

minor changes

* Update sky_location.py

updated docstring, did changes to make the code look more simple

* Update sky_location.py

typos corrected

* Update sky_location.py

code climate changes

* Update sky_location.py

* Update sky_location.py

Included the definitions rvs_polaz and rvs_radec

* Update transforms.py

Included the definitions decra2polaz and polaz2decra

* Update sky_location.py

* Update transforms.py

code climate issues fixed

* Update sky_location.py

code climate issues fixed

* Update sky_location.py

code climate issues

* Update sky_location.py

fixed code climate issues

* Update sky_location.py

fixed code climate issues

* Update sky_location.py

* Update pycbc/distributions/sky_location.py

Co-authored-by: Tito Dal Canton <[email protected]>

* Update sky_location.py

* Update sky_location.py

* Update transforms.py

modified polaz2decra to polaz2radec

* Update sky_location.py

* Update sky_location.py

Forgotten polaz2decra --> polaz2radec

* Update sky_location.py

Code climate fixes

* Update sky_location.py

Updated the code for checking the effectiveness of running from config file

* Create sky_location_old.py

* modified to include from config

* Modified to run from config file

* Added Fisher distribution to the lista of available distributions

* Delete sky_location_old.py

* Update sky_location.py

* Renamed Fisher to FisherSky

* Updated the name to FisherSky

* changed Fisher to FisherSky in __all__

* removed decra2polaz and polaz2radec to conversions

* Included def decra2polaz, polaz2radec in line 859

* Added angle_unit option

* removed decra2polaz and polaz2decra

* made carresction

* made corrections

* Corrected the typing error while removing recra2polaz

* Modified angle_unit and updated init file

* Empty line on 852 retrieved

* corrected

* Empty line on852 retrieved

* Fixed code climate issues

* Sigma conversion taken care of

* Fixed code climate issues

* Update sky_location.py

Codeclimate

* Update test_distributions.py

Adding fisher_sky to the list of distributions that do not undergo the distribution unittests.

* Update sky_location.py

Syntax fixed.

* Update sky_location.py complying to codeclimate

* Refactor using scipy's Rotation

* Improve docstring and make warning stricter

* Update pycbc/distributions/sky_location.py

Co-authored-by: Francesco Pannarale <[email protected]>

* Update pycbc/distributions/sky_location.py

right angle to correct angle

Co-authored-by: Francesco Pannarale <[email protected]>

* Update sky_location.py

logging issue solved

---------

Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>

* add in several missing marginalization combinations (#4376)

* allow polarization marginalization with earth rotation

* add in missing marg combos

* fixes

* fixes

* fixes

* cleanup

* cc

* cc

* Try to fix CI tests (#4394)

* Switch FFTW/MKL order of preference

* Swap backend order and force gomp import

* FIx typo

* Do need openmpi in tox.ini

* Fix bug in MKL error function

* Force FFTW on this one.

* CC

* Requested Josh changes

* Add bank corner plot (#4339)

* Add corner plotting script for template banks

* Comment where premerger duration has been subtracted from template duration in bank_conversions

* add premerger_duration to bank_conversions.py

* Move premerger duration into bank_conversions

* Use bank's approximant

* Dont plot signular values as default

* CC complaints

* docstring

* TDC comments

* Keep approximant kwarg as default, TDC comments 2

* Added an OSError exception to MKL check (#4390)

* Add a possibility for defaultvalues sections in config files (#4385)

* Add a possibility for defaultvalues sections in config files

* do not allow [section] and [section-defaultvalues] options

* defaultvalues for top-level sections only

* underscore on add_ini_opts

* Remove default sections from get_subsections (#4399)

* Update setup-python action to v4 (#4400)

* Replace LOSC with GWOSC (#4401)

* Replace LOSC with GWOSC

* Fix leftover Python 2-ism

* Try to improve codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Exclude IFO sections when looping through the get_subsections (#4402)

* Exclude IFO sections when looping through the get_subsections for plots

* Add an example whihc would have previously failed into the search config

* Offline singles in workflow (#4386)

* Add single-detector options into workflow

* Fix a couple of isses which make results pages weird

* fix

* fix gen.sh to use correct workflow generator

* revert change made in error

* A couple of points which state coincident when it may not be

---------

Co-authored-by: Thomas Dent <[email protected]>
Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Gareth S Cabourn Davies <[email protected]>
Co-authored-by: Prasia-Pankunni <[email protected]>
Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Alex Nitz <[email protected]>
Co-authored-by: Arthur Tolley <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>

* Adding SEOBNRv5Peak key and function in the final frequency cutoff dictionary (#4408)

Co-authored-by: KHUN PHUKON <[email protected]>

* Injection kde (#4409)

* unexpected events

* added temp-volume in cl

* added injection kde

* significance bug for trigger_fit. A couple of minor fixes (#4407)

* Upgrades to pycbc_grb_inj_finder (#4364)

* retaining single ifo injection information

* Update pycbc_grb_inj_finder

Update according to requested changes from pull request review.

* Adding more specific comments

Co-authored-by: Francesco Pannarale <[email protected]>

* Adding more specific comments

out_dict changed to "output dictionary" in comment

Co-authored-by: Francesco Pannarale <[email protected]>

* Remove xlen

Removed xlen from cp_dataset_cont and instead calculated the length of xidentifier within the function.

* Reverting comment changes

* Remove extra comment

* Francesco's bug fix

Modified to simplify and correct where ifo information was jumbled together

* remove trig_dict and comment

* Removed extra lines

Removed line and datasets = {}

* Update to copyright comment and imports

Added our names and accounted for older python version.

---------

Co-authored-by: Francesco Pannarale <[email protected]>

* Simple bugfix (#4413)

Co-authored-by: Bhooshan Gadre <[email protected]>

* Move PyCBC venvs to new IGWN CVMFS server (#4412)

* First part of moving venv build to igwn

* TESTING HACK. REVERT ME BEFORE MERGING!!!

* Try this

* Testing

* Revert "Testing"

This reverts commit c0b9ea0.

* Add new CVMFS commands

* Remove rsync filter which isn't working

* Add error wrap abort handler

* Update tools/venv_transfer_commands.sh

Co-authored-by: Duncan Macleod <[email protected]>

* Fix Ian's idiocy (attempt to) and HostChecking

* Set to abort for one last test

* One final publish test

* Add a chmod on the root CVMFS directory

* Prepare for merging

* Fix mistake

---------

Co-authored-by: Duncan Macleod <[email protected]>

* Get FFT options in pycbc_condition_strain (#4404)

* pycbc_optimize_snr: Implement mass dependent bounds on spin parameters (#4176)

* Applying spin bounds based upon object mass~

* Updated reference to correct paper

* Adding spin bounds based on upper bounds of masses

* removing unnecessary additions

* Changing max mass2 to the right quantities

* Creating maxeta + reusing maxm1

* minor cleanup

* fixed min / max eta values

* fixed min / max eta

* Correcting maxm2 equation

* more readable/less redundant comment

---------

Co-authored-by: Thomas Dent <[email protected]>

* Allow SNR optimizer to use candidate point in initial array (#4393)

* allow SNR optimizer to use the candidate point in the initial array of test points

* update two LISA docs (#4417)

* Update injection_smbhb.ini

* Update lisa_smbhb_relbin.ini

* Update lisa_smbhb_relbin.ini

* Update lisa_smbhb_relbin.ini

* Update lisa_smbhb_ldc_pe.rst

* Allow pycbc_multi_inspiral to search over a sky grid (#4380)

* Allow pycbc_multi_inspiral to search over a sky grid

* Adding the possibility to give either a sky grid or a sky position

* Change the names of the command lines --ra and --dec to --latitude and --longitude to match the changes made in pycbc_multi_inspiral and remove --processing-scheme because it makes the example crash

* Compute the antenna pattern only once for each sky position by keeping fp and fc together

* Replace the number of detectors from a hardcoded value to a variable

* add TDI AE PSD with confusion noise for LISA (#4389)

* add TDI AE PSD with confusion noise

* Update analytical_space.py

* fix cc issue

* Update analytical.py

* EM brute (#4299)

* Space before colons in eos_utils.py help messages

* Enable extrapolation in eos_utils.py when NS is out of bounds

* Same check on shape in primary_mass and secondary mass

* Function to swap masses and spins of object 1 and object2

* Allow user to request for remnant_mass calculations: 1) extrapolation, 2) swapping object 1 and object 2 if mass2 > mass1, 3) treat as a (dark) BBH and NSBH with NS mass greater than a value probided by the user

* Renamned swap_companions to ensure_obj1_is_primary and generalized it

* codeclimate

* codeclimate

* Fixed line that was too long

* Fixed docstring and using keywords explicitly in all remnant_mass_* calls

* Renamed max_ns_mass as ns_bh_mass_boundary and improved help messages of functions

* Improved docstings

* Comprehensions in pycbc_brute_bank

* Complying to naming convention for spin components in spherical coordinates

* Update conversions.py

Fixing single code climate complaint.

* Prevent duplicates of the HLV page_ifar plot on the summary page (#4421)

* Prevent duplicates of the HLV page_ifar plot on the summary page

* Rename 'coincidences' as appropriate

* add high frequency sky location dependent response for long detectors (#4377)

* reset commit, add in high freqeuncy response to detector

* update

* fixes

* Make pegasus 5.0.5 work (#4276)

* Revert "try pinning pegasus version (#4265)"

This reverts commit 398e135.

* Set for_planning flag for subworkflow output map

* Add version check so we don't break 5.0.3

* Also include 5.0.4

* make ci faster (#4414)

* update

* move long log into debug level, allow to set debug level

* fix workflow

* more explicit control of running external scripts for docs setup at same time

* fixes

* update

* try this

* try this ...

* update

* fixes

* update

* updates

* mistake in example search config file (#4425)

* mistake in example search config file

* add failure if search runs too long

* Range_temp kde (#4424)

* unexpected events

* added temp-volume in cl

* setting the range for temp_kde

* setting min based on max kde

* logging for min template kde ratio

---------

Co-authored-by: Thomas Dent <[email protected]>

* Pygrb offline workflow (#4288)

* Fixed indexing and autochisq argument in pycbc_multi_inspiral

* Only Hanford is labelled with a 1: avoiding pycbc_pygrb_plot_snr_timeseries failure.  trigs rather than trigs_or_injs as variable name in pycbc_pygrb_plot_coh_ifosnr

* Simplified and update pycbc_pygrb_plot_injs_results; adapted pycbc_pygrb_pp_workflow

* Forgotten comma

* Massive commit that stitches together pycbc_make_offline_grb_workflow and pycbc_pygrb_pp_workflow, by picking up the latter as a SubWorkflow.

* Massive commit that stitches together pycbc_make_offline_grb_workflow and pycbc_pygrb_pp_workflow, by picking up the latter as a SubWorkflow.

* Code-climate fixes

* More code-climate

* Import error flagged by code-climate

* Forgotten import fixes

* Minor cleanup of comments or commented out code lines

* Syntax fix

* Defining 2 empty file lists

* First part of PR 4288 review comments

* One more TODO

* opt_to_file is now configparser_value_to_file in pycbc/workflow/core.py

* Upgraded configparser_value_to_file with attrs keyword argument

* Using configparser_value_to_file throughout the PyGRB workflow generator

* Removing 2 unused PyGRB functions

* Removed LigolwCBCAlignTotalSpinExecutable, LigolwCBCJitterSkylocExecutable, and PycbcDarkVsBrightInjectionsExecutable support in general and removed LalappsInspinjExecutable support from PyGRB

* Codeclimate

* bank_veto_file variable needs to be a FileList, not a File

* Bug with tags for single IFO SNR plots workflow generator

* Fixing pycbc_pygrb_plot_coh_ifosnr

* Fixing layout of chi-square plots and section label for individual detector SNRs

* Codeclimate

* Codeclimate

* Codeclimate

* Removed FIXME for testing purposes

* First changes following up on PR review

* Second round of changes following up on PR review

* Flagging with an underscore grb utility functions not used outside the files they are defined in

* Flagging with an underscore grb utility functions not used outside the files they are defined in

* Removed _get_antenna_factors from PyGRB utily file

* Removed commented out function from pycbc_pygrb_plot_injs_results

* Substituted old unused code with a TODO

* Adding injection set name as an option in pycbc_pygrb_efficiency

* extend --> append for single files

* Documenting setup_pygrb_pp_workflow more appropriately

* Reverted name of _get_antenna_responses to get_antenna_responses; cleaner handling of bank-veto-bank-file for pycbc_multi_inspiral job setup.

* Line shortened

* Avoid skymap v1.1.0 (#4429)

* only download LISA response file once (#4423)

* Update analytical_space.py

* Update analytical_space.py

* Introduced a new arguement --skymap-only-ifos in pycbc_live (#4346)

* Introduced a new arguement --skymap-only-ifos in pycbc_live

* Default skymap_only_ifos changed

* skymap_only_ifos as an attribute of LiveEventManager

* Test skymaps with V1 as skymap_only_ifos

* singles with skymap_only_ifos

* Removing commented lines etc

* A default seed for SNR opt

* Poking CI tests

I remove an unnecessary FIXME in the comments, but mostly doing this to restart the CI tests, which seem to have not linked up right.

---------

Co-authored-by: Souradeep Pal <[email protected]>
Co-authored-by: Ian Harry <[email protected]>

* Fix stageout (#4428)

* Fix the help message for stageout script

* Fix the help message for stageout and copy map scripts

---------

Co-authored-by: connor-mcisaac <[email protected]>

* more physical rwrap for LISA SMBHB/SOBHB injection (follow-up fix 2) (#4350)

* more physical rwrap

* fix cc issues

* Update waveform.py

* Update companion.txt

* Update requirements.txt

* Update requirements.txt

* Update companion.txt

* 10 times tau

* Update companion.txt

* Update requirements.txt

* Update setup.py

* add check

* Update waveform.py

* allow user to choose rwarp

* fix cc issue

* set 0.2 as minimum rwrap

* Update waveform.py

* Remove most X509/globus links and hooks from PyCBC (#4375)

* Remove most X509/globus links and hooks from PyCBC

* Need to update this one as well

* Don't need these options now

* Update docs/workflow/pycbc_make_coinc_search_workflow.rst

Co-authored-by: Gareth S Cabourn Davies <[email protected]>

---------

Co-authored-by: Gareth S Cabourn Davies <[email protected]>

* fftw_fix (#4418)

* Allow pycbc inspiral to run with only numpy fft

* Remove Unneeded function from npfft.py

---------

Co-authored-by: Neeresh Kumar Perla <[email protected]>

* Add argument verification for singles (#4365)

* Add argument verification for singles

* Apply suggestions from code review

* remove --enable-single-detector-background

* Some more option-checking, and making fixed ifar option actually possible!

* fix options in example

* CC, typo

* CC

* Move versioning page generation into its own executable (#4431)

* Move versioning page generation into its own executable

* Allow duplicated executables

* 🤦

* CC changes

* CC

* save_fig_with_metadata cannot be imported from versioning

* Don't like spaces in arguments

* Ian's comments

* Dont pass executables as files

* Same type output from get_code_version_numbers

* Rename --longitude and --latitude back to --ra and --dec (#4433)

* bugfix - allow non-running of singles (#4439)

* bugfix - allow non-running of singles

* CC

* Revrt wrongly-added test chnages

* Add IMRPhenomXAS template duration (#4410)

* Add IMRPhenomXAS template duration

* removing assertion

* fixed fomatting for codeclimate

* Adding PhenomXAS duration and removing space fixes

* removing extra new line

---------

Co-authored-by: Bhooshan Gadre <[email protected]>

* Treat CVMFS better (#4440)

* dqsegdb2 update (#4442)

* dqsegdb2 update

* Update requirements-igwn.txt

Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Gareth S Cabourn Davies <[email protected]>
Co-authored-by: Arthur Tolley <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Praveen Kumar <[email protected]>
Co-authored-by: Prasia-Pankunni <[email protected]>
Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Alex Nitz <[email protected]>
Co-authored-by: Khun Sang Phukon <[email protected]>
Co-authored-by: KHUN PHUKON <[email protected]>
Co-authored-by: ETVincent <[email protected]>
Co-authored-by: Bhooshan Uday Varsha Gadre <[email protected]>
Co-authored-by: Bhooshan Gadre <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>
Co-authored-by: Shichao Wu <[email protected]>
Co-authored-by: hoangstephanie <[email protected]>
Co-authored-by: SouradeepPal <[email protected]>
Co-authored-by: Souradeep Pal <[email protected]>
Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: connor-mcisaac <[email protected]>
Co-authored-by: Neeresh Kumar <[email protected]>
Co-authored-by: Neeresh Kumar Perla <[email protected]>
@spxiwh
Copy link
Contributor

spxiwh commented Jul 26, 2023

So there is one copy of all the tests for any pull request and there is one copy of all the tests for any commit to origin. So for pull requests from origin we get two copies of both of the tests. A further set of the tests would run after the PR is merged, but that is normal.

.... There are also a few potential security concerns around pull requests from origin.

PRAVEEN-mnl pushed a commit to PRAVEEN-mnl/pycbc that referenced this pull request Nov 3, 2023
* dqsegdb2 update

* Update requirements-igwn.txt

Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>
bhooshan-gadre pushed a commit to bhooshan-gadre/pycbc that referenced this pull request Mar 4, 2024
* dqsegdb2 update

* Update requirements-igwn.txt

Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>
acorreia61201 pushed a commit to acorreia61201/pycbc that referenced this pull request Apr 4, 2024
* dqsegdb2 update

* Update requirements-igwn.txt

Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants