Skip to content

Comments

Cpuopt#1

Closed
josh-willis wants to merge 60 commits intogwastro:masterfrom
josh-willis:cpuopt
Closed

Cpuopt#1
josh-willis wants to merge 60 commits intogwastro:masterfrom
josh-willis:cpuopt

Conversation

@josh-willis
Copy link
Contributor

Per LSC redmine issue #2092, this pull request is to merge the cpuopt branch from my personal repository with master on the ligo-cbc repository. A high-level summary of some of the main changes can be found in that redmine issue.

ahnitz and others added 30 commits May 8, 2015 08:44
…ut) callback is enabled which acts to replace the fft
ahnitz pushed a commit that referenced this pull request May 26, 2015
Update and rename docs/pycbc_config_file_instructions.rst to docs/wor…
@ahnitz ahnitz closed this Jun 17, 2015
josh-willis pushed a commit to josh-willis/pycbc that referenced this pull request Jun 20, 2015
Clarify logic of clustered thresholding in comments
a-r-williamson added a commit that referenced this pull request Jun 24, 2015
duncan-brown pushed a commit that referenced this pull request Sep 26, 2015
ahnitz added a commit that referenced this pull request Nov 12, 2015
Added p-values to non-cum plot, and some cosemetic changes
ahnitz pushed a commit that referenced this pull request Jul 29, 2016
* add spa compression; make mchirp calling, return more similar

* fix bug in FilterBank that ignores dtype

* add pycbc_compress_bank

* fixed copyright

* store correct values (#1)

* added precision to compress bank; removed fixed points option

* use end_idx for kmax

* add catch for discontinuity in time estimate

* have segment length computed on the fly

* moved compress_waveform to compress module

* compute segment length using log2

* fix bugs in seg length estimation and indexing

* fix bug in returning sample points; increase seg length for time estimate
duncan-brown pushed a commit that referenced this pull request Feb 17, 2017
…1455)

* E@H: build script (Etch/Cygwin): fixes to PyInstaller

- fix PyINstaller version check
- don't override version information set in configuration section
- make PyInstaller find the Python library on Cygwin instaed of using hardcoded path
- disable patching PyInstaller signal handling for now, as it doesn't help

* Revert "E@H: don't require to set redundant environment variables"

This reverts commit 9a58e4d.

* E@H: don't require to set redundant environment variables

- try calling "pkg-config --version"; if this fails, set NO_PKGCONFIG automatically
- if getting LEVEL2_CACHE_SIZE from environment, don't call getconf even if NO_GETCONF is not set

* E@H: build script (Etch/Cygwin): updated todo list

* E@H: build script (Etch/Cygwin): patch PyInstaller bootloader to not fork a second process

- this allows to control the pycbcb_inspiral process to be controlled by signals, e.g. SIGSTOP

* remove obsolete pycbc_build_cygwin.sh

- was merged with pycbc_build_etch.sh

* Revert "E@H: PyInstaller bootloader patch: pass SIGSTOP and SIGCONT to the client process"

could never work, since SIGSTOP can't be caught or intercepted

This reverts commit 1b04599.

* renamed pycbc_build_etch.sh -> pycbc_build_etch+cygwin.sh

* E@H: added and updated comments

* E@H Etch /usr/local compile: only add archive.debian.org if not already in /etc/apt/sources.list

* E@H: build script (Etch): if not already set, set environment variable LC_ALL from LANG

* E@H: build script (Etch/Cygwin): fixed indentation, only whitespace change

* E@H: build script (Etch/Cygwin): added self test and weave cache build

* E@H: tools for validation

- the validate_test_CBC1 binary is currently built from a private git repo (project-daemons),
  and even withing that the build is somewhat tricky, requiring a couple of files and a patch
  to the Makefile for building on Cygwin. validator_test_CBC1.zip contains binaries for Linux
  and Cygwin.

* E@H: tool to check for GW150914 trigger in a result file (from Tito)

* E@H: build script (Etch/Cygwin): use github/bema-ligo/pycbc repo

- checkout einsteinathome_testing branch
- update every time
- get rid of gitmaster.atlas repos automatially

* E@H: build script (Etch/Cygwin): update pycbc-glue to 0.9.8

* E@H: build script (Etch/Cygwin): added log messages for test analysis

* E@H: build script (Etch/Cygwin): update pycbc-glue to 0.9.8

* E@H: build script (Etch/Cygwin): improvements to test analysis:

- use Tito's GW150914 short template bank
- verify md5 checksums of downlloaded data files
- don't pipe output into additional log file (which hides exit status in the pipe)
- use Tito's script to check result

* E@H: build script (Etch/Cygwin): create 'pythoncompiled.zip' from scratch

* E@H: build script (Etch/Cygwin): added (back) autogating params to test analysis

* E@H: patches to LALSuite, currently unused

* E@H: tool for validation - 5column version

* E@H: build script (Etch/Cygwin): use approximant from Tom's sample analysis for test

* E@H: build script (Etch/Cygwin): patch libframe such that frame files are read in binary mode

* E@H: build script (Etch/Cygwin): allow command-line options:

--no-cleanup      : don't remove build directories after successful build
--no-pycbc-update : don't update local pycbc repo from branch $pycbc_branch
--verbose-python  : run PyInstalled Python in verbose mode, showing imports

* E@H: build script (Etch/Cygwin): use approximant IMRPhenomD for test analysis

- only check md5 of downloaded files once

* E@H: build script (Etch/Cygwin): add SEOBNRv2ChirpTimeSS.dat for approximant IMRPhenomD to weave cache file

* E@H: build script (Etch/Cygwin): properly patch pycbc-glue to initialize *.tp_base at run time

* E@H: build script (Etch/Cygwin): updated FIXME/todo list

* E@H: build script (Etch/Cygwin): when downloading files, remove versions with a wrong checksum first

* E@H: build script: history of pycbc build scripts as previously managed in lalsuite

* E@H: build script (Etch/Cygwin): append 'Linux64' or 'Windows64' to the zipfile names depending on the build

* E@H: files to ignore in tools/einsteinathome

* Revert "E@H hacks: centralize constant weave flags in variable DEFAULT_WEAVE_FLAGS"

This reverts commit 1f613e0.

* E@H: centralize management of WEAVE_FLAGS

- get rid of hardcoded '-march=native -O3 -w' scattered around
- '-march=native -O3 -w' becomes default for WEAVE_FLAGS
- environment WEAVE_FLAGS are added to the default unless these contain '-march=',
  in which case the environment settings override the default

* Revert "E@H: tool to create a fake fstab for the Cygwin process"

This reverts commit d56f431.

* E@H: tool to create a fake fstab for the Cygwin process

- mount points /project and /projects/<project_dir_name>

- cross-compile on a Linux machine with MinGW installed, e.g.:
  ssh wheezy-qemu x86_64-w64-mingw32-gcc -x c - -o fstab.exe < fstab.c &&
  scp wheezy-qemu:fstab.exe .

* E@H: build script (Etch/Cygwin): compile pycbc-glue from git repo

* E@H: build script (Etch/Cygwin): pycbc git repo management

- checkout branch 'master' from ligo-cbc and 'einsteinathome_testing' from bema-ligo
- allow to switch branches with command-line option --bema-testing
- allow to clone from scratch with command-line option --scratch-pycbc

* E@H: build script (Etch/Cygwin): keep SOURCE directory outside the PYCBC directory

* E@H: build script (Etch/Cygwin): keep preinst tarballs in SOURCE directory

* E@H: build script (Etch/Cygwin): unify naming of flags about python module installation

* E@H: progress.c: fixes and improvements

- add help messga ('-h')
- add 'copy' feature ('-c')
- fix stupid typo in command-line parsing

* E@H: build script (Etch/Cygwin): fix indentation

* Revert "E@H: progress.c: fixes and improvements"

This reverts commit 75029ff.

* E@H: progress.c: fixes and improvements

- add help messga ('-h')
- add 'copy' feature ('-c')
- fix stupid typo in command-line parsing

* E@H: build script (Etch/Cygwin): added kombine==0.5.0 to (PyCBC) requirements

* E@H: build script (Etch/Cygwin): on Linux, build BOINC wrapper and 'progress' tool

- On Windows, 'wrapper', 'progress' and 'fstab' require different build environments

* E@H: build script (Etch/Cygwin): build 'progress' and 'fstab' tools on Windows

- needs additional installation of 'mingw64-x86_64-gcc-core' on Cygwin

* E@H: progress.c: finish parsing on SIGTERM (on Linux)

* Revert "E@H: progress.c: finish parsing on SIGTERM (on Linux)"

reverted for aggregation

This reverts commit 7c8b367.

* Revert "E@H: progress.c: fixes and improvements"

reverted for aggregation

This reverts commit a4a7a28.

* E@H: fixes and improvements to progress.c

- fixed stupid typo in command-line parsing
- added 'help' usage messga ('-h')
- added 'copy' feature ('-c')
- finish parsing on SIGTERM (on Linux)

* E@H: build script (Etch/Cygwin): log HEAD of dynamically updated git repos

* E@H: build script (Etch/Cygwin): use local (Atlas) copy of openssl source

* E@H: build script (Etch/Cygwin): fix check for existing BOINC repo

* E@H: build script (Etch/Cygwin): added --clean build feature

* E@H: build script (Etch/Cygwin): set trusted-sites for PIP in environment variable

* E@H: build script (Etch/Cygwin): don't pre-install requirements for pegasus and PyCBC anymore

- helps to adapt to furture changes in requirements for PyCBC

* E@H: build script (Etch/Cygwin): downgrade setuptools to version listed in PyCBC/requirements.txt

- helps to adapt to furture changes in requirements for PyCBC

* E@H: build script (Etch/Cygwin): remove obsolete code

- don't need to build pylal from lalsuite at all

* E@H: build script (Etch/Cygwin): workaround for (apparent) certificate problem with pypi/h5py

- hopefully transient

* E@H Etch /usr/local compile: added libsqlite3-dev to packages

* E@H: build script (Etch/Cygwin): create (empty) 'Tcl/Tk data directories in the bundle'

- I really don't know why the application needs to crash without these

* E@H: build script (Etch/Cygwin): simplify Pegasus-WMS compilation

* E@H: build script (Etch/Cygwin): exit with nonzero status in case of an error

* E@H: build script: additions for OSX build

* E@H: extend fstab to create paths without blanks on OSX (and possibly Linux)

- this creates a symlink '/tmp/BOINC.project.<project-dir-name>
  that will need to be cleaned up (by 'progress')

* E@H: build script: build 'normal' fstab on Linux as well

* E@H: GNU 'error' extensions are not available on OSX

- for now work around by trivial implementation of error() function

* E@H: build script: try using DONT_RESOLVE_LALCACHE_PATH to avoid path with blanks

- see eah_cbc branch in gitmaster/einsteinathome/lalsuite repo

* E@H: build script: use central function for determining md5 checksum of files

* E@H: build script: patch libframe on OSX

- allows to set libframe debug level
- tried to make libframe read frame files from paths with blanks,
  didn't work at all

* Revert "E@H: build script: patch libframe on OSX"

didn't help

This reverts commit 14d531b.

* E@H: build script: use FrameCPP (from ldas-tools-2.4.2) instead of FrameLib (on OSX)

* E@H: build script: force 'einsteinathome_testing' branch on OSX

* E@H: build script: patch ldas-tools .pc to avoid -std=c++ flags

* E@H: build script: don't build wrapper (on Linux), repo is currently not accessible

* E@H: build script: added option --clean-lalsuite

- builds lalsuite from a clean git clone

* E@H: build script: clone official lalsuite master (by default)

- necessary patches have been pushed upstream

* E@H: build script: oops - LAL_FRAME_LIBRARY is an environment variable

... not a compilation flag

* E@H: build script: enable SSE(2) in FFTW compilation

* E@H: build script: pull lalsuite repo for every (clean) build

* E@H: build script: enable AVX in FFTW compilation

* E@H: build script: disable AVX on OSX

- until I figured out how to use the correct assembler

* E@H: (git)ignore OSX64 binaries

* E@H: build script: build zip after adding the XML-softlink

- don't know where that got messed up

* E@H: build script: fix error.h for OSX

- OSX doesn't have a GNU error C extension, so drop an 'error.h' header
  with a fake 'error' function somewhere for scipy wave to pick it up

* Revert "E@H: GNU 'error' extensions are not available on OSX"

This reverts commit a4487ff.

* Revert "E@H: build script: force 'einsteinathome_testing' branch on OSX"

This reverts commit 5366c14

* E@H: build script: log WORKSPACE variable (for Jenkins jobs)

* E@H: build script: build PyInstaller bootloader

- got accidentally disabled in OSX experiments

* E@H: build script: add frontend for a Cygwin build slave for a Jenkins job

* E@H: fix zipping app bundle by reordering platform-specific commands

* E@H: build script: more flexible 'Cygwin frontend'

* E@H: build script: reset current LALSuite HEAD before pulling

* E@H: build script: allow to specify a commit/branch/tag to build lalsuite from

* E@H: build script: log current WORKSPACE value before unsetting it

* E@H: build script: use a login shell when running build remotely

* E@H: build script: fetch only (known) plain files from remote Cygwin host

* E@H: build script: log environment at the beginning

* E@H: build script: update default CYGWIN_HOST

* E@H: build script: add --clean-sundays option (perform a clean build once every sunday)

* E@H: build script: handle remote Cygwin execution before command-line parsing

- command-line arguments should only affect the place where the app is actually built

* E@H build script: added primitive locking mechanism

* E@H: build script: locking needs to happen where the build actually runs (remote Cygwin)

* E@H: build script: fix locking message

* E@H: build script: identify Ubuntu 6.06 as equivalent to Debian 4.0

* E@H: build script: pin Linux gcc version to 4.8.5 (supports AVX)

* E@H: build script: use a local PyCBC source tree if it already exists

* Revert "E@H: build script: use a local PyCBC source tree if it already exists"

didn't work

This reverts commit 66e011d.

* E@H: build script: use a local PyCBC source tree if it already exists

* Revert "E@H: build script: use a local PyCBC source tree if it already exists"

didn't work either

This reverts commit bd731c3.

* E@H: build script: renamed --scratch-pycbc -> --clean-pycbc; --clean-sundays doesn't clean static libraries

* E@H: build script: update AEI git URL, allow to specify 'HEAD' as lalsuite commit

* E@H build script: create the SOURCE directory before dealing with command-line args

* E@H build script: fix cloning lalsuite

* E@H build script: set a local dymmy user before cherry-picking commits in git repos

* E@H build script: moved local source tarballs to AFS to become independent of HFS troubles

* E@H: renamed pycbc_build_etch+cygwin.sh -> pycbc_eah_build.sh

* E@H build: use PyInstaller-tkinter-runtime-hook, also build pycbc_condition_strain onefile bundle

- to be used once #1027 was merged

* E@H build: build pycbc_condition_strain only on Linux

* E@H build: build pycbc_condition_strain bundle with standard (unpatched) bootloader

* E@H build: use scipy runtime-hook for pycbc_condition_strain

* E@H build: always build FrameL

- this only affects the OSX build. Apparently currently not everything
  in LAL uses LALFrame, but some parts (here: LALInspiral) call
  FrameL routines directly. Thus FrameL can't (yet) completely be
  replaced by FrameC.

* E@H build: set gfortran matching gcc version

* E@H build: don't build pylal from lalsuite

* E@H build: build own gcc-4.5.8 and Python 2.7.10 on Linux

* E@H build: minor fix in PATH setting

* E@H build: try linking gcc-4.8.5 instead of building it

* E@H build: clean pycbc/ directory with --clean

* E@H build: OSX: extend compiler flags rather than overriding

* E@H build script: implement proper '--help', only print environment if --print-env is given

* E@H build: link gfortran (if configured) before first call

* E@H build: hdfgroup apparently has certificate problems, use http for now

* E@H build: on OSX, libgfortran.so is named libgfortran.dylib

* E@H build: prepare for using a different BUILD DIR NAME than the generic 'pycbc'

* E@H build: no need to install scipy from git on Linux, use pip-install

* E@H build: check for Cygwin instead of falling back to it

- treat a ssh-remote build as an additional OS option

* E@H build: keep a copy of an unpatched PyInstaller installation

... and restore it after the test analysis, such that a onefile
bundle can be created with that which contains the waeve cache

* E@H build: added option '--force-debian4'

* E@H build: additionally build a pycbc_inspiral onefile PyInstaller bundle

- contains the precompiled weave code cache
- for running on OSG

* E@H build: build the pycbc_inspiral onefile bundle as 'pycbc_inspiral_osg'

* E@H build: there seem to be occasional issues with hdfgroup.org, use local (Atlas) copy of hdf5 source

* E@H set FIXED_WEAVE_CACHE when building the weave code cache

* E@H build: removed unused code (compile gcc)

* E@H build: rename pycbc_inspiral_osg built from a tagged version

* run new pycbc_inspiral command line and allow sl6

* make script executable

* unbundle roms first as p need to be bank file

* removed typo giving duplicate argument

* build SPAtmplt code and symmetric clustering

* E@H build: download test data from Albert

* E@H build: fix typo introduced in bema-ligo PR #1

- soft-link the correct gcc version

* E@H build: simplify & unify Pegasus and MPLD install

* E@H build: get rid of build options no longer used

* E@H build: pegasus.isi.edu has occasional download failures, use AEI copy

* E@H build: change build dir to 'pycbc-build'

* E@H build: add an option to build from a specific --pycbc-commit

* E@H build: rebase DLLs befor building PyCBC

- this _might_ fix a recurring problem with fork+git+pycbe

* E@H build: record the list of ROM files for later use

* add a command line argument to drop extra so files into the bundle

* use curl for file urls

* E@H build: use the correct libgcc_s.1.dylib for the PyInstaller bundle

* E@H build: with --clean, remove pip cache on OSX, too

* E@H build: removed obsolete FIXME message

* E@H build: no need to build pycbc-glue specially anymore

* E@H build: make pegasus source download more robust

* E@H build: ignore warnings on git-describe

* E@H build: Install Jinja2==2.8.1 before PyCBC

- PyInstaller 9d0e0ad4 crashes with newer Jinja2

* E@H build: fix messages installing PyCBC

* E@H build: special h5py treatment is no longer needed

* E@H build: apparently six and pkgconfig still need to be installed separately

* E@H build: fix check for libgcc_s.1.dylib

- in case we need to use a different gcc version on OSX

* E@H build script: updated source URLs

* E@H build: bump up FFTW version (3.3.3 -> 3.3.5)

* E@H build script: updated log messages

* E@H build: enable AVX for FFTW on OSX

* E@H build: don't need to build mpld3-0.3git anymore

* E@H build script: reworked comment about rebasing DLLs

* E@H build: install particular version of matplotlib

- as specified in pycbc/requirements.txt

* E@H build: code to manually build subprocess32

- in case we need matplotlib>=2.0.0

* E@H: link gcc _after_ evaluating command-line options

... or else the gcc links will be removed with e.g. --clean

* E@H build script: set common defauls for OS-specific settings

- should make the OS-specific sections shorter and more readable

* E@H build: add a switch whether to rebase DLLs before git-checkout PyCBC

* E@H build: log when building tarballs (takes a while...)

* E@H Etch /usr/local compile: updated URLs, install JRE1.6.45u for Jenkins

* E@H build: link gcc on OSX, always set FC+CC+CXX, updated PyPI URL

* E@H build: downgrading setuptools seems no longer necessary

* E@H build: update bootloader patch for PyInstaller>=3.0

* E@H build: set PYCBC_HOOKS_DIR

- required if mods to hook-pycbc #1390 is merged

* Revert "E@H build: downgrading setuptools seems no longer necessary"

still needed, it seems, at least for PyInstaller 2.1

This reverts commit fd7c925.

* Revert "E@H build: link gcc on OSX, always set FC+CC+CXX, updated PyPI URL"

reverted to urgently fix OSX build, apparently needs rethinking

This reverts commit 4b687c4.

* E@H build: updated PyPI URL

* E@H build: always set CC,CXX

* E@H build: PyInstaller: always git clean before build, handle version=HEAD

* E@H build: clean git directory before building PyCBC

* E@H build: added option --no-anlysis

* E@H build: mark all the hacks necessary to use PyInstller <3.0

- so that these can be easilly switched off when no longer needed

* E@H build: fix subprocess32 build

* allow user to specify a remote and branch

* add help message

* fixed remote adding

* use git protocol not http

* don't fail on first checkout

* E@H: build script: history of pycbc build scripts as previously managed in lalsuite

- updated, previous version contained mostly empty files

* E@H: build: update PyInstaller version to 3.2.1

* Revert "E@H: patches to LALSuite, currently unused"

This reverts commit d7a895c.

* Revert "E@H: build script (Etch/Cygwin): compile scipy from source on Cygwin"

This reverts commit d2dbf48.

* Revert "E@H: glob.h/c source, might become helpful"

This reverts commit ea68b9b.

* E@H: fstab: removed obsolete code & added a header

* E@H: updated READMEs

* E@H build: update PyInstaller git repo if already cloned

* E@H build: back to PyInstaller 2.1 on OSX

- currently can't compile the bootloader of PyI 3.2.1:

  pyi_utils.c:852:4: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    OSErr err = AEGetParamDesc(theAppleEvent, keyDirectObject, typeAEList, &docList);

* E@H build: use PyInstaller 3.2.1 for OSX, but not (yet) for Cygwin

- the OSX bootloader variant 'debugw' regularly uses declarations mixed in code, so disable
  -Wdeclaration-after-statement when compiling with -Werror on OSX

- PyInstaller 3.2.1 fails badly on Cygwin (
  76187 INFO: Looking for dynamic libraries:
  76609 WARNING: Can not get binary dependencies for file:
  PyInstaller/lib/pefile.py", line 26, in <module>
    from past.builtins import cmp, long
  ImportError: No module named past.builtins
  IOError: Python library not found: libpython2.7m.dll, libpython2.7.dll, libpython27m.dll, libpython27.dll

* E@H build: update hdf5 version to 1.8.13

- got a version mismatch error 1.8.12 vs. 1.8.13, not sure where the latter comes from

* E@H build: pkgconfig still needs to be installed manually

* E@H build: always pull 'master' from ligo-cbc

* E@H build: don't untar ROMs if these are already there

* add extra include path to find Python.h

* use PYTHON_PREFIX rather than PREFIX

* allow the user to specific an extra bank file and approximant

* skip empty approximants and banks

* fixed debugging message

* use bash arrays to avoid nested for loops

* restore accidentally deleted line

* allow user to specify an alternate rom path

* E@H build: appaently FFTW's AVX code segfaults when CFLAGS contains a stray ' '

- happens only when then analysis is executed on an AVX-capable machine

* E@H build script: remove special treatment of bema-ligo remote

- also fix bug introduced in commit:273e766c that makes Cygwin's git crash

* E@H build: trying to work with some strangely behaving git versions

* E@H build script: avoid executing while loop in a pipe (subshell)
 using a HERE document

* E@H build: back to PyInstaller 2.1 on Deb4 until Python.h issue is solved

* E@H: avoid name clash in f00701a

* E@H build: fix weave include path for PyInstaller>=3.0

- see #6

* E@H build: fix weave include path for PyInstaller>=3.0

- see #6

* E@H build: implement changes suggested in #6

"Allow the user to specific an extra bank file and approximant"

* E@H build script: added option --clean-weave-cache

* E@H build: updated PyPi download URLs

* Revert "E@H build: back to PyInstaller 2.1 on Deb4 until Python.h issue is solved"

This reverts commit b94495c.

* E@H build: fix --clean-weave-cache

* E@H: avoid overloading reserved name in CBC1H5tool.py

* E@H build script: reformat help message
hagabbar referenced this pull request in hagabbar/pycbc_copy May 24, 2017
…wastro#1455)

* E@H: build script (Etch/Cygwin): fixes to PyInstaller

- fix PyINstaller version check
- don't override version information set in configuration section
- make PyInstaller find the Python library on Cygwin instaed of using hardcoded path
- disable patching PyInstaller signal handling for now, as it doesn't help

* Revert "E@H: don't require to set redundant environment variables"

This reverts commit 9a58e4d.

* E@H: don't require to set redundant environment variables

- try calling "pkg-config --version"; if this fails, set NO_PKGCONFIG automatically
- if getting LEVEL2_CACHE_SIZE from environment, don't call getconf even if NO_GETCONF is not set

* E@H: build script (Etch/Cygwin): updated todo list

* E@H: build script (Etch/Cygwin): patch PyInstaller bootloader to not fork a second process

- this allows to control the pycbcb_inspiral process to be controlled by signals, e.g. SIGSTOP

* remove obsolete pycbc_build_cygwin.sh

- was merged with pycbc_build_etch.sh

* Revert "E@H: PyInstaller bootloader patch: pass SIGSTOP and SIGCONT to the client process"

could never work, since SIGSTOP can't be caught or intercepted

This reverts commit 1b04599.

* renamed pycbc_build_etch.sh -> pycbc_build_etch+cygwin.sh

* E@H: added and updated comments

* E@H Etch /usr/local compile: only add archive.debian.org if not already in /etc/apt/sources.list

* E@H: build script (Etch): if not already set, set environment variable LC_ALL from LANG

* E@H: build script (Etch/Cygwin): fixed indentation, only whitespace change

* E@H: build script (Etch/Cygwin): added self test and weave cache build

* E@H: tools for validation

- the validate_test_CBC1 binary is currently built from a private git repo (project-daemons),
  and even withing that the build is somewhat tricky, requiring a couple of files and a patch
  to the Makefile for building on Cygwin. validator_test_CBC1.zip contains binaries for Linux
  and Cygwin.

* E@H: tool to check for GW150914 trigger in a result file (from Tito)

* E@H: build script (Etch/Cygwin): use github/bema-ligo/pycbc repo

- checkout einsteinathome_testing branch
- update every time
- get rid of gitmaster.atlas repos automatially

* E@H: build script (Etch/Cygwin): update pycbc-glue to 0.9.8

* E@H: build script (Etch/Cygwin): added log messages for test analysis

* E@H: build script (Etch/Cygwin): update pycbc-glue to 0.9.8

* E@H: build script (Etch/Cygwin): improvements to test analysis:

- use Tito's GW150914 short template bank
- verify md5 checksums of downlloaded data files
- don't pipe output into additional log file (which hides exit status in the pipe)
- use Tito's script to check result

* E@H: build script (Etch/Cygwin): create 'pythoncompiled.zip' from scratch

* E@H: build script (Etch/Cygwin): added (back) autogating params to test analysis

* E@H: patches to LALSuite, currently unused

* E@H: tool for validation - 5column version

* E@H: build script (Etch/Cygwin): use approximant from Tom's sample analysis for test

* E@H: build script (Etch/Cygwin): patch libframe such that frame files are read in binary mode

* E@H: build script (Etch/Cygwin): allow command-line options:

--no-cleanup      : don't remove build directories after successful build
--no-pycbc-update : don't update local pycbc repo from branch $pycbc_branch
--verbose-python  : run PyInstalled Python in verbose mode, showing imports

* E@H: build script (Etch/Cygwin): use approximant IMRPhenomD for test analysis

- only check md5 of downloaded files once

* E@H: build script (Etch/Cygwin): add SEOBNRv2ChirpTimeSS.dat for approximant IMRPhenomD to weave cache file

* E@H: build script (Etch/Cygwin): properly patch pycbc-glue to initialize *.tp_base at run time

* E@H: build script (Etch/Cygwin): updated FIXME/todo list

* E@H: build script (Etch/Cygwin): when downloading files, remove versions with a wrong checksum first

* E@H: build script: history of pycbc build scripts as previously managed in lalsuite

* E@H: build script (Etch/Cygwin): append 'Linux64' or 'Windows64' to the zipfile names depending on the build

* E@H: files to ignore in tools/einsteinathome

* Revert "E@H hacks: centralize constant weave flags in variable DEFAULT_WEAVE_FLAGS"

This reverts commit 1f613e0.

* E@H: centralize management of WEAVE_FLAGS

- get rid of hardcoded '-march=native -O3 -w' scattered around
- '-march=native -O3 -w' becomes default for WEAVE_FLAGS
- environment WEAVE_FLAGS are added to the default unless these contain '-march=',
  in which case the environment settings override the default

* Revert "E@H: tool to create a fake fstab for the Cygwin process"

This reverts commit d56f431.

* E@H: tool to create a fake fstab for the Cygwin process

- mount points /project and /projects/<project_dir_name>

- cross-compile on a Linux machine with MinGW installed, e.g.:
  ssh wheezy-qemu x86_64-w64-mingw32-gcc -x c - -o fstab.exe < fstab.c &&
  scp wheezy-qemu:fstab.exe .

* E@H: build script (Etch/Cygwin): compile pycbc-glue from git repo

* E@H: build script (Etch/Cygwin): pycbc git repo management

- checkout branch 'master' from ligo-cbc and 'einsteinathome_testing' from bema-ligo
- allow to switch branches with command-line option --bema-testing
- allow to clone from scratch with command-line option --scratch-pycbc

* E@H: build script (Etch/Cygwin): keep SOURCE directory outside the PYCBC directory

* E@H: build script (Etch/Cygwin): keep preinst tarballs in SOURCE directory

* E@H: build script (Etch/Cygwin): unify naming of flags about python module installation

* E@H: progress.c: fixes and improvements

- add help messga ('-h')
- add 'copy' feature ('-c')
- fix stupid typo in command-line parsing

* E@H: build script (Etch/Cygwin): fix indentation

* Revert "E@H: progress.c: fixes and improvements"

This reverts commit 75029ff.

* E@H: progress.c: fixes and improvements

- add help messga ('-h')
- add 'copy' feature ('-c')
- fix stupid typo in command-line parsing

* E@H: build script (Etch/Cygwin): added kombine==0.5.0 to (PyCBC) requirements

* E@H: build script (Etch/Cygwin): on Linux, build BOINC wrapper and 'progress' tool

- On Windows, 'wrapper', 'progress' and 'fstab' require different build environments

* E@H: build script (Etch/Cygwin): build 'progress' and 'fstab' tools on Windows

- needs additional installation of 'mingw64-x86_64-gcc-core' on Cygwin

* E@H: progress.c: finish parsing on SIGTERM (on Linux)

* Revert "E@H: progress.c: finish parsing on SIGTERM (on Linux)"

reverted for aggregation

This reverts commit 7c8b367.

* Revert "E@H: progress.c: fixes and improvements"

reverted for aggregation

This reverts commit a4a7a28.

* E@H: fixes and improvements to progress.c

- fixed stupid typo in command-line parsing
- added 'help' usage messga ('-h')
- added 'copy' feature ('-c')
- finish parsing on SIGTERM (on Linux)

* E@H: build script (Etch/Cygwin): log HEAD of dynamically updated git repos

* E@H: build script (Etch/Cygwin): use local (Atlas) copy of openssl source

* E@H: build script (Etch/Cygwin): fix check for existing BOINC repo

* E@H: build script (Etch/Cygwin): added --clean build feature

* E@H: build script (Etch/Cygwin): set trusted-sites for PIP in environment variable

* E@H: build script (Etch/Cygwin): don't pre-install requirements for pegasus and PyCBC anymore

- helps to adapt to furture changes in requirements for PyCBC

* E@H: build script (Etch/Cygwin): downgrade setuptools to version listed in PyCBC/requirements.txt

- helps to adapt to furture changes in requirements for PyCBC

* E@H: build script (Etch/Cygwin): remove obsolete code

- don't need to build pylal from lalsuite at all

* E@H: build script (Etch/Cygwin): workaround for (apparent) certificate problem with pypi/h5py

- hopefully transient

* E@H Etch /usr/local compile: added libsqlite3-dev to packages

* E@H: build script (Etch/Cygwin): create (empty) 'Tcl/Tk data directories in the bundle'

- I really don't know why the application needs to crash without these

* E@H: build script (Etch/Cygwin): simplify Pegasus-WMS compilation

* E@H: build script (Etch/Cygwin): exit with nonzero status in case of an error

* E@H: build script: additions for OSX build

* E@H: extend fstab to create paths without blanks on OSX (and possibly Linux)

- this creates a symlink '/tmp/BOINC.project.<project-dir-name>
  that will need to be cleaned up (by 'progress')

* E@H: build script: build 'normal' fstab on Linux as well

* E@H: GNU 'error' extensions are not available on OSX

- for now work around by trivial implementation of error() function

* E@H: build script: try using DONT_RESOLVE_LALCACHE_PATH to avoid path with blanks

- see eah_cbc branch in gitmaster/einsteinathome/lalsuite repo

* E@H: build script: use central function for determining md5 checksum of files

* E@H: build script: patch libframe on OSX

- allows to set libframe debug level
- tried to make libframe read frame files from paths with blanks,
  didn't work at all

* Revert "E@H: build script: patch libframe on OSX"

didn't help

This reverts commit 14d531b.

* E@H: build script: use FrameCPP (from ldas-tools-2.4.2) instead of FrameLib (on OSX)

* E@H: build script: force 'einsteinathome_testing' branch on OSX

* E@H: build script: patch ldas-tools .pc to avoid -std=c++ flags

* E@H: build script: don't build wrapper (on Linux), repo is currently not accessible

* E@H: build script: added option --clean-lalsuite

- builds lalsuite from a clean git clone

* E@H: build script: clone official lalsuite master (by default)

- necessary patches have been pushed upstream

* E@H: build script: oops - LAL_FRAME_LIBRARY is an environment variable

... not a compilation flag

* E@H: build script: enable SSE(2) in FFTW compilation

* E@H: build script: pull lalsuite repo for every (clean) build

* E@H: build script: enable AVX in FFTW compilation

* E@H: build script: disable AVX on OSX

- until I figured out how to use the correct assembler

* E@H: (git)ignore OSX64 binaries

* E@H: build script: build zip after adding the XML-softlink

- don't know where that got messed up

* E@H: build script: fix error.h for OSX

- OSX doesn't have a GNU error C extension, so drop an 'error.h' header
  with a fake 'error' function somewhere for scipy wave to pick it up

* Revert "E@H: GNU 'error' extensions are not available on OSX"

This reverts commit a4487ff.

* Revert "E@H: build script: force 'einsteinathome_testing' branch on OSX"

This reverts commit 5366c14

* E@H: build script: log WORKSPACE variable (for Jenkins jobs)

* E@H: build script: build PyInstaller bootloader

- got accidentally disabled in OSX experiments

* E@H: build script: add frontend for a Cygwin build slave for a Jenkins job

* E@H: fix zipping app bundle by reordering platform-specific commands

* E@H: build script: more flexible 'Cygwin frontend'

* E@H: build script: reset current LALSuite HEAD before pulling

* E@H: build script: allow to specify a commit/branch/tag to build lalsuite from

* E@H: build script: log current WORKSPACE value before unsetting it

* E@H: build script: use a login shell when running build remotely

* E@H: build script: fetch only (known) plain files from remote Cygwin host

* E@H: build script: log environment at the beginning

* E@H: build script: update default CYGWIN_HOST

* E@H: build script: add --clean-sundays option (perform a clean build once every sunday)

* E@H: build script: handle remote Cygwin execution before command-line parsing

- command-line arguments should only affect the place where the app is actually built

* E@H build script: added primitive locking mechanism

* E@H: build script: locking needs to happen where the build actually runs (remote Cygwin)

* E@H: build script: fix locking message

* E@H: build script: identify Ubuntu 6.06 as equivalent to Debian 4.0

* E@H: build script: pin Linux gcc version to 4.8.5 (supports AVX)

* E@H: build script: use a local PyCBC source tree if it already exists

* Revert "E@H: build script: use a local PyCBC source tree if it already exists"

didn't work

This reverts commit 66e011d.

* E@H: build script: use a local PyCBC source tree if it already exists

* Revert "E@H: build script: use a local PyCBC source tree if it already exists"

didn't work either

This reverts commit bd731c3.

* E@H: build script: renamed --scratch-pycbc -> --clean-pycbc; --clean-sundays doesn't clean static libraries

* E@H: build script: update AEI git URL, allow to specify 'HEAD' as lalsuite commit

* E@H build script: create the SOURCE directory before dealing with command-line args

* E@H build script: fix cloning lalsuite

* E@H build script: set a local dymmy user before cherry-picking commits in git repos

* E@H build script: moved local source tarballs to AFS to become independent of HFS troubles

* E@H: renamed pycbc_build_etch+cygwin.sh -> pycbc_eah_build.sh

* E@H build: use PyInstaller-tkinter-runtime-hook, also build pycbc_condition_strain onefile bundle

- to be used once gwastro#1027 was merged

* E@H build: build pycbc_condition_strain only on Linux

* E@H build: build pycbc_condition_strain bundle with standard (unpatched) bootloader

* E@H build: use scipy runtime-hook for pycbc_condition_strain

* E@H build: always build FrameL

- this only affects the OSX build. Apparently currently not everything
  in LAL uses LALFrame, but some parts (here: LALInspiral) call
  FrameL routines directly. Thus FrameL can't (yet) completely be
  replaced by FrameC.

* E@H build: set gfortran matching gcc version

* E@H build: don't build pylal from lalsuite

* E@H build: build own gcc-4.5.8 and Python 2.7.10 on Linux

* E@H build: minor fix in PATH setting

* E@H build: try linking gcc-4.8.5 instead of building it

* E@H build: clean pycbc/ directory with --clean

* E@H build: OSX: extend compiler flags rather than overriding

* E@H build script: implement proper '--help', only print environment if --print-env is given

* E@H build: link gfortran (if configured) before first call

* E@H build: hdfgroup apparently has certificate problems, use http for now

* E@H build: on OSX, libgfortran.so is named libgfortran.dylib

* E@H build: prepare for using a different BUILD DIR NAME than the generic 'pycbc'

* E@H build: no need to install scipy from git on Linux, use pip-install

* E@H build: check for Cygwin instead of falling back to it

- treat a ssh-remote build as an additional OS option

* E@H build: keep a copy of an unpatched PyInstaller installation

... and restore it after the test analysis, such that a onefile
bundle can be created with that which contains the waeve cache

* E@H build: added option '--force-debian4'

* E@H build: additionally build a pycbc_inspiral onefile PyInstaller bundle

- contains the precompiled weave code cache
- for running on OSG

* E@H build: build the pycbc_inspiral onefile bundle as 'pycbc_inspiral_osg'

* E@H build: there seem to be occasional issues with hdfgroup.org, use local (Atlas) copy of hdf5 source

* E@H set FIXED_WEAVE_CACHE when building the weave code cache

* E@H build: removed unused code (compile gcc)

* E@H build: rename pycbc_inspiral_osg built from a tagged version

* run new pycbc_inspiral command line and allow sl6

* make script executable

* unbundle roms first as p need to be bank file

* removed typo giving duplicate argument

* build SPAtmplt code and symmetric clustering

* E@H build: download test data from Albert

* E@H build: fix typo introduced in bema-ligo PR #1

- soft-link the correct gcc version

* E@H build: simplify & unify Pegasus and MPLD install

* E@H build: get rid of build options no longer used

* E@H build: pegasus.isi.edu has occasional download failures, use AEI copy

* E@H build: change build dir to 'pycbc-build'

* E@H build: add an option to build from a specific --pycbc-commit

* E@H build: rebase DLLs befor building PyCBC

- this _might_ fix a recurring problem with fork+git+pycbe

* E@H build: record the list of ROM files for later use

* add a command line argument to drop extra so files into the bundle

* use curl for file urls

* E@H build: use the correct libgcc_s.1.dylib for the PyInstaller bundle

* E@H build: with --clean, remove pip cache on OSX, too

* E@H build: removed obsolete FIXME message

* E@H build: no need to build pycbc-glue specially anymore

* E@H build: make pegasus source download more robust

* E@H build: ignore warnings on git-describe

* E@H build: Install Jinja2==2.8.1 before PyCBC

- PyInstaller 9d0e0ad4 crashes with newer Jinja2

* E@H build: fix messages installing PyCBC

* E@H build: special h5py treatment is no longer needed

* E@H build: apparently six and pkgconfig still need to be installed separately

* E@H build: fix check for libgcc_s.1.dylib

- in case we need to use a different gcc version on OSX

* E@H build script: updated source URLs

* E@H build: bump up FFTW version (3.3.3 -> 3.3.5)

* E@H build script: updated log messages

* E@H build: enable AVX for FFTW on OSX

* E@H build: don't need to build mpld3-0.3git anymore

* E@H build script: reworked comment about rebasing DLLs

* E@H build: install particular version of matplotlib

- as specified in pycbc/requirements.txt

* E@H build: code to manually build subprocess32

- in case we need matplotlib>=2.0.0

* E@H: link gcc _after_ evaluating command-line options

... or else the gcc links will be removed with e.g. --clean

* E@H build script: set common defauls for OS-specific settings

- should make the OS-specific sections shorter and more readable

* E@H build: add a switch whether to rebase DLLs before git-checkout PyCBC

* E@H build: log when building tarballs (takes a while...)

* E@H Etch /usr/local compile: updated URLs, install JRE1.6.45u for Jenkins

* E@H build: link gcc on OSX, always set FC+CC+CXX, updated PyPI URL

* E@H build: downgrading setuptools seems no longer necessary

* E@H build: update bootloader patch for PyInstaller>=3.0

* E@H build: set PYCBC_HOOKS_DIR

- required if mods to hook-pycbc gwastro#1390 is merged

* Revert "E@H build: downgrading setuptools seems no longer necessary"

still needed, it seems, at least for PyInstaller 2.1

This reverts commit fd7c925.

* Revert "E@H build: link gcc on OSX, always set FC+CC+CXX, updated PyPI URL"

reverted to urgently fix OSX build, apparently needs rethinking

This reverts commit 4b687c4.

* E@H build: updated PyPI URL

* E@H build: always set CC,CXX

* E@H build: PyInstaller: always git clean before build, handle version=HEAD

* E@H build: clean git directory before building PyCBC

* E@H build: added option --no-anlysis

* E@H build: mark all the hacks necessary to use PyInstller <3.0

- so that these can be easilly switched off when no longer needed

* E@H build: fix subprocess32 build

* allow user to specify a remote and branch

* add help message

* fixed remote adding

* use git protocol not http

* don't fail on first checkout

* E@H: build script: history of pycbc build scripts as previously managed in lalsuite

- updated, previous version contained mostly empty files

* E@H: build: update PyInstaller version to 3.2.1

* Revert "E@H: patches to LALSuite, currently unused"

This reverts commit d7a895c.

* Revert "E@H: build script (Etch/Cygwin): compile scipy from source on Cygwin"

This reverts commit d2dbf48.

* Revert "E@H: glob.h/c source, might become helpful"

This reverts commit ea68b9b.

* E@H: fstab: removed obsolete code & added a header

* E@H: updated READMEs

* E@H build: update PyInstaller git repo if already cloned

* E@H build: back to PyInstaller 2.1 on OSX

- currently can't compile the bootloader of PyI 3.2.1:

  pyi_utils.c:852:4: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    OSErr err = AEGetParamDesc(theAppleEvent, keyDirectObject, typeAEList, &docList);

* E@H build: use PyInstaller 3.2.1 for OSX, but not (yet) for Cygwin

- the OSX bootloader variant 'debugw' regularly uses declarations mixed in code, so disable
  -Wdeclaration-after-statement when compiling with -Werror on OSX

- PyInstaller 3.2.1 fails badly on Cygwin (
  76187 INFO: Looking for dynamic libraries:
  76609 WARNING: Can not get binary dependencies for file:
  PyInstaller/lib/pefile.py", line 26, in <module>
    from past.builtins import cmp, long
  ImportError: No module named past.builtins
  IOError: Python library not found: libpython2.7m.dll, libpython2.7.dll, libpython27m.dll, libpython27.dll

* E@H build: update hdf5 version to 1.8.13

- got a version mismatch error 1.8.12 vs. 1.8.13, not sure where the latter comes from

* E@H build: pkgconfig still needs to be installed manually

* E@H build: always pull 'master' from ligo-cbc

* E@H build: don't untar ROMs if these are already there

* add extra include path to find Python.h

* use PYTHON_PREFIX rather than PREFIX

* allow the user to specific an extra bank file and approximant

* skip empty approximants and banks

* fixed debugging message

* use bash arrays to avoid nested for loops

* restore accidentally deleted line

* allow user to specify an alternate rom path

* E@H build: appaently FFTW's AVX code segfaults when CFLAGS contains a stray ' '

- happens only when then analysis is executed on an AVX-capable machine

* E@H build script: remove special treatment of bema-ligo remote

- also fix bug introduced in commit:273e766c that makes Cygwin's git crash

* E@H build: trying to work with some strangely behaving git versions

* E@H build script: avoid executing while loop in a pipe (subshell)
 using a HERE document

* E@H build: back to PyInstaller 2.1 on Deb4 until Python.h issue is solved

* E@H: avoid name clash in f00701a

* E@H build: fix weave include path for PyInstaller>=3.0

- see gwastro#6

* E@H build: fix weave include path for PyInstaller>=3.0

- see gwastro#6

* E@H build: implement changes suggested in gwastro#6

"Allow the user to specific an extra bank file and approximant"

* E@H build script: added option --clean-weave-cache

* E@H build: updated PyPi download URLs

* Revert "E@H build: back to PyInstaller 2.1 on Deb4 until Python.h issue is solved"

This reverts commit b94495c.

* E@H build: fix --clean-weave-cache

* E@H: avoid overloading reserved name in CBC1H5tool.py

* E@H build script: reformat help message
tdent referenced this pull request in tdent/pycbc Aug 22, 2017
fix time offset and length
dethodav referenced this pull request in dethodav/pycbc Oct 13, 2020
Add support for dynesty's dynamic nested sampler (gwastro#3288)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants