Skip to content

Commit 7c1d364

Browse files
martinkilbingermartinkilbinger
authored
Merge pull request #116 from martinkilbinger/pre_v2
ShapePipe v1.3 v1.4 validation, post-processing, cosmology inference
2 parents 7ab0001 + 9d5a5b3 commit 7c1d364

File tree

71 files changed

+3727
-10689
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

71 files changed

+3727
-10689
lines changed

‎.gitignore

+8
Original file line numberDiff line numberDiff line change
@@ -150,3 +150,11 @@ notebooks/cosmo_val/cosmo_val.ipynb
150150
notebooks/cosmo_val/match_LF_SP.ipynb
151151

152152
notebooks/examples/*.ipynb
153+
154+
cosmo_inference/data/
155+
cosmo_inference/cosmocov_config/cosmocov_*
156+
cosmo_inference/cosmosis_config/cosmosis_pipeline_*
157+
cosmo_inference/cosmosis_config/priors_*
158+
cosmo_inference/cosmosis_config/values_*
159+
160+
notebooks/tests/star_response.ipynb

‎cosmo_inference/README.md

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# UNIONS Cosmological Inference Pipeline
2+
by Lisa Goh, CEA Paris-Saclay
3+
4+
This folder contains the files neccessary to run the cosmological inference pipeline on the UNIONS galaxy catalogues.
5+
6+
### Requirements
7+
To run the pipeline, one would need to have installed [CosmoSIS](https://cosmosis.readthedocs.io/en/latest/) and [CosmoCov](https://github.com/CosmoLike/CosmoCov).
8+
9+
### To Run
10+
Run the bash script within this folder
11+
12+
```
13+
$ ./pipeline.sh
14+
```
15+
with one of the following flags:
16+
17+
`--pcf`: This step runs the `cosmo_val.py` script to calculate the various 2 point correlation functions. It will also write the $\xi_{pm}$ correlation functions in a fits file.
18+
19+
`--covmat`: The covariance matrix is calculated here using CosmoCov, by reading in the `./cosmocov_config/cosmocov_{output_root}.ini` file. **Hence make sure the `output_root` here corresponds to the one entered in the prompt**.
20+
21+
`--inference`: This step writes out the relevant `./cosmosis_config/cosmosis_{output_root}.ini` file, in order to run CosmoSIS to conduct the cosmological inference. It also combines the data needed by CosmoSIS: the $\xi_{pm}$ fits files calculated in `cosmo_val.py`, the covariance matrix, and the nz catalogue, into a single `.fits` file. You can submit the job submission bash script to run CosmoSIS on your cluster. Here, an example `submit.sh` script is provided (assuming SLURM architecture, currently running on CEA feynman cluster).
22+
23+
`--mcmc_process`: You can finally analyse the chains with the `MCMC.ipynb` notebook.
24+
25+

‎cosmo_inference/cosmocov_config/cosmocov.ini

+57
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Cosmological parameters
2+
#
3+
Omega_m : 0.25
4+
Omega_v : 0.75
5+
sigma_8 : 0.8
6+
n_spec : 0.95
7+
w0 : -1
8+
wa : 0
9+
omb : 0.044
10+
h0 : 0.7
11+
12+
13+
# Survey and galaxy parameters
14+
15+
# area in degrees
16+
# n_gal,lens_n_gal in gals/arcmin^2
17+
18+
; FOR SHAPEPIPE 3500
19+
; area : 3218.19
20+
; sourcephotoz : multihisto
21+
; lensphotoz : multihisto
22+
; source_tomobins : 1
23+
; lens_tomobins : 1
24+
; sigma_e : 0.491712
25+
; source_n_gal : 8.42
26+
27+
; FOR SHAPEPIPE 1500
28+
area : 1453
29+
sourcephotoz : multihisto
30+
lensphotoz : multihisto
31+
source_tomobins : 1
32+
lens_tomobins : 1
33+
sigma_e : 0.4808326112068524
34+
source_n_gal : 7.92
35+
36+
# IA parameters
37+
IA : 1
38+
A_ia : 0.0
39+
eta_ia : 0.0
40+
oneplusz0_ia: 0.6
41+
42+
# Covariance paramters
43+
#
44+
# tmin,tmax in arcminutes
45+
tmin : 1
46+
tmax : 200
47+
ntheta : 20
48+
49+
# whether to calculate the non-Gaussian part as well
50+
ng : 1
51+
cng : 1
52+
53+
# covariances for shear-shear, shear-pos or pos-pos
54+
ss : true
55+
ls : false
56+
ll : false
57+
filename : out_cov

‎cosmo_inference/cosmosis_config/cosmosis_pipeline.ini

+88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
#parameters used elsewhere in this file
2+
[DEFAULT]
3+
# Specify the directory of your cosmological CosmoSIS library
4+
COSMOSIS_DIR = /home/guerrini/cosmosis-standard-library
5+
6+
7+
[pipeline]
8+
modules = consistency camb load_nz_fits linear_alignment projection 2pt_shear add_xi_sys 2pt_like
9+
likelihoods = 2pt_like
10+
extra_output = cosmological_parameters/omega_lambda cosmological_parameters/S_8
11+
timing = T
12+
debug = T
13+
14+
[runtime]
15+
sampler = metropolis
16+
resume = T
17+
verbosity = debug
18+
19+
[output]
20+
format = text
21+
lock = F
22+
23+
[metropolis]
24+
samples = 10000000
25+
26+
[consistency]
27+
file = %(COSMOSIS_DIR)s/utility/consistency/consistency_interface.py
28+
verbose = F
29+
30+
[camb]
31+
file = %(COSMOSIS_DIR)s/boltzmann/camb/camb_interface.py
32+
mode=all
33+
lmax=2508
34+
feedback=0
35+
do_reionization=F
36+
kmin=1e-5
37+
kmax=20.0
38+
nk=200
39+
zmax=5.0
40+
zmax_background=5.0
41+
nz_background=500
42+
halofit_version=takahashi
43+
nonlinear=pk
44+
neutrino_hierarchy=normal
45+
kmax_extrapolate = 500.0
46+
47+
[load_nz_fits]
48+
file = %(COSMOSIS_DIR)s/number_density/load_nz_fits/load_nz_fits.py
49+
nz_file =%(FITS_FILE)s
50+
data_sets = SOURCE
51+
52+
[linear_alignment]
53+
file = %(COSMOSIS_DIR)s/intrinsic_alignments/la_model/linear_alignments_interface.py
54+
method = bk_corrected
55+
56+
[projection]
57+
file = %(COSMOSIS_DIR)s/structure/projection/project_2d.py
58+
ell_min_logspaced = 1.0
59+
ell_max_logspaced = 25000.0
60+
n_ell_logspaced = 400
61+
shear-shear = source-source
62+
get_kernel_peaks = F
63+
verbose = F
64+
65+
[2pt_shear]
66+
file = %(COSMOSIS_DIR)s/shear/cl_to_xi_nicaea/nicaea_interface.so
67+
corr_type = 0 ; shear_cl -> shear_xi
68+
69+
[add_xi_sys]
70+
file = %(COSMOSIS_DIR)s/shear/xi_sys/xi_sys_psf.py
71+
data_file=%(FITS FILE)s
72+
rho_stats_name=RHO_STATS
73+
xi_plus_name=XI_PLUS
74+
xi_minus_name=XI_MINUS
75+
samples=%(SAMPLES_FILE)s
76+
77+
78+
[2pt_like]
79+
file = %(COSMOSIS_DIR)s/likelihood/2pt/2pt_like.py
80+
data_file=%(FITS_FILE)s
81+
gaussian_covariance=F
82+
covmat_name=COVMAT
83+
cut_zeros=F
84+
data_sets=XI_PLUS XI_MINUS
85+
like_name=2pt_like
86+
87+
angle_range_XI_PLUS_1_1= 1.0 200.0
88+
angle_range_XI_MINUS_1_1= 1.0 200.0

‎cosmo_inference/cosmosis_config/priors.ini

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
[cosmological parameters]

‎cosmo_inference/cosmosis_config/values.ini

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
[cosmological_parameters]
2+
omega_m = 0.05 0.29 0.7
3+
sigma_8_input = 0.2 0.8 1.8
4+
5+
h0 = 0.6727
6+
ombh2 = 0.02236
7+
n_s = 0.9649
8+
A_s = 2.101e-09
9+
tau = 0.0544
10+
11+
w = -1.0
12+
massive_nu = 1
13+
massless_nu = 2.046
14+
omega_k = 0.0
15+
wa = 0.0
16+
17+
18+
[intrinsic_alignment_parameters]
19+
A = -3.0 1.0 3.0
20+
21+
# [shear_calibration_parameters]
22+
; #m0 = -0.8 0.012 0.8

‎cosmo_inference/notebooks/MCMC.ipynb

+274
Large diffs are not rendered by default.

‎cosmo_inference/pipeline.sh

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
#!/bin/bash
2+
3+
# Transform long options to short ones
4+
for arg in "$@"; do
5+
shift
6+
case "$arg" in
7+
'--help') set -- "$@" '-h' ;;
8+
'--pcf') set -- "$@" '-p' ;;
9+
'--covmat') set -- "$@" '-c' ;;
10+
'--inference') set -- "$@" '-i' ;;
11+
'--mcmc_process') set -- "$@" '-m' ;;
12+
*) set -- "$@" "$arg" ;;
13+
esac
14+
done
15+
16+
# Parse short options
17+
OPTIND=1
18+
while getopts "hpcim" opt
19+
do
20+
case "$opt" in
21+
'h')
22+
echo "Please input a flag: --help, --pcf, --covmat, --inference or --mcmc_process ";
23+
exit 0
24+
;;
25+
'p')
26+
echo "Running cosmo_val.py to calculate 2 point correlation functions";
27+
python notebooks/cosmo_val/cosmo_val.py
28+
;;
29+
'c')
30+
read -p 'ROOT: ' root;
31+
read -p 'NZ FILE:' nz_file;
32+
read -p 'PATH COSMOCOV: ' cosmocov;
33+
echo "Calculating covariance matrices with CosmoCov";
34+
python scripts/cosmocov_process.py $root $nz_file $cosmocov
35+
;;
36+
'i')
37+
read -p 'ROOT: ' root;
38+
read -p 'XI_PLUS/XI_MINUS FITS FILE FOLDER: ' xi_folder;
39+
read -p 'NZ FILE:' nz_file;
40+
read -p 'RHO_STATS FILE FOLDER: ' rho_stats_folder;
41+
read -p 'COVMAT TXT FILE:' covmat;
42+
read -p 'OUTPUT MCMC CHAIN FOLDER: ' data;
43+
44+
out_file="data/${root}/cosmosis_${root}.fits";
45+
46+
#LG: add check if xi_plus/xi_minus fits file exists
47+
python scripts/cosmosis_fitting.py $root $xi_folder $covmat $nz_file $rho_stats_folder $out_file;
48+
49+
sed -i "/^\[DEFAULT\]/a\FITS_FILE = ${out_file}" cosmosis_config/cosmosis_pipeline_${root}.ini;
50+
sed -i "/^\[DEFAULT\]/a\SAMPLES_FILE = ${rho_stats_folder}/samples_${root}.npy" cosmosis_config/cosmosis_pipeline_${root}.ini
51+
sed -i "/^\[output\]/a\filename = ${data}/samples_${root}.txt" cosmosis_config/cosmosis_pipeline_${root}.ini;
52+
sed -i "/^\[pipeline\]/a\values = cosmosis_config/values_${root}.ini" cosmosis_config/cosmosis_pipeline_${root}.ini;
53+
sed -i "/^\[pipeline\]/a\priors = cosmosis_config/priors_${root}.ini" cosmosis_config/cosmosis_pipeline_${root}.ini;
54+
55+
echo "Prepared CosmoSIS configuration file in cosmosis_config/cosmosis_pipeline_${root}.ini";
56+
echo "You can now run the inference with the command: cosmosis cosmosis_config/cosmosis_pipeline_${root}.ini"
57+
;;
58+
'm')
59+
# LG: also convert this into a script to directly output contour plots
60+
echo "Run the cosmo_inference/notebooks/MCMC.ipynb notebook to analyse your chains"
61+
;;
62+
'?')
63+
print_usage >&2;
64+
exit 1
65+
;;
66+
esac
67+
done
68+
shift $(expr $OPTIND - 1) # remove options from positional parameters

0 commit comments

Comments
 (0)