From d90c6ef3fba26a29b03de099e176e729c0b06446 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Sun, 30 Mar 2025 19:58:08 -0400 Subject: [PATCH 1/8] Create production_liasion_guidelines.md --- _resources/production_liasion_guidelines.md | 117 ++++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100644 _resources/production_liasion_guidelines.md diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md new file mode 100644 index 0000000..f3bde99 --- /dev/null +++ b/_resources/production_liasion_guidelines.md @@ -0,0 +1,117 @@ +--- +title: Production Liasion Guidelines +description: Production Liasion Guidelines +name: production liasion guidelines +layout: default +--- + +{% include layouts/title.md %} + +# Getting User Accounts + +The following steps assume that you have an account on one or more of the access points (APs) at JeffersonLab, Open Science Grid, or Brookhaven National Lab. The instructions to get accounts on each of the access points can be found in the links below: +1. [Jefferson Lab Account Access](https://misportal.jlab.org/jlabAccess/) +2. [Open Science Grid Account Access](https://portal.osg-htc.org/application) +3. [Brookhaven Lab Account Access](https://www.sdcc.bnl.gov/information/getting-started/new-user-account) + +Follow the instructions in the respective links to get your account and then log in to the access point. + +### Jefferson Lab +```bash +ssh @scilogin.jlab.org -Y +# Use your 2-factor Safenet password +``` +```bash +ssh @osg-eic -Y +# Use your 2-factor Safenet password +``` + +If you are having trouble with account access, call JeffersonLab helpdesk at +1(757)2697155. + +### Open Science Grid +```bash +ssh @ap23.uc.osg-htc.org -Y +# If you already set up your SSH keys on ci-connect, you will only need to enter your 2-factor password +``` + +If you are having trouble with account access, email support@osg-htc.org. + +### Brookhaven National Lab +```bash +ssh @ssh.sdcc.bnl.gov -Y +``` + +If you are having trouble with account access, email RT-RACF-UserAccounts@bnl.gov. + +--- + +# Getting a Certificate from CILogon + +You will need to obtain your user certificate using the CILogon web UI. Follow the steps below to get a user certificate: + +1. Open the [CILogon page](https://cilogon.org) in your browser. +2. Search for your institution or scroll through the list and select it. + + **Warning:** + Do not use Google, GitHub, or ORCID as providers since they are not widely supported in the OSG. If your institution is not on the list, contact your institution's IT support to check if they can support CILogon. + +3. Click the "Log On" button and enter your institutional credentials if prompted. +4. After successfully entering your credentials, click on the **Create Password-Protected Certificate** link. +5. Enter a password that is at least 12 characters long and then click on the **Get New Certificate** button. +6. Click **Download Your Certificate** to download your certificate in `.p12` format. The certificate will be protected by the password you created. + +--- + +# Generating User Keys + +1. Open `eic-shell`: + ```bash + curl -L https://github.com/eic/eic-shell/raw/main/install.sh | bash + ./eic-shell + ``` +2. Generate the keys. You will be prompted to enter the import password you used when generating the certificate. Then, exit out of the container: + ```bash + openssl pkcs12 -in usercred.p12 -out usercert.pem -clcerts -nokeys -legacy + openssl pkcs12 -in usercred.p12 -out userkey.pem -nocerts -nodes -legacy + exit + ``` +3. Move certificates and keys to the `.globus` folder: + ```bash + mkdir ~/.globus + mv user* ~/.globus + chmod 600 ~/.globus/usercert.pem + chmod 600 ~/.globus/userkey.pem + ``` + +--- + +# Installing voms-client + +You will need the `voms-client` on a Linux machine. If you are on your local Linux machine, then do: + +```bash +sudo apt-get install voms-clients-java +``` +All the access points should already have the `voms-client` installed. So, you can skip this step if you are already on one. + +--- + +# Running voms-proxy-init + +Run the following command to generate a voms proxy certificate with a validity of at least 2 months (1460 hours): + +```bash +voms-proxy-init --hours 1460 +``` + +This will create an X.509 proxy certificate in your `/tmp` directory. You can rename it as `x509_user_proxy` and use it for production or uploading files to the Jefferson Lab storage system. + +--- + +# Send Information + +Run the following command and email the output to `panta@jlab.org`: + +```bash +voms-proxy-info | grep "issuer" | awk -F":" '{print $2}' +``` From 6eb8e17c43aa53130e0d55343cf6ac4f6389c888 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Sun, 30 Mar 2025 20:17:43 -0400 Subject: [PATCH 2/8] Reorganize the Rucio write access information under one section --- _resources/production_liasion_guidelines.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index f3bde99..df0ee4e 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -45,7 +45,8 @@ If you are having trouble with account access, email RT-RACF-UserAccounts@bnl.go --- -# Getting a Certificate from CILogon +# Getting Write Access to Jefferson Lab Rucio Storage Endpoint (RSE) +### Getting a Certificate from CILogon You will need to obtain your user certificate using the CILogon web UI. Follow the steps below to get a user certificate: @@ -62,7 +63,7 @@ You will need to obtain your user certificate using the CILogon web UI. Follow t --- -# Generating User Keys +### Generating User Keys 1. Open `eic-shell`: ```bash @@ -85,7 +86,7 @@ You will need to obtain your user certificate using the CILogon web UI. Follow t --- -# Installing voms-client +### Installing voms-client You will need the `voms-client` on a Linux machine. If you are on your local Linux machine, then do: @@ -96,7 +97,7 @@ All the access points should already have the `voms-client` installed. So, you c --- -# Running voms-proxy-init +### Running voms-proxy-init Run the following command to generate a voms proxy certificate with a validity of at least 2 months (1460 hours): @@ -108,10 +109,11 @@ This will create an X.509 proxy certificate in your `/tmp` directory. You can re --- -# Send Information +### Send Information Run the following command and email the output to `panta@jlab.org`: ```bash voms-proxy-info | grep "issuer" | awk -F":" '{print $2}' ``` +You only need to do this once. From ba15869adf6461969add69faa6c0695cbdf0ceab Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Sun, 30 Mar 2025 22:13:09 -0400 Subject: [PATCH 3/8] Instruction to add a new dataset in Production --- _resources/production_liasion_guidelines.md | 31 +++++++++++++++++---- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index df0ee4e..2901f51 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -7,6 +7,7 @@ layout: default {% include layouts/title.md %} +--- # Getting User Accounts The following steps assume that you have an account on one or more of the access points (APs) at JeffersonLab, Open Science Grid, or Brookhaven National Lab. The instructions to get accounts on each of the access points can be found in the links below: @@ -43,8 +44,8 @@ ssh @ssh.sdcc.bnl.gov -Y If you are having trouble with account access, email RT-RACF-UserAccounts@bnl.gov. ---- +--- # Getting Write Access to Jefferson Lab Rucio Storage Endpoint (RSE) ### Getting a Certificate from CILogon @@ -61,7 +62,7 @@ You will need to obtain your user certificate using the CILogon web UI. Follow t 5. Enter a password that is at least 12 characters long and then click on the **Get New Certificate** button. 6. Click **Download Your Certificate** to download your certificate in `.p12` format. The certificate will be protected by the password you created. ---- + ### Generating User Keys @@ -84,7 +85,7 @@ You will need to obtain your user certificate using the CILogon web UI. Follow t chmod 600 ~/.globus/userkey.pem ``` ---- + ### Installing voms-client @@ -95,7 +96,7 @@ sudo apt-get install voms-clients-java ``` All the access points should already have the `voms-client` installed. So, you can skip this step if you are already on one. ---- + ### Running voms-proxy-init @@ -107,7 +108,7 @@ voms-proxy-init --hours 1460 This will create an X.509 proxy certificate in your `/tmp` directory. You can rename it as `x509_user_proxy` and use it for production or uploading files to the Jefferson Lab storage system. ---- + ### Send Information @@ -117,3 +118,23 @@ Run the following command and email the output to `panta@jlab.org`: voms-proxy-info | grep "issuer" | awk -F":" '{print $2}' ``` You only need to do this once. + +--- +# Integrating a New Dataset in Production + +Run eic-shell and set the rucio variables +```bash +./eic-shell +export RUCIO_CONFIG=/opt/campaigns/hepmc3/scripts/rucio.cfg +export X509_USER_PROXY=x509_user_proxy +``` + +Then you can transfer the files from the source location to desired directory structure on JLAB RSE following the [input pre-processing guidelines](https://eic.github.io/epic-prod/documentation/input_preprocessing.html). An example is shown here: +``` +python /opt/campaigns/hepmc3/scripts/register_to_rucio.py \ +-f "/work/eic3/users/sjdkay/Mar2025_Campaign_Input/Afterburner_Output/kaonLambda/10on130/DEMPgen-v1.2.3_K+LambdaDEMP_10on130_q2_20_35.hepmc3.tree.root" \ +-d "/EVGEN/EXCLUSIVE/DEMP/DEMPgen-1.2.3/10x130/q2_20_35/K+Lambda/DEMPgen-1.2.3_10x130_K+Lambda_q2_20_35_hiAcc.hepmc3.tree.root" \ +-s epic -r EIC-XRD +``` + +Once they are uploaded, create a PR similar to [this](https://eicweb.phy.anl.gov/EIC/campaigns/datasets/-/merge_requests/89/diffs) on the [datasets repository](https://github.com/eic/simulation_campaign_datasets/) with csv file catalogues for the respective datasets and integration line in the config.yml file. From d396086d3db48550a488d4de5efe9c3d2638acc0 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Sun, 30 Mar 2025 22:27:56 -0400 Subject: [PATCH 4/8] Update link to example PR for dataset integration --- _resources/production_liasion_guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index 2901f51..131720c 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -137,4 +137,4 @@ python /opt/campaigns/hepmc3/scripts/register_to_rucio.py \ -s epic -r EIC-XRD ``` -Once they are uploaded, create a PR similar to [this](https://eicweb.phy.anl.gov/EIC/campaigns/datasets/-/merge_requests/89/diffs) on the [datasets repository](https://github.com/eic/simulation_campaign_datasets/) with csv file catalogues for the respective datasets and integration line in the config.yml file. +Once they are uploaded, create a PR similar to [this](https://eicweb.phy.anl.gov/EIC/campaigns/datasets/-/merge_requests/94/diffs) on the [datasets repository](https://github.com/eic/simulation_campaign_datasets/) with csv file catalogues for the respective datasets and integration line in the config.yml file. From 9341d73fcd467c67d5b2e98cf16bb107e37d9006 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Wed, 30 Apr 2025 14:49:01 -0400 Subject: [PATCH 5/8] Change link for opensciencegrid signup --- _resources/production_liasion_guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index 131720c..4000272 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -12,7 +12,7 @@ layout: default The following steps assume that you have an account on one or more of the access points (APs) at JeffersonLab, Open Science Grid, or Brookhaven National Lab. The instructions to get accounts on each of the access points can be found in the links below: 1. [Jefferson Lab Account Access](https://misportal.jlab.org/jlabAccess/) -2. [Open Science Grid Account Access](https://portal.osg-htc.org/application) +2. [Open Science Grid Account Access](https://https://www.ci-connect.net/signup) 3. [Brookhaven Lab Account Access](https://www.sdcc.bnl.gov/information/getting-started/new-user-account) Follow the instructions in the respective links to get your account and then log in to the access point. From f8800b1651f159649474ec6aa5602bbced8bea77 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Wed, 30 Apr 2025 15:10:15 -0400 Subject: [PATCH 6/8] Update instructions to get x509 proxy for Rucio write access --- _resources/production_liasion_guidelines.md | 54 +++++---------------- 1 file changed, 12 insertions(+), 42 deletions(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index 4000272..b87b148 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -47,45 +47,19 @@ If you are having trouble with account access, email RT-RACF-UserAccounts@bnl.go --- # Getting Write Access to Jefferson Lab Rucio Storage Endpoint (RSE) -### Getting a Certificate from CILogon +### Getting a Certificate and Key -You will need to obtain your user certificate using the CILogon web UI. Follow the steps below to get a user certificate: +1. If you have a jlab account, file a ticket with [Service Now](https://jlab.servicenowservices.com/) asking for a certificate to be able access ePIC Rucio in the context of OSG job submission. If you don't have a jlab account, contact the [production WG](https://eic.github.io/epic-prod/about/contact.html) to file a ticket on your behalf. -1. Open the [CILogon page](https://cilogon.org) in your browser. -2. Search for your institution or scroll through the list and select it. - - **Warning:** - Do not use Google, GitHub, or ORCID as providers since they are not widely supported in the OSG. If your institution is not on the list, contact your institution's IT support to check if they can support CILogon. - -3. Click the "Log On" button and enter your institutional credentials if prompted. -4. After successfully entering your credentials, click on the **Create Password-Protected Certificate** link. -5. Enter a password that is at least 12 characters long and then click on the **Get New Certificate** button. -6. Click **Download Your Certificate** to download your certificate in `.p12` format. The certificate will be protected by the password you created. - - - -### Generating User Keys - -1. Open `eic-shell`: - ```bash - curl -L https://github.com/eic/eic-shell/raw/main/install.sh | bash - ./eic-shell - ``` -2. Generate the keys. You will be prompted to enter the import password you used when generating the certificate. Then, exit out of the container: +2. You will receive an email from support@cert-manager.com with the instruction to download your .cer file. +​ +3. Once you have received this .cer file, transfer this to the access point you will be using. Please request the production WG to provide you with the key corresponding to your .cer file. Your key will be placed in the access point according to your request (BNL, JLAB or OSG). +​ +4. Move certificates and keys to a desired location and change the permissions. ```bash - openssl pkcs12 -in usercred.p12 -out usercert.pem -clcerts -nokeys -legacy - openssl pkcs12 -in usercred.p12 -out userkey.pem -nocerts -nodes -legacy - exit + chmod 600 usercert.cer + chmod 600 userkey.key ``` -3. Move certificates and keys to the `.globus` folder: - ```bash - mkdir ~/.globus - mv user* ~/.globus - chmod 600 ~/.globus/usercert.pem - chmod 600 ~/.globus/userkey.pem - ``` - - ### Installing voms-client @@ -96,26 +70,22 @@ sudo apt-get install voms-clients-java ``` All the access points should already have the `voms-client` installed. So, you can skip this step if you are already on one. - - ### Running voms-proxy-init Run the following command to generate a voms proxy certificate with a validity of at least 2 months (1460 hours): ```bash -voms-proxy-init --hours 1460 +voms-proxy-init -cert=usercert.cer -key=userkey.key -out=x509_user_proxy -hours=1460 ``` -This will create an X.509 proxy certificate in your `/tmp` directory. You can rename it as `x509_user_proxy` and use it for production or uploading files to the Jefferson Lab storage system. - - +This will create an proxy certificate in your current directory. ### Send Information Run the following command and email the output to `panta@jlab.org`: ```bash -voms-proxy-info | grep "issuer" | awk -F":" '{print $2}' +voms-proxy-info -file=x509_user_proxy | grep "issuer" | awk -F":" '{print $2}' ``` You only need to do this once. From 3c3d97d7691db7dfd04a391e9b370e858a8d5d67 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Thu, 1 May 2025 09:23:26 -0400 Subject: [PATCH 7/8] Update process of getting accounts and access to login nodes at different sites --- _resources/production_liasion_guidelines.md | 23 +++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index b87b148..4ae9006 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -10,18 +10,28 @@ layout: default --- # Getting User Accounts -The following steps assume that you have an account on one or more of the access points (APs) at JeffersonLab, Open Science Grid, or Brookhaven National Lab. The instructions to get accounts on each of the access points can be found in the links below: +You should get a computing user account on JeffersonLab, Open Science Grid, or Brookhaven National Lab. The instructions to get accounts on each of the sites can be found in the links below: 1. [Jefferson Lab Account Access](https://misportal.jlab.org/jlabAccess/) -2. [Open Science Grid Account Access](https://https://www.ci-connect.net/signup) +2. [Open Science Grid Account Access](https://www.ci-connect.net/signup) 3. [Brookhaven Lab Account Access](https://www.sdcc.bnl.gov/information/getting-started/new-user-account) -Follow the instructions in the respective links to get your account and then log in to the access point. +Obtaining access to multiple sites is good for redundancy. + + +# Getting Approved for OSG Access Points (APs) ### Jefferson Lab ```bash ssh @scilogin.jlab.org -Y # Use your 2-factor Safenet password ``` + +You have to setup the 2-factor Safenet password by calling the helpdesk at +1(757)2697155 after your account creation is done. You will receive the relevant links on your [JLab email](https://webmail.jlab.org/) which can be accessed by your regular JLab password. + +Once you have access to the JLab login node, file a ticket with [JLab Service Now](https://jlab.servicenowservices.com) to gain access to osg-eic access point. + +Mention in the ticket that you will be submitting simulation jobs to OSG on behalf of the ePIC experiment. Once approved, you will be able to login to osg-eic after logging into the JLab login node. + ```bash ssh @osg-eic -Y # Use your 2-factor Safenet password @@ -42,8 +52,13 @@ If you are having trouble with account access, email support@osg-htc.org. ssh @ssh.sdcc.bnl.gov -Y ``` -If you are having trouble with account access, email RT-RACF-UserAccounts@bnl.gov. +Once you have access to the sdcc login node, file a ticket with RT-RACF-UserAccounts@bnl.gov to get access to the osgsub01 node and mention that that you will be using it to submit jobs on behalf of the ePIC collaboration. Once approved, you can log onto the access point after logging onto the sdcc login node. + +```bash +ssh @osgsub01 -Y +``` +Report any difficulties to the above-mentioned email for filing tickets. --- # Getting Write Access to Jefferson Lab Rucio Storage Endpoint (RSE) From a2761d8279f1f92b3ad59dbeb8d92f1839dc1547 Mon Sep 17 00:00:00 2001 From: Sakib Rahman Date: Thu, 1 May 2025 09:29:32 -0400 Subject: [PATCH 8/8] Update production_liasion_guidelines.md --- _resources/production_liasion_guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_resources/production_liasion_guidelines.md b/_resources/production_liasion_guidelines.md index 4ae9006..308f696 100644 --- a/_resources/production_liasion_guidelines.md +++ b/_resources/production_liasion_guidelines.md @@ -114,7 +114,7 @@ export RUCIO_CONFIG=/opt/campaigns/hepmc3/scripts/rucio.cfg export X509_USER_PROXY=x509_user_proxy ``` -Then you can transfer the files from the source location to desired directory structure on JLAB RSE following the [input pre-processing guidelines](https://eic.github.io/epic-prod/documentation/input_preprocessing.html). An example is shown here: +Then you can transfer the files from the source location to desired directory structure on JLAB RSE following the [input pre-processing guidelines](https://eic.github.io/epic-prod/documentation/input_preprocessing.html). Make sure that the dataset can be traced to a version controlled github repo before this transfer happens because you will need the version tag for the directory structure and nomenclature. An example of the transfer is shown here: ``` python /opt/campaigns/hepmc3/scripts/register_to_rucio.py \ -f "/work/eic3/users/sjdkay/Mar2025_Campaign_Input/Afterburner_Output/kaonLambda/10on130/DEMPgen-v1.2.3_K+LambdaDEMP_10on130_q2_20_35.hepmc3.tree.root" \