Skip to content
Merged

Deploy #1150

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
ae36032
Update gary.yaml
D2SD May 12, 2025
75110cc
add honolulu (dev)
D2SD May 12, 2025
32820d1
migrate commonIndustryLookup and highestWageLookup endpoints to point…
alebjanes May 16, 2025
876af3e
add bls ids custom attribute
alebjanes May 22, 2025
100d22b
Merge pull request #1142 from DataUSA/university-lookups-migration
alebjanes May 22, 2025
ce9bacd
migrate neighbors endpoint to point to tesseract for Occupations
alebjanes May 23, 2025
3f520e9
Merge pull request #1143 from DataUSA/occupations-neighbors
alebjanes May 23, 2025
5ce45bc
add custom pums id to geo custom attributes
alebjanes May 27, 2025
c992e6b
Merge pull request #1144 from DataUSA/geo-custom-attribute
alebjanes May 27, 2025
04b9967
remove cart, vizbuilder and maps
alebjanes Jun 5, 2025
e4b10f1
Merge pull request #1145 from DataUSA/remove-vizbuilder
alebjanes Jun 5, 2025
18ae26b
remove routes (visualize, cart, map and coronavirus)
alebjanes Jun 5, 2025
3c8e4ca
fix footer flex spacing
alebjanes Jun 5, 2025
6714705
Merge pull request #1146 from DataUSA/remove-vizbuilder
alebjanes Jun 5, 2025
55e0f78
add new custom attributes for geo profile
alebjanes Jun 5, 2025
9e5606f
Merge pull request #1147 from DataUSA/geo-customAttributes
alebjanes Jun 5, 2025
fe48cab
fix includeIncome custom attribute
alebjanes Jun 5, 2025
6e1905e
add geo custom attributes
alebjanes Jun 9, 2025
5220dbf
remove old covid scripts
alebjanes Jun 9, 2025
d5731df
migrate /cache/sctg to point to tesseract
alebjanes Jun 9, 2025
2234d0c
migrate /cache/opeid to point to tesseract
alebjanes Jun 9, 2025
a2108ba
migrate /cache/urls to point to tesseract
alebjanes Jun 9, 2025
929eed9
migrate /cache/pops to tesseract
alebjanes Jun 10, 2025
e46a778
migrate cache/parents to point to tesseract
alebjanes Jun 11, 2025
eee5211
migrate cache/datasets to point to tesseract
alebjanes Jun 11, 2025
1727b34
migrate customAttributes and crosswalks to point to tesseract
alebjanes Jun 12, 2025
851aeb4
modify parents for CIPs to handle different datatype of ID
alebjanes Jun 12, 2025
8a157e2
migrate geomap of home to tesseract
alebjanes Jun 12, 2025
0580cca
add measures to cache and replace /api/cubes with /api/measures
alebjanes Jun 12, 2025
6f1575a
remove console.log
alebjanes Jun 12, 2025
e0a7b0d
remove logiclayer
alebjanes Jun 12, 2025
2c49413
removes logiclayer package
alebjanes Jun 12, 2025
ad3a908
add two custom attributes to geo profile
alebjanes Jun 13, 2025
1625242
add data for home geomap to cache + slugs from db
alebjanes Jun 13, 2025
746fe52
fix specialTessCut
alebjanes Jun 13, 2025
170da17
add cd pipeline to vm
D2SD Jun 18, 2025
f721317
fix syntax
D2SD Jun 18, 2025
32c3f60
fix cloudbuild reference
D2SD Jun 18, 2025
2835b61
fix env var reference
D2SD Jun 18, 2025
c9410db
fix env var reference
D2SD Jun 18, 2025
147cd81
remove host
D2SD Jun 18, 2025
a1ab4a1
update service port
D2SD Jun 18, 2025
82cb5e7
disable gke dev pipeline
D2SD Jun 18, 2025
3f3fa2b
add replicas to compose
D2SD Jun 18, 2025
71a355e
increase replicas compose
D2SD Jun 18, 2025
7c5acf0
add google analytics to compose
D2SD Jun 19, 2025
887986e
Merge pull request #1148 from DataUSA/vm-cd-pipeline
D2SD Jun 19, 2025
e83787f
Merge pull request #1149 from DataUSA/migrate-cache
alebjanes Jul 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions .github/workflows/google-gce-dev.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
name: "Deploy to GCE development"

on:
push:
branches: [ "develop" ]

env:
GCP_PROJECT_ID: ${{ vars.GCP_PROJECT_ID }}
GCP_ARTIFACT_REGISTRY_NAME: ${{ vars.GCP_ARTIFACT_REGISTRY_NAME }}
GCP_ARTIFACT_REGISTRY_LOCATION: ${{ vars.GCP_ARTIFACT_REGISTRY_LOCATION }}
GCP_IMAGE_NAME: ${{ vars.GCP_IMAGE_NAME }}
GCP_VM_IP: ${{ vars.GCP_VM_IP }}
GCP_VM_USER: ${{ vars.GCP_VM_USER }}

CANON_API: ${{ vars.CANON_API }}
CANON_CMS_CUBES: ${{ vars.CANON_CMS_CUBES }}
CANON_CMS_ENABLE: ${{ vars.CANON_CMS_ENABLE }}
CANON_CMS_FORCE_HTTPS: ${{ vars.CANON_CMS_FORCE_HTTPS }}
CANON_CMS_GENERATOR_TIMEOUT: ${{ vars.CANON_CMS_GENERATOR_TIMEOUT }}
CANON_CMS_LOGGING: ${{ vars.CANON_CMS_LOGGING }}
CANON_CMS_MINIMUM_ROLE: ${{ vars.CANON_CMS_MINIMUM_ROLE }}
CANON_CMS_REQUESTS_PER_SECOND: ${{ vars.CANON_CMS_REQUESTS_PER_SECOND }}
CANON_CONST_CART: ${{ vars.CANON_CONST_CART }}
CANON_CONST_CUBE: ${{ vars.CANON_CONST_CUBE }}
CANON_CONST_TESSERACT: ${{ vars.CANON_CONST_TESSERACT }}
CANON_DB_NAME: ${{ vars.CANON_DB_NAME }}
CANON_DB_USER: ${{ vars.CANON_DB_USER }}
CANON_GEOSERVICE_API: ${{ vars.CANON_GEOSERVICE_API }}
CANON_GOOGLE_ANALYTICS: ${{ vars.CANON_GOOGLE_ANALYTICS }}
CANON_LANGUAGES: ${{ vars.CANON_LANGUAGES }}
CANON_LANGUAGE_DEFAULT: ${{ vars.CANON_LANGUAGE_DEFAULT }}
CANON_LOGICLAYER_CUBE: ${{ vars.CANON_LOGICLAYER_CUBE }}
CANON_LOGICLAYER_SLUGS: ${{ vars.CANON_LOGICLAYER_SLUGS }}
CANON_LOGINS: ${{ vars.CANON_LOGINS }}
GA_KEYFILE: ${{ vars.GA_KEYFILE }}


jobs:
build:
environment: development-vm
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Authenticate with Google Cloud
uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}

- name: Build Docker Image
run: |-
gcloud builds submit \
--quiet \
--timeout=30m \
--config=cloudbuild.yml \
--substitutions=_GCP_PROJECT_ID=${{ env.GCP_PROJECT_ID }},_GCP_ARTIFACT_REGISTRY_NAME=${{ env.GCP_ARTIFACT_REGISTRY_NAME }},_GCP_ARTIFACT_REGISTRY_LOCATION=${{ env.GCP_ARTIFACT_REGISTRY_LOCATION }},_GCP_IMAGE_NAME=${{ env.GCP_IMAGE_NAME }},_GCP_IMAGE_TAG=${{ github.sha }},_GCP_IMAGE_ENVIRONMENT=${{ env.GCP_IMAGE_NAME }}

deploy:
needs: build
environment: development-vm
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Authenticate with Google Cloud
uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}

- name: Deploy to Compute Engine
run: |
SSH_DIR=~/.ssh
mkdir -p $SSH_DIR
echo "${{ secrets.GCP_SSH_PRIVATE_KEY }}" > $SSH_DIR/id_rsa
chmod 600 $SSH_DIR/id_rsa
ssh-keyscan -H ${{ env.GCP_VM_IP }} >> $SSH_DIR/known_hosts

# Define remote path
REMOTE_PATH="/home/${{ env.GCP_VM_USER }}/${{ env.GCP_ARTIFACT_REGISTRY_NAME }}-${{ env.GCP_IMAGE_NAME }}"

# Create remote directory and transfer files in one SSH session
echo "Creating remote path"
ssh -i $SSH_DIR/id_rsa ${{ env.GCP_VM_USER }}@${{ env.GCP_VM_IP }} "mkdir -p $REMOTE_PATH"
echo "Sending files"
scp -i $SSH_DIR/id_rsa compose.yaml deploy_to_vm.sh ${{ env.GCP_VM_USER }}@${{ env.GCP_VM_IP }}:$REMOTE_PATH

ssh -i $SSH_DIR/id_rsa ${{ env.GCP_VM_USER }}@${{ env.GCP_VM_IP }} 'bash -s' << 'ENDSSH'

# Go to working directory
echo "Going to working directory"
cd "/home/${{ env.GCP_VM_USER }}/${{ env.GCP_ARTIFACT_REGISTRY_NAME }}-${{ env.GCP_IMAGE_NAME }}"

# Create .env.gcp file
echo "Creating .env.gcp file"
{
echo "GCP_IMAGE_TAG=${{ github.sha }}"
echo "GCP_PROJECT_ID=${{ env.GCP_PROJECT_ID }}"
echo "GCP_IMAGE_NAME=${{ env.GCP_IMAGE_NAME }}"
echo "GCP_ARTIFACT_REGISTRY_NAME=${{ env.GCP_ARTIFACT_REGISTRY_NAME }}"
echo "GCP_ARTIFACT_REGISTRY_LOCATION=${{ env.GCP_ARTIFACT_REGISTRY_LOCATION }}"
echo "GCP_VM_USER=${{ env.GCP_VM_USER }}"
echo "CANON_DB_HOST=${{ secrets.CANON_DB_HOST}}"
echo "CANON_DB_PW=${{ secrets.CANON_DB_PW}}"
echo "CANON_HOTJAR=${{ secrets.CANON_HOTJAR}}"
echo "CANON_API=${{ env.CANON_API }}"
echo "CANON_CMS_CUBES=${{ env.CANON_CMS_CUBES }}"
echo "CANON_CMS_ENABLE=${{ env.CANON_CMS_ENABLE }}"
echo "CANON_CMS_FORCE_HTTPS=${{ env.CANON_CMS_FORCE_HTTPS }}"
echo "CANON_CMS_GENERATOR_TIMEOUT=${{ env.CANON_CMS_GENERATOR_TIMEOUT }}"
echo "CANON_CMS_LOGGING=${{ env.CANON_CMS_LOGGING }}"
echo "CANON_CMS_MINIMUM_ROLE=${{ env.CANON_CMS_MINIMUM_ROLE }}"
echo "CANON_CMS_REQUESTS_PER_SECOND=${{ env.CANON_CMS_REQUESTS_PER_SECOND }}"
echo "CANON_CONST_CART=${{ env.CANON_CONST_CART }}"
echo "CANON_CONST_CUBE=${{ env.CANON_CONST_CUBE }}"
echo "CANON_CONST_TESSERACT=${{ env.CANON_CONST_TESSERACT }}"
echo "CANON_DB_NAME=${{ env.CANON_DB_NAME }}"
echo "CANON_DB_USER=${{ env.CANON_DB_USER }}"
echo "CANON_GEOSERVICE_API=${{ env.CANON_GEOSERVICE_API }}"
echo "CANON_GOOGLE_ANALYTICS=${{ env.CANON_GOOGLE_ANALYTICS }}"
echo "CANON_LANGUAGES=${{ env.CANON_LANGUAGES }}"
echo "CANON_LANGUAGE_DEFAULT=${{ env.CANON_LANGUAGE_DEFAULT }}"
echo "CANON_LOGICLAYER_CUBE=${{ env.CANON_LOGICLAYER_CUBE }}"
echo "CANON_LOGICLAYER_SLUGS=${{ env.CANON_LOGICLAYER_SLUGS }}"
echo "CANON_LOGINS=${{ env.CANON_LOGINS }}"
echo "GA_KEYFILE=${{ env.GA_KEYFILE }}"
} > .env.gcp

echo "Adding Google Analytics credentials to ./google directory"
mkdir -p ./google

cat << EOF > ./google/googleAnalyticsKey.json
${{ secrets.GA_KEYFILE }}
EOF

bash ./deploy_to_vm.sh

ENDSSH
2 changes: 1 addition & 1 deletion .github/workflows/google-registry-gke-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ name: "[GCP][DEV] Build NextJS to Registry and Deploy via Helm"

on:
push:
branches: [ "develop" ]
branches: [ "disabled" ]
paths:
- .github/workflows/google-registry-gke-dev.yaml
- helm/templates
Expand Down
Loading