Skip to content

Commit c3b9920

Browse files
ThomasKranitsasThomasKranitsas
authored
Merge pull request #61 from topcoder-platform/develop
Prod release
2 parents bdae8ce + f5b5a21 commit c3b9920

22 files changed

+10807
-3047
lines changed

‎README.md

Lines changed: 65 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,33 @@
11
# Topcoder Challenge API
22

3-
## Dependencies
3+
This microservice provides access and interaction with all sorts of Challenge data.
44

5-
- nodejs https://nodejs.org/en/ (v10)
6-
- DynamoDB
7-
- AWS S3
8-
- Elasticsearch v6
9-
- Docker, Docker Compose
5+
### Deployment status
6+
Dev: [![CircleCI](https://circleci.com/gh/topcoder-platform/challenge-api/tree/develop.svg?style=svg)](https://circleci.com/gh/topcoder-platform/challenge-api/tree/develop) Prod: [![CircleCI](https://circleci.com/gh/topcoder-platform/challenge-api/tree/master.svg?style=svg)](https://circleci.com/gh/topcoder-platform/challenge-api/tree/master)
7+
8+
## Swagger definition
9+
10+
- [Swagger](https://api.topcoder.com/v5/challenges/docs/)
11+
12+
## Intended use
13+
14+
- Production API
15+
16+
## Related repos
17+
18+
- [Resources API](https://github.com/topcoder-platform/resources-api)
19+
- [ES Processor](https://github.com/topcoder-platform/challenge-processor-es) - Updates data in ElasticSearch
20+
- [Legacy Processor](https://github.com/topcoder-platform/legacy-challenge-processor) - Moves data from DynamoDB back to Informix
21+
- [Legacy Migration Script](https://github.com/topcoder-platform/legacy-challenge-migration-script) - Moves data from Informix to DynamoDB
22+
- [Frontend App](https://github.com/topcoder-platform/challenge-engine-ui)
23+
24+
## Prerequisites
25+
- [NodeJS](https://nodejs.org/en/) (v10)
26+
- [DynamoDB](https://aws.amazon.com/dynamodb/)
27+
- [AWS S3](https://aws.amazon.com/s3/)
28+
- [Elasticsearch v6](https://www.elastic.co/)
29+
- [Docker](https://www.docker.com/)
30+
- [Docker Compose](https://docs.docker.com/compose/)
1031

1132
## Configuration
1233

@@ -41,51 +62,15 @@ The following parameters can be set in config files or in env variables:
4162
- RESOURCES_API_URL: TC resources API base URL
4263
- GROUPS_API_URL: TC groups API base URL
4364
- PROJECTS_API_URL: TC projects API base URL
65+
- TERMS_API_URL: TC Terms API Base URL
4466
- COPILOT_RESOURCE_ROLE_IDS: copilot resource role ids allowed to upload attachment
4567
- HEALTH_CHECK_TIMEOUT: health check timeout in milliseconds
4668
- SCOPES: the configurable M2M token scopes, refer `config/default.js` for more details
4769
- M2M_AUDIT_HANDLE: the audit name used when perform create/update operation using M2M token
4870

49-
Set the following environment variables so that the app can get TC M2M token (use 'set' insted of 'export' for Windows OS):
50-
51-
- export AUTH0_CLIENT_ID=8QovDh27SrDu1XSs68m21A1NBP8isvOt
52-
- export AUTH0_CLIENT_SECRET=3QVxxu20QnagdH-McWhVz0WfsQzA1F8taDdGDI4XphgpEYZPcMTF4lX3aeOIeCzh
53-
- export AUTH0_URL=https://topcoder-dev.auth0.com/oauth/token
54-
- export AUTH0_AUDIENCE=https://m2m.topcoder-dev.com/
55-
56-
Also properly configure AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, ATTACHMENT_S3_BUCKET, IS_LOCAL_DB config parameters.
57-
58-
Test configuration is at `config/test.js`. You don't need to change them.
59-
The following test parameters can be set in config file or in env variables:
60-
61-
- ADMIN_TOKEN: admin token
62-
- COPILOT_TOKEN: copilot token
63-
- USER_TOKEN: user token
64-
- EXPIRED_TOKEN: expired token
65-
- INVALID_TOKEN: invalid token
66-
- M2M_FULL_ACCESS_TOKEN: M2M full access token
67-
- M2M_READ_ACCESS_TOKEN: M2M read access token
68-
- M2M_UPDATE_ACCESS_TOKEN: M2M update (including 'delete') access token
69-
- S3_ENDPOINT: endpoint of AWS S3 API, for unit and e2e test only; default to `localhost:9000`
70-
71-
## AWS S3 Setup
72-
Go to https://console.aws.amazon.com/ and login. Choose S3 from Service folder and click `Create bucket`. Following the instruction to create S3 bucket.
73-
74-
## Local services setup
75-
In the `local` folder, run `docker-compose up`
76-
It starts Elasticsearch, DynamoDB and S3 compatible server.
71+
You can find sample `.env` files inside the `/docs` directory.
7772

78-
## Mock api
79-
For postman verification, please use the mock api under mock-api folder. It provides mock endpoint to fetch challenge resources and groups.
80-
You need to ensure DynamoDB configuration in `mock-api/config/default.js` is consistent with `config/default.js`
81-
Go to `mock-api` folder and run commands `npm i` and `npm start` to start the mock-api listening on port 4000
82-
83-
## Create Tables
84-
1. Make sure DynamoDB are running as per instructions above.
85-
2. Make sure you have configured all config parameters. Refer [Configuration](#configuration)
86-
3. Run `npm run create-tables` to create tables.
87-
88-
## Scripts
73+
## Available commands
8974
1. Drop/delete tables: `npm run drop-tables`
9075
2. Creating tables: `npm run create-tables`
9176
3. Seed/Insert data to tables: `npm run seed-tables`
@@ -99,6 +84,22 @@ Go to `mock-api` folder and run commands `npm i` and `npm start` to start the mo
9984

10085
## Local Deployment
10186

87+
### AWS S3 Setup
88+
Go to https://console.aws.amazon.com/ and login. Choose S3 from Service folder and click `Create bucket`. Following the instruction to create S3 bucket.
89+
90+
### Local services setup
91+
In the `local` folder, run `docker-compose up` to start Elasticsearch, DynamoDB and S3 compatible server.
92+
93+
### Create Tables
94+
1. Make sure DynamoDB are running as per instructions above.
95+
2. Make sure you have configured all config parameters. Refer [Configuration](#configuration)
96+
3. Run `npm run create-tables` to create tables.
97+
98+
### Mock API
99+
The provided mock API provides mock endpoint to fetch challenge resources and groups so you don't have to deploy the related services locally.
100+
You need to ensure DynamoDB configuration in `mock-api/config/default.js` is consistent with `config/default.js`
101+
Go to `mock-api` folder and run commands `npm i` and `npm start` to start the mock-api listening on port 4000
102+
102103
- Install dependencies `npm install`
103104
- Run lint `npm run lint`
104105
- Run lint fix `npm run lint:fix`
@@ -110,8 +111,27 @@ Go to `mock-api` folder and run commands `npm i` and `npm start` to start the mo
110111
- App is running at `http://localhost:3000`
111112
- Start mock-api, go to `mock-api` folder, run `npm i` and `npm start`, mock api is running at `http://localhost:4000`
112113

114+
## Production deployment
115+
116+
- TBD
117+
113118
## Running tests
114119

120+
### Configuration
121+
122+
Test configuration is at `config/test.js`. You don't need to change them.
123+
The following test parameters can be set in config file or in env variables:
124+
125+
- ADMIN_TOKEN: admin token
126+
- COPILOT_TOKEN: copilot token
127+
- USER_TOKEN: user token
128+
- EXPIRED_TOKEN: expired token
129+
- INVALID_TOKEN: invalid token
130+
- M2M_FULL_ACCESS_TOKEN: M2M full access token
131+
- M2M_READ_ACCESS_TOKEN: M2M read access token
132+
- M2M_UPDATE_ACCESS_TOKEN: M2M update (including 'delete') access token
133+
- S3_ENDPOINT: endpoint of AWS S3 API, for unit and e2e test only; default to `localhost:9000`
134+
115135
### Prepare
116136
- Start Local services.
117137
- Start Mock API.
@@ -159,4 +179,3 @@ Refer to the verification document `Verification.md`
159179

160180
- In the app-constants.js Topics field, the used topics are using a test topic,
161181
the suggested ones are commented out, because these topics are not created in TC dev Kafka yet.
162-

‎app.js

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,16 @@ const HttpStatus = require('http-status-codes')
1313
const logger = require('./src/common/logger')
1414
const interceptor = require('express-interceptor')
1515
const fileUpload = require('express-fileupload')
16+
const YAML = require('yamljs')
17+
const swaggerUi = require('swagger-ui-express')
18+
const challengeAPISwaggerDoc = YAML.load('./docs/swagger.yaml')
1619

1720
// setup express app
1821
const app = express()
1922

23+
// serve challenge V5 API swagger definition
24+
app.use('/v5/challenges/docs', swaggerUi.serve, swaggerUi.setup(challengeAPISwaggerDoc))
25+
2026
app.use(cors({
2127
exposedHeaders: [
2228
'X-Prev-Page',

‎config/default.js

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ module.exports = {
2222
KAFKA_ERROR_TOPIC: process.env.KAFKA_ERROR_TOPIC || 'common.error.reporting',
2323

2424
AMAZON: {
25-
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID || 'FAKE_ACCESS_KEY',
26-
AWS_SECRET_ACCESS_KEY: process.env.AWS_SECRET_ACCESS_KEY || 'FAKE_SECRET_ACCESS_KEY',
25+
// Uncomment for local deployment
26+
// AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID || 'FAKE_ACCESS_KEY',
27+
// AWS_SECRET_ACCESS_KEY: process.env.AWS_SECRET_ACCESS_KEY || 'FAKE_SECRET_ACCESS_KEY',
2728
AWS_REGION: process.env.AWS_REGION || 'ap-northeast-1',
2829
IS_LOCAL_DB: process.env.IS_LOCAL_DB ? process.env.IS_LOCAL_DB === 'true' : true,
2930
DYNAMODB_URL: process.env.DYNAMODB_URL || 'http://localhost:7777',
@@ -45,7 +46,8 @@ module.exports = {
4546
? Number(process.env.FILE_UPLOAD_SIZE_LIMIT) : 50 * 1024 * 1024, // 50M
4647
RESOURCES_API_URL: process.env.RESOURCES_API_URL || 'http://localhost:4000/v5/resources',
4748
GROUPS_API_URL: process.env.GROUPS_API_URL || 'http://localhost:4000/v5/groups',
48-
PROJECTS_API_URL: process.env.PROJECTS_API_URL || 'http://localhost:4000/v4/projects',
49+
PROJECTS_API_URL: process.env.PROJECTS_API_URL || 'http://localhost:4000/v5/projects',
50+
TERMS_API_URL: process.env.TERMS_API_URL || 'http://localhost:4000/v5/terms',
4951
// copilot resource role ids allowed to upload attachment
5052
COPILOT_RESOURCE_ROLE_IDS: process.env.COPILOT_RESOURCE_ROLE_IDS
5153
? process.env.COPILOT_RESOURCE_ROLE_IDS.split(',') : ['10ba038e-48da-487b-96e8-8d3b99b6d18b'],

‎config/test.js

Lines changed: 6 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/dev.env

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
AUTH0_CLIENT_ID=8QovDh27SrDu1XSs68m21A1NBP8isvOt
2+
AUTH0_CLIENT_SECRET=3QVxxu20QnagdH-McWhVz0WfsQzA1F8taDdGDI4XphgpEYZPcMTF4lX3aeOIeCzh
3+
AUTH0_URL=https://topcoder-dev.auth0.com/oauth/token
4+
AUTH0_AUDIENCE=https://m2m.topcoder-dev.com/

‎docs/prod.env

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
AUTH0_CLIENT_ID=
2+
AUTH0_CLIENT_SECRET=
3+
AUTH0_URL=
4+
AUTH0_AUDIENCE=
5+
AUTH_SECRET=
6+
BUSAPI_URL=
7+
AWS_ACCESS_KEY_ID=
8+
AWS_SECRET_ACCESS_KEY=
9+
AWS_REGION=
10+
IS_LOCAL_DB=false
11+
DYNAMODB_URL=
12+
ATTACHMENT_S3_BUCKET=
13+
S3_API_VERSION=
14+
ES_HOST=
15+
ES_API_VERSION=
16+
ES_INDEX=
17+
ES_TYPE=
18+
ES_REFRESH=true
19+
RESOURCES_API_URL=
20+
GROUPS_API_URL=
21+
PROJECTS_API_URL=
22+
COPILOT_RESOURCE_ROLE_IDS=
23+
M2M_AUDIT_HANDLE=

0 commit comments

Comments
 (0)