Skip to content

Commit dbbf36d

Browse files
ThomasKranitsasThomasKranitsas
committed
Initial commit
0 parents  commit dbbf36d

39 files changed

+12442
-0
lines changed

‎.gitignore

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
8+
# Runtime data
9+
pids
10+
*.pid
11+
*.seed
12+
*.pid.lock
13+
14+
# Directory for instrumented libs generated by jscoverage/JSCover
15+
lib-cov
16+
17+
# Coverage directory used by tools like istanbul
18+
coverage
19+
20+
# nyc test coverage
21+
.nyc_output
22+
23+
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
24+
.grunt
25+
26+
# Bower dependency directory (https://bower.io/)
27+
bower_components
28+
29+
# node-waf configuration
30+
.lock-wscript
31+
32+
# Compiled binary addons (https://nodejs.org/api/addons.html)
33+
build/Release
34+
35+
# Dependency directories
36+
node_modules/
37+
jspm_packages/
38+
39+
# TypeScript v1 declaration files
40+
typings/
41+
42+
# Optional npm cache directory
43+
.npm
44+
45+
# Optional eslint cache
46+
.eslintcache
47+
48+
# Optional REPL history
49+
.node_repl_history
50+
51+
# Output of 'npm pack'
52+
*.tgz
53+
54+
# Yarn Integrity file
55+
.yarn-integrity
56+
57+
# dotenv environment variables file
58+
.env
59+
60+
# next.js build output
61+
.next

‎README.md

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Topcoder Member API v5
2+
3+
## Dependencies
4+
5+
- nodejs https://nodejs.org/en/ (v10)
6+
- DynamoDB
7+
- AWS S3
8+
- Elasticsearch v6
9+
- Docker, Docker Compose
10+
11+
## Configuration
12+
13+
Configuration for the application is at `config/default.js`.
14+
The following parameters can be set in config files or in env variables:
15+
16+
- LOG_LEVEL: the log level, default is 'debug'
17+
- PORT: the server port, default is 3000
18+
- AUTH_SECRET: The authorization secret used during token verification.
19+
- VALID_ISSUERS: The valid issuer of tokens.
20+
- AUTH0_URL: AUTH0 URL, used to get M2M token
21+
- AUTH0_PROXY_SERVER_URL: AUTH0 proxy server URL, used to get M2M token
22+
- AUTH0_AUDIENCE: AUTH0 audience, used to get M2M token
23+
- TOKEN_CACHE_TIME: AUTH0 token cache time, used to get M2M token
24+
- AUTH0_CLIENT_ID: AUTH0 client id, used to get M2M token
25+
- AUTH0_CLIENT_SECRET: AUTH0 client secret, used to get M2M token
26+
- BUSAPI_URL: Bus API URL
27+
- KAFKA_ERROR_TOPIC: Kafka error topic used by bus API wrapper
28+
- AMAZON.AWS_ACCESS_KEY_ID: The Amazon certificate key to use when connecting. Use local dynamodb you can set fake value
29+
- AMAZON.AWS_SECRET_ACCESS_KEY: The Amazon certificate access key to use when connecting. Use local dynamodb you can set fake value
30+
- AMAZON.AWS_REGION: The Amazon certificate region to use when connecting. Use local dynamodb you can set fake value
31+
- AMAZON.IS_LOCAL_DB: Use Amazon DynamoDB Local or server.
32+
- AMAZON.DYNAMODB_URL: The local url if using Amazon DynamoDB Local
33+
- AMAZON.PHOTO_S3_BUCKET: the AWS S3 bucket to store photos
34+
- AMAZON.S3_API_VERSION: the AWS S3 API version
35+
- ES: config object for Elasticsearch
36+
- ES.HOST: Elasticsearch host
37+
- ES.API_VERSION: Elasticsearch API version
38+
- ES.ES_INDEX: Elasticsearch index name
39+
- ES.ES_TYPE: Elasticsearch index type
40+
- FILE_UPLOAD_SIZE_LIMIT: the file upload size limit in bytes
41+
- ID_FIELDS: member identifiable info fields, only admin, M2M, or member himself can get these fields
42+
- PHOTO_URL_TEMPLATE: photo URL template, its <key> will be replaced with S3 object key
43+
- VERIFY_TOKEN_EXPIRATION: verify token expiration in minutes
44+
- EMAIL_VERIFY_AGREE_URL: email verify agree URL, the <emailVerifyToken> will be replaced with generated verify token
45+
- EMAIL_VERIFY_DISAGREE_URL: email verify disagree URL
46+
- SCOPES: the configurable M2M token scopes, refer `config/default.js` for more details
47+
48+
49+
Set the following environment variables used by bus API to get TC M2M token (use 'set' insted of 'export' for Windows OS):
50+
```
51+
export AUTH0_CLIENT_ID=8QovDh27SrDu1XSs68m21A1NBP8isvOt
52+
export AUTH0_CLIENT_SECRET=3QVxxu20QnagdH-McWhVz0WfsQzA1F8taDdGDI4XphgpEYZPcMTF4lX3aeOIeCzh
53+
export AUTH0_URL=https://topcoder-dev.auth0.com/oauth/token
54+
export AUTH0_AUDIENCE=https://m2m.topcoder-dev.com/
55+
```
56+
57+
Also properly configure AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, ATTACHMENT_S3_BUCKET, IS_LOCAL_DB config parameters.
58+
59+
Test configuration is at `config/test.js`. You don't need to change them.
60+
The following test parameters can be set in config file or in env variables:
61+
62+
- ADMIN_TOKEN: admin token
63+
- USER_TOKEN: user token
64+
- EXPIRED_TOKEN: expired token
65+
- INVALID_TOKEN: invalid token
66+
- M2M_FULL_ACCESS_TOKEN: M2M full access token
67+
- M2M_READ_ACCESS_TOKEN: M2M read access token
68+
- M2M_UPDATE_ACCESS_TOKEN: M2M update (including 'delete') access token
69+
- S3_ENDPOINT: endpoint of AWS S3 API, for unit and e2e test only; default to `localhost:9000`
70+
71+
## AWS S3 Setup
72+
Go to https://console.aws.amazon.com/ and login. Choose S3 from Service folder and click `Create bucket`. Following the instruction to create S3 bucket.
73+
After creating bucket, click `Permissions` tab of the bucket, in the `Block public access` section, disable the block, so that public access
74+
is allowed, then we can upload public accessible photo to the bucket.
75+
76+
## Local services setup
77+
In the `local` folder, run `docker-compose up`
78+
It starts Elasticsearch, DynamoDB and S3 compatible server.
79+
80+
## Create Tables
81+
1. Make sure DynamoDB are running as per instructions above.
82+
2. Make sure you have configured all config parameters. Refer [Configuration](#configuration)
83+
3. Run `npm run create-tables` to create tables.
84+
85+
## Scripts
86+
1. Drop DB tables: `npm run drop-tables`
87+
2. Create DB tables: `npm run create-tables`
88+
3. Initialize/Clear database: `npm run init-db`
89+
4. Create Elasticsearch index: `npm run init-es`, or to re-create index: `npm run init-es force`
90+
5. Seed/Insert data to ES and DB: `npm run seed-data`
91+
6. View DB table data: `npm run view-db-data <ModelName>`, ModelName can be `Member`
92+
7. View ES data: `npm run view-es-data`
93+
94+
## Local Deployment
95+
96+
- Install dependencies `npm install`
97+
- Run lint `npm run lint`
98+
- Run lint fix `npm run lint:fix`
99+
- initialize Elasticsearch, create configured Elasticsearch index if not present: `npm run init-es`,
100+
or re-create the index: `npm run init-es force`
101+
- Create tables `npm run create-tables`
102+
- Clear and init db data `npm run init-db`
103+
- Start app `npm start`
104+
- App is running at `http://localhost:3000`
105+
106+
## Local Deployment with Docker
107+
108+
To run the app using docker, follow the below steps:
109+
110+
1. Navigate to the directory `docker`
111+
112+
2. Rename the file `sample.api.env` to `api.env`
113+
114+
3. Set the required AWS credentials in the file `api.env`, you may add more env variables to this file if needed
115+
116+
4. Once that is done, run the following command
117+
118+
```
119+
docker-compose up
120+
```
121+
122+
5. When you are running the application for the first time, It will take some time initially to download the image and install the dependencies
123+
124+
125+
## Running tests
126+
127+
### Prepare
128+
- Various config parameters should be properly set.
129+
- Start Local services.
130+
- Create DynamoDB tables.
131+
- Initialize ES index.
132+
- No need to seed test data, the tests should run in clean environment, if there are data in ES and DB, then clear them by running
133+
`npm run init-db` and `npm run init-es force`
134+
135+
136+
### Running unit tests
137+
To run unit tests alone
138+
139+
```bash
140+
npm run test
141+
```
142+
143+
To run unit tests with coverage report
144+
145+
```bash
146+
npm run test:cov
147+
```
148+
149+
### Running integration tests
150+
To run integration tests alone
151+
152+
```bash
153+
npm run e2e
154+
```
155+
156+
To run integration tests with coverage report
157+
158+
```bash
159+
npm run e2e:cov
160+
```
161+
162+
## Verification
163+
Refer to the verification document `Verification.md`
164+
165+
## Notes
166+
167+
- all JWT tokens provided in Postman environment file and tests are created in `https://jwt.io` with secret `mysecret`
168+

‎Verification.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# TopCoder Member API v5 Verification
2+
3+
## Postman tests
4+
- clear the environment, run command `npm run init-db` and `npm run init-es force`
5+
- create test data in ES and DB, run `npm run seed-data`
6+
- import Postman collection and environment in the docs folder to Postman
7+
- run the tests
8+
9+
## DynamoDB Verification
10+
Run command `npm run view-db-data <ModelName>` to view table data, ModelName can be `Member`
11+
12+
## S3 Verification
13+
14+
Login to AWS Console, S3 service, view the bucket content.
15+
16+
## ElasticSearch Verification
17+
18+
Run command `npm run view-es-data` to view data store in ES.
19+
20+
## Bus Event Verification
21+
22+
- login `https://lauscher.topcoder-dev.com/` with credential `tonyj / appirio123`
23+
- then select topic to view, see app-constants.js TOPICS field for used topics, then click `View` button to view related messages
24+

‎app-bootstrap.js

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
/**
2+
* App bootstrap
3+
*/
4+
global.Promise = require('bluebird')

‎app-constants.js

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
/**
2+
* App constants
3+
*/
4+
const ADMIN_ROLES = ['administrator', 'admin']
5+
6+
const EVENT_ORIGINATOR = 'topcoder-member-api'
7+
8+
const EVENT_MIME_TYPE = 'application/json'
9+
10+
const TOPICS = {
11+
MemberUpdated: 'member.action.profile.update',
12+
EmailChanged: 'member.action.email.profile.emailchange.verification'
13+
}
14+
15+
module.exports = {
16+
ADMIN_ROLES,
17+
EVENT_ORIGINATOR,
18+
EVENT_MIME_TYPE,
19+
TOPICS
20+
}

0 commit comments

Comments
 (0)