Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
f128464
updating the server code
rkrishnasanka Aug 18, 2022
58700e0
added flaskimplementaiton into workspace
rkrishnasanka Aug 18, 2022
6147fc3
Updated server code
rkrishnasanka Aug 18, 2022
3924808
Got the basic login system working
rkrishnasanka Aug 18, 2022
9443768
Added files for baseline
rkrishnasanka Aug 18, 2022
993d35e
Threw in a whole chunk of skeleton code
rkrishnasanka Aug 18, 2022
4930df8
Completed the CRUD Rest API skeleton. Need to test all the pathways n…
rkrishnasanka Aug 18, 2022
e8cf487
Need to modify the filesystem functions onto aws s3
rkrishnasanka Aug 18, 2022
4e5a695
Updated flask implementation with more read info
rkrishnasanka Aug 19, 2022
9bffa4a
Updated the code to make it break with the blueprint classes
rkrishnasanka Aug 19, 2022
2b3cf98
Updated the things to kind of work, now debugging all the crud operat…
rkrishnasanka Aug 21, 2022
731a80a
Updated the dev server to be more flexible for development work using…
rkrishnasanka Sep 7, 2022
91947d7
modified some pyenv stuff. Most likely will change it back later on.
rkrishnasanka Sep 7, 2022
2d69966
Removed the redis dependency form dockerfile
rkrishnasanka Sep 7, 2022
98728c8
Added a more detailed how-to guide for converting mint to parchmint b…
rkrishnasanka Oct 1, 2022
fd88ea6
Updated the pyfluigi project
rkrishnasanka Oct 1, 2022
c99ff14
Got the REST Signup and Authentication API working
rkrishnasanka Nov 20, 2022
622022a
Updated the user api to send out all the user data
rkrishnasanka Nov 20, 2022
8073965
Added the Zip file download capability and removed the update endpoin…
rkrishnasanka Nov 20, 2022
f21b8cb
- Added the uploads directory for flask in the server config
rkrishnasanka Nov 21, 2022
c92b4a1
Added update_file as a method to the s3 filesystem stub
rkrishnasanka Nov 21, 2022
f37ad77
Got the basic socketio initialized and running. Also created a test p…
rkrishnasanka Nov 21, 2022
1d17dac
Switched out the socketio version
rkrishnasanka Nov 21, 2022
74e3232
- Fixed issue with the contain parameter
rkrishnasanka Nov 21, 2022
d63237e
Changes:
rkrishnasanka Nov 23, 2022
7ed0cb7
Updated the code to be compatible with downloading the data from s3 a…
rkrishnasanka Nov 23, 2022
30d2b87
Updated the job api, etc.
rkrishnasanka Nov 23, 2022
3d0691d
Updated API code to be more kosher
rkrishnasanka Nov 28, 2022
d97d937
Fixed all issues in the job downloading process
rkrishnasanka Nov 28, 2022
91a424d
Updated the ziphelper
rkrishnasanka Nov 28, 2022
f8535f5
A lot of changes to dev infrastructure and everything
rkrishnasanka Nov 29, 2022
e590691
A lot of changes to dev infrastructure and everything
rkrishnasanka Nov 29, 2022
5de4bcc
Saving all the code changes before pull updates and also trying out o…
rkrishnasanka Jan 19, 2023
5102b4b
Added the skeleton of the new api consumptions services
rkrishnasanka Jan 19, 2023
99ca081
Merge branch 'flask-reimplementation' of https://github.com/cidarlab/…
rkrishnasanka Feb 19, 2023
9fc8159
Experiment/openapi-flask (#18)
rkrishnasanka Feb 19, 2023
87c7339
Merge branch 'flask-reimplementation' of https://github.com/cidarlab/…
rkrishnasanka Feb 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
33 changes: 7 additions & 26 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,41 +10,22 @@ ENV PYTHONFAULTHANDLER=1 \
PIP_NO_CACHE_DIR=off \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
POETRY_VERSION=1.0.0
POETRY_VERSION=1.1.14


# [Optional] Uncomment this section to install additional OS packages.
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends curl build-essential \
software-properties-common python3-pip make build-essential \
python-dev libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
libsqlite3-dev curl libffi-dev redis-server openjdk-8-jre-headless \
graphviz libgraphviz-dev libcairo2-dev pkg-config python3-dev python3.8-venv \
python3-setuptools
software-properties-common python3-pip make build-essential \
python-dev libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
libsqlite3-dev curl libffi-dev redis-server openjdk-8-jre-headless \
graphviz libgraphviz-dev libcairo2-dev pkg-config python3-dev python3.8-venv \
python3-setuptools liblzma-dev \
python3-pygraphviz apt-utils


RUN pip install "poetry==$POETRY_VERSION"

# # Pyenv for our baseline python environment for poetry later on.
RUN git clone git://github.com/yyuu/pyenv.git .pyenv
RUN git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

ENV PYENV_ROOT $HOME/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH

RUN pyenv install 3.8.0
RUN pyenv global 3.8.0

RUN poetry config virtualenvs.create false
RUN pip install --upgrade pip
RUN pip install setuptools pygraphviz


# Installing Nodejs
RUN curl -sL --location https://deb.nodesource.com/setup_16.x | bash -
RUN apt-get -y install nodejs

#Remove this later on
RUN apt-get -y install redis-server


67 changes: 51 additions & 16 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,26 +1,61 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.194.3/containers/ubuntu
// https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/docker-existing-docker-compose
// If you want to run as a non-root user in the container, see .devcontainer/docker-compose.yml.
{
"name": "Ubuntu",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick an Ubuntu version: hirsute, focal, bionic
"args": { "VARIANT": "focal" }
},
"name": "Neptune Dev - Existing Docker Compose (Extend)",

// Set *default* container specific settings.json values on container create.
"settings": {},
// Update the 'dockerComposeFile' list if you have more compose files or use different names.
// The .devcontainer/docker-compose.yml file contains any overrides you need/want to make.
"dockerComposeFile": [
"../docker-compose.yml",
"docker-compose.extend.yml"
],

// The 'service' property is the name of the service for the container that VS Code should
// use. Update this value and .devcontainer/docker-compose.yml to the real service name.
"service": "dev",

// Add the IDs of extensions you want installed when the container is created.
"extensions": [],
// The optional 'workspaceFolder' property is the path VS Code should open by default when
// connected. This is typically a file mount in .devcontainer/docker-compose.yml
"workspaceFolder": "/workspace",

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "uname -a",
// Uncomment the next line if you want start specific services in your Docker Compose config.
"runServices": ["s3local", "broker", "backend-database"],

// Uncomment the next line if you want to keep your containers running after VS Code shuts down.
"shutdownAction": "stopCompose",

// Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
//"remoteUser": "vscode"
}
// Uncomment the next line to run commands after the container is created - for example installing curl.
// "postCreateCommand": "cd flask-reimplementation && poetry install --no-interaction --no-ansi && cd ../job-runner && pip install .",

// Uncomment to connect as a non-root user if you've added one. See https://aka.ms/vscode-remote/containers/non-root.
// "remoteUser": "vscode"
"customizations":{"vscode": {
"extensions": [
"GitHub.copilot",
"LeoJhonSong.python-extension-pack",
"littlefoxteam.vscode-python-test-adapter",
"njqdev.vscode-python-typehint",
"redhat.vscode-yaml",
"streetsidesoftware.code-spell-checker",
"yzhang.markdown-all-in-one",
"VisualStudioExptTeam.vscodeintellicode",
"GitHub.vscode-pull-request-github",
"bungcip.better-toml",
"njpwerner.autodocstring",
"ms-python.python",
"VisualStudioExptTeam.vscodeintellicode"
]
}
},
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/git-lfs:1": {},
"ghcr.io/devcontainers/features/node:1": {},
"ghcr.io/devcontainers/features/python:1": {},
"ghcr.io/devcontainers-contrib/features/poetry:2": {}
}
}
3 changes: 2 additions & 1 deletion .env
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# celery details
CELERY_BROKER_URL="redis://broker:6379/0"
CELERY_TASK_META_COLLECTION_NAME="celery_task_meta"

# mongodb details
MONGO_HOST="backend-database"
MONGO_PORT="27017"
MONGO_INITDB_ROOT_USERNAME="root"
MONGO_INITDB_ROOT_PASSWORD="rootpassword"
MONGO_INITDB_NAME="jobs"
MONGO_INITDB_NAME="neptune"

# Neptune details
NEPTUNE_MONGODB_DBNAME="neptune"
Expand Down
3 changes: 2 additions & 1 deletion Neptune-UI/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@
"@sentry/vue": "^6.1.0",
"axios": "^0.19.2",
"core-js": "^3.6.2",
"socket.io-client": "^4.5.3",
"vue": "^2.6.11",
"vue-i18n": "^8.15.3",
"vue-monaco": "^1.1.0",
"vue-router": "^3.1.3",
"vue-socket.io": "^3.0.7",
"vue-socket.io-extended": "^4.2.0",
"vue-terminal-ui": "^0.1.6",
"vuetify": "^2.2.6",
"vuex": "^3.1.2",
Expand Down
3 changes: 3 additions & 0 deletions Neptune-UI/src/assets/css/materialdesignicons.min.css

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions Neptune-UI/src/assets/css/materialdesignicons.min.css.map

Large diffs are not rendered by default.

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
36 changes: 23 additions & 13 deletions Neptune-UI/src/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,16 @@ import "./plugins/vee-validate";
import "./plugins/vue-world-map";
import vuetify from "./plugins/vuetify";
import i18n from "./i18n";
import VueSocketIO from "vue-socket.io";


import * as Sentry from "@sentry/vue";
import { Integrations } from "@sentry/tracing";

import VueSocketIOExt from 'vue-socket.io-extended';
import { io } from 'socket.io-client';



Sentry.init({
Vue,
dsn:
Expand All @@ -39,18 +44,23 @@ Sentry.init({

Vue.config.productionTip = false;

Vue.use(
new VueSocketIO({
debug: true,
connection: "http://" + window.location.hostname + ":3000"
// vuex: {
// store,
// actionPrefix: 'SOCKET_',
// mutationPrefix: 'SOCKET_'
// },
// options: { path: "/my-app/" } //Optional options
})
);
const socket = io('http://127.0.0.1:8080');

Vue.use(VueSocketIOExt, socket);


// Vue.use(
// new VueSocketIO({
// debug: true,
// connection: "localhost:8080" //"http://" + window.location.hostname
// // vuex: {
// // store,
// // actionPrefix: 'SOCKET_',
// // mutationPrefix: 'SOCKET_'
// // },
// // options: { path: "/my-app/" } //Optional options
// })
// );

router.beforeEach((to, from, next) => {
const authRequired = to.matched.some(route => route.meta.requiresAuth);
Expand Down
8 changes: 8 additions & 0 deletions Neptune-UI/src/services/auth_header.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
export default function authHeader() {
const user = JSON.parse(localStorage.getItem('user'));
if (user && user.accessToken) {
return { Authorization: 'Bearer ' + user.accessToken };
} else {
return {};
}
}
33 changes: 33 additions & 0 deletions Neptune-UI/src/services/auth_service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import axios from axios;


class AuthService {
login(user) {
return axios
.post(API_URL + 'signin', {
username: user.username,
password: user.password
})
.then(response => {
if (response.data.accessToken) {
localStorage.setItem('user', JSON.stringify(response.data));
}

return response.data;
});
}

logout() {
localStorage.removeItem('user');
}

register(user) {
return axios.post(API_URL + 'signup', {
username: user.username,
email: user.email,
password: user.password
});
}
}

export default new AuthService();
5 changes: 5 additions & 0 deletions Neptune-UI/src/services/file_service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class FileService {

}

export default new FileService();
5 changes: 5 additions & 0 deletions Neptune-UI/src/services/job_service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class JobService {

}

export default new JobService();
9 changes: 9 additions & 0 deletions Neptune-UI/src/services/user_service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
const API_URL = '/api/v2/user'

class UserService {
getUserData() {
return axios.get(API_URL, { headers: authHeader(), });
}
}

export default new UserService();
5 changes: 5 additions & 0 deletions Neptune-UI/src/services/workspace_service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class WorkSpaceService {

}

export default new WorkSpaceService();
5 changes: 5 additions & 0 deletions Neptune-UI/src/store.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ export default new Vuex.Store({
currentFile: null,
currentWorkspace: null,
currentUser: null,
authToken:''
},
mutations: {
SET_BAR_IMAGE (state, payload) {
Expand All @@ -39,6 +40,9 @@ export default new Vuex.Store({
},
SET_CURRENT_USER(state, payload){
state.currentUser = payload
},
SET_AUTH_TOKEN(state, payload){
state.authToken = payload
}
},
getters: {
Expand All @@ -47,6 +51,7 @@ export default new Vuex.Store({
currentFile: state => state.currentFile,
currentWorkspace: state => state.currentWorkspace,
currentUser: state => state.currentUser,
authToken: state => state.authToken
},
actions: {

Expand Down
11 changes: 10 additions & 1 deletion Neptune-UI/src/utils.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,13 @@
export function getprettytimestamp (datestring = Date.now().toString()){
let dateobject = new Date(datestring)
return dateobject.toLocaleDateString()+ " " + dateobject.toLocaleTimeString()
}
}

export default function authHeader() {
const user = JSON.parse(localStorage.getItem('user'));
if (user && user.accessToken) {
return { Authorization: 'Bearer ' + user.accessToken };
} else {
return {};
}
}
14 changes: 13 additions & 1 deletion Neptune-UI/src/views/pages/Landing.vue
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
<v-img
:src="require(`@/assets/Neptune-logo-white-text.png`)"
height="300px"
contain="true"
/>
</v-slide-y-transition>
</v-row>
Expand Down Expand Up @@ -39,6 +38,19 @@
},
],
}),
mounted () {
console.log('Landing.vue mounted hook')
this.$socket.client.emit('echo', 'Hello World!')
},
sockets: {
connect () {
console.log('socket connected')
},
echo (val) {
console.log('this method was fired by the socket server. eg: io.emit("customEmit", data)')
console.log(val)
},
},
methods:{
login: function(event){
console.log("TEST")
Expand Down
Loading