Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*.example
.dockerignore
.git
.gitignore
Dockerfile
README.md
__pycache__
docker-compose.yml
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
db.sqlite3
*.pyc
.env.*
__pycache__
db.sqlite3
eve_data/*.yaml
dscantool/settings.py
96 changes: 96 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
### BUILDER

FROM python:3.9-alpine as builder

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

WORKDIR /usr/src/main

COPY ./requirements.txt .

RUN set -eux; \
apk add \
gcc \
libc-dev \
libffi-dev \
postgresql-dev \
python3-dev \
musl-dev \
; \
pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/main/wheels -r requirements.txt

### FINAL

FROM python:3.9-alpine

ENV DSCAN_HOME_DIR ${DSCAN_HOME_DIR:-/dscantool}
ENV DSCAN_WORK_DIR $DSCAN_HOME_DIR/app
ENV DSCAN_STATIC_DIR $DSCAN_HOME_DIR/static
ENV DSCAN_GROUP ${DSCAN_GROUP:-dscantool}
ENV DSCAN_GID ${DSCAN_GID:-10000}
ENV DSCAN_USER ${DSCAN_USER:-dscantool}
ENV DSCAN_UID ${DSCAN_UID:-10000}
ENV DSCAN_PORT ${DSCAN_PORT:-8000}
ENV GOSU_VERSION 1.16

WORKDIR $DSCAN_WORK_DIR

COPY --from=builder /usr/src/main/wheels /wheels
COPY --from=builder /usr/src/main/requirements.txt .
COPY . $DSCAN_WORK_DIR

# Install gosu
RUN set -eux; \
\
apk add --no-cache --virtual .gosu-deps \
ca-certificates \
dpkg \
gnupg \
; \
\
dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
\
# verify the signature
export GNUPGHOME="$(mktemp -d)"; \
gpg --batch --keyserver hkps://keyserver.ubuntu.com --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
\
# clean up fetch dependencies
apk del --no-network .gosu-deps; \
\
chmod +x /usr/local/bin/gosu; \
# verify that the binary works
gosu --version; \
gosu nobody true

# Install dscantool
RUN \
set -eux; \
mkdir -p $DSCAN_WORK_DIR; \
mkdir -p $DSCAN_STATIC_DIR; \
addgroup -g $DSCAN_GID $DSCAN_GROUP; \
adduser -u $DSCAN_UID -h $DSCAN_HOME_DIR -G $DSCAN_GROUP -D $DSCAN_USER; \
apk add libpq tini; \
pip install --no-cache /wheels/*; \
chmod +x $DSCAN_WORK_DIR/run.sh; \
chown -R $DSCAN_USER:$DSCAN_GROUP $DSCAN_HOME_DIR

# Get the EVE Static Data Export (SDE)
RUN \
set -eux; \
wget https://eve-static-data-export.s3-eu-west-1.amazonaws.com/tranquility/sde.zip; \
unzip sde.zip; \
for f in `find sde -iname "groupIDs.yaml"`; do cp $f ./eve_data/; done; \
for f in `find sde -iname "typeIDs.yaml"`; do cp $f ./eve_data/; done; \
rm -rf sde*

EXPOSE $DSCAN_PORT
VOLUME $DSCAN_STATIC_DIR
ENTRYPOINT ["tini", "--"]

CMD ["./run.sh"]
48 changes: 38 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,54 @@
# Nimos' Intel Tool

Nimos' Intel Tool is a web application that parses DScans and local scans from the game Eve Online.
Nimos' Intel Tool is a web application that parses DScans and local scans from the game Eve Online.
I made it because my previous favorite Intel Tool dscan.me shut down and I wanted another one with a dark theme.

## Standalone installation

(I have not tested this method)

## Installation
1. Clone this repository
2. Install requirements with `pip3 install -r requirements.txt` or from another source of your liking
3. Edit `dscan/settings.py` to your liking
4. Install database with `python3 manage.py migrate`
2. Install requirements with `pip install -r requirements.txt` or from another source of your liking
3. Edit `dscantool/settings.py` to your liking
4. Install database with `python manage.py migrate`
5. Generate static files `python manage.py collectstatic --noinput`

### Optional: Import Type information from SDE

## Optional: Import Type information from SDE
Note: Nimos' Intel Tool pulls all missing type information from ESI when needed. When starting out with an empty cache that can make the first scans take a couple of seconds to parse, but once a ship type is cached it won't be requested again. You can import all ship type information from SDE to "seed" the cache, but doing so is optional and the script is no longer maintained. Use at your own discretion or feel free to import the SDE another way.

1. Get the EVE Static Data Export (SDE) from https://developers.eveonline.com/resource/resources
2. Extract `groupIDs.yaml` and `typeIDs.yaml` from the SDE into eve_data
3. Run `python3 manage.py import_types`

2. Extract `groupIDs.yaml` and `typeIDs.yaml` from the SDE into `eve_data`
3. Run `python manage.py import_sde`

## Docker installation

### Environment Variables

| Variable | Default | Dscription |
|----------------------|-------------------------------------------------------|------------|
|`DSCAN_SECRET_KEY` |`&f8lk5k-b$^)-sdfij8gu9df7g9aa2zagdfh8iooijioukhrs_` |A secret key for a particular Django installation.|
|`DSCAN_ALLOWED_HOST` |`dscan.nimos.ws` |String representing the host/domain name that this Django site can serve.|
|`DSCAN_DB_ENGINE` |`django.db.backends.postgresql` |The database backend to use. The built-in database backends are: `django.db.backends.postgresql`, `django.db.backends.mysql`, `django.db.backends.sqlite3`, `django.db.backends.oracle`|
|`DSCAN_DB_NAME` |`dscantool` |The name of the database to use.|
|`DSCAN_DB_USER` |`dscantool` |The username to use when connecting to the database.|
|`DSCAN_DB_PASSWORD` |`dscantool` |The password to use when connecting to the database.|
|`DSCAN_DB_HOST` |`127.0.0.1` |Which host to use when connecting to the database.|
|`DSCAN_DB_PORT` |`5432` |The port to use when connecting to the database.|
|`DSCAN_LANGUAGE_CODE` |`en-us` |A string representing the language code for this installation.|
|`DSCAN_STATIC_URL` |`/static/` |URL to use when referring to static files located in `DSCAN_STATIC_ROOT`.|
|`DSCAN_STATIC_ROOT` |`/dscantool/static/` |The absolute path to the directory where collectstatic will collect static files for deployment.|
|`DSCAN_SITE_NAME` |`Nimos' Intel Tools` |The site name.|
|`DSCAN_EVE_IMAGE_URL` |`https://image.eveonline.com` |Link to EVE Online image server.|
|`DSCAN_REPOSITORY_URL`|`https://github.com/nimos/dscantool` |Link to development repository.|
|`DSCAN_ZKILLBOARD_URL`|`https://zkillboard.com` |Link to zKillboard.|
|`DSCAN_DOTLAN_URL` |`https://evemaps.dotlan.net` |Link to Dotlan.|
|`DSCAN_FEEDBACK` |`Brought to you by Nimos Endashi • Discord: Nimos#4951`|Feedback information.|

## Notes

This is just a little 2-day project, so don't expect the sleekest code.

This tool relies heavily on caching. The first few scans will be slow until the database has seen most ship types and corporations, after a couple of scans performance should increase significantly. The up-side is that you never have to update the static data like you have to for many other EVE tools.

This tool has been tested with MySQL and SQLite. Feel free to use other database backends, but they might require minor fixes. When using MySQL, make sure your tables (specifically the dscan_scan and esi_invtype) use the 'utf8mb4' encoding. Otherwise it will error out trying to save certain characters.
This tool has been tested with MySQL and SQLite. Feel free to use other database backends, but they might require minor fixes. When using MySQL, make sure your tables (specifically the `dscan_scan` and `esi_invtype`) use the 'utf8mb4' encoding. Otherwise it will error out trying to save certain characters.
55 changes: 55 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
version: '3.9'

#
# !!! ATTENTION !!!
# It doesn't work.
# Use it as an example.
#

networks:
front:
dscan:

services:

nginx:
image: "nginx:alpine"
container_name: nginx
restart: always
volumes:
- ./dscan/dscantool/static:/dscantool/static
ports:
- 80:80
networks:
- front

dscan:
build:
context: ./
dockerfile: Dockerfile
container_name: dscan
restart: always
environment:
- DSCAN_ALLOWED_HOST=dscan.localdomain
- DSCAN_DB_HOST=dscan_postgres
volumes:
- ./dscan/dscantool/static:/dscantool/static
depends_on:
- dscan_postgres
networks:
- front
- dscan
logging: *default-logging

dscan_postgres:
image: "postgres:15-alpine"
container_name: dscan_postgres
restart: always
environment:
- POSTGRES_DB=dscantool
- POSTGRES_USER=dscantool
- POSTGRES_PASSWORD=dscantool
networks:
- dscan
volumes:
- ./dscan/postgres/data:/var/lib/postgresql/data
5 changes: 2 additions & 3 deletions dscan/templates/about.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% extends "base.html" %}
{% block title %}- About{%endblock title %}
{% block title %}About | {{site_name}}{%endblock title %}
{% block content %}

<div class="container flexbox" style="margin-top: 100px">
Expand All @@ -25,9 +25,8 @@ <h4>License</h4>
GNU Affero General Public License for more details.
</p>
<p>
The full license can be found <a href="https://github.com/nimos/dscantool">here</a>.
The full license can be found <a href="https://choosealicense.com/licenses/agpl-3.0/">here</a>.
</p>
</div>
</div>
{% endblock content %}

30 changes: 14 additions & 16 deletions dscan/templates/base.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,16 @@
<html>
<head>
<meta charset="utf-8">
<link href="https://fonts.googleapis.com/css?family=Bitter|Roboto:400,700|Roboto+Mono" rel="stylesheet">
<link rel="stylesheet"
href="https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/css/bootstrap.min.css"
integrity="sha384-GJzZqFGwb1QTTN6wy59ffF1BuGJpLSa9DkKMp0DgiMDm4iYMj70gZWKYbI706tWS"
crossorigin="anonymous">
<link href="https://fonts.googleapis.com/css?family=Bitter|Roboto:400,700|Roboto+Mono" rel="stylesheet" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.6.2/dist/css/bootstrap.min.css" integrity="sha384-xOolHFLEh07PJGoPkLv1IbcEPTNtaed2xpHsD9ESMhqIYd0nLMwNLD69Npy4HI+N" crossorigin="anonymous">
<link rel="shortcut icon" type="image/png" href="/static/favicon.png"/>
<link rel="shortcut icon" type="image/svg+xml" href="/static/favicon.svg"/>
{% if debug %}
<link href="{% static "css/main.less" %}" rel="stylesheet/less">
{% else %}
<link href="{% static "css/main-compiled.css" %}" rel="stylesheet">
{% endif %}
<title>{% block title %}Nimos' Intel Tool{% endblock title %}</title>
<title>{% block title %}{{site_name}}{% endblock title %}</title>
{% block meta %}
{% endblock meta %}
</head>
Expand All @@ -26,14 +23,9 @@
{% endblock content %}


<div style="position: relative;" class="footer text-shadow container-fluid">
<div class="text-shadow container-fluid">
<div class="row">
<div class="col"></div>
<div class="col">
<span>Brought to you by Nimos Endashi • Discord: Nimos#4951 <br>
<a href="/about/">All Eve Related Materials are property of CCP Games</a> • <a href="/about/">Released under AGPL3</a> • <a href="//github.com/nimos/dscantool">Github</a></span>
</div>
<div class="col stats">
<div class="col order-12 stats">
<div>
<ul>
<li>L: {{stats.l}} / D: {{stats.d}}</li>
Expand All @@ -42,14 +34,20 @@
</div>
</div>
</div>
<div class="row">
<div class="col-12 text-center">
<span>{{feedback}}<br>
<a href="/about/">All Eve Related Materials are property of CCP Games</a> • <a href="/about/">Released under AGPL3</a> • <a href="{{repository_url}}">Github</a></span>
</div>
</div>
</div>

{% if debug %}
<script src="//cdnjs.cloudflare.com/ajax/libs/less.js/3.8.1/less.min.js" ></script>
{% endif %}
<script src="https://code.jquery.com/jquery-latest.min.js" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/js/bootstrap.min.js" integrity="sha384-B0UglyR+jN6CkvvICOB2joaf5I4l3gm9GU6Hc1og6Ls7i6U/mkkaduKaBhlAXv9k" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.slim.min.js" integrity="sha384-DfXdz2htPH0lsSSs5nCTpuj/zy4C+OGpamoFVy38MVBnE+IbbVYUew+OrCXaRkfj" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@4.6.2/dist/js/bootstrap.bundle.min.js" integrity="sha384-Fy6S3B9q64WdZWQUiU+q4/2Lc9npb8tCaSX9FK7E8HnRr0Jz8D6OP9dO5Vg3Q9ct" crossorigin="anonymous"></script>
{% block scripts %}
{% endblock %}
</body>
</html>
</html>
25 changes: 11 additions & 14 deletions dscan/templates/dscan.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,31 +3,28 @@
{% block meta %}
<meta property="og:title" content="Scan{% if solarSystem %} of {{solarSystem}} at {%else%} taken at {%endif%}{{created|date:"H:i:s"}}" />
<meta property="og:url" content="{{request.get_full_path}}" />
<meta property="og:description"
content="{{summaryText}}" />
<meta property="og:site_name" content="Directional Scan" />
<meta property="og:description" content="{{summaryText}}" />
<meta property="og:site_name" content="Directional Scan | {{site_name}}" />
{% endblock meta %}

{% block title %}
D-Scan {{token}} - Nimos' Intel Tools
D-Scan {{token}} | {{site_name}}
{% endblock title %}

{% block content %}
<div class="container">
<div class="card">
<div class="navbar navbar-inverse">
<span>
<a href="/">Home</a> &gt; Directional Scan
<span>
<a href="/">Home</a> &gt; Directional Scan
<small>[{{token}}] / Created {{created|date:"M d, Y - H:i:s"}} ({{ created|timesince }} ago)</small>
</span>
<span>
Solar System:
{% if solarSystem %}
{{solarSystem}}
<a target="_blank" href="https://evemaps.dotlan.net/system/{{solarSystem}}/" class="external-icon dotlan"></a>
<a target="_blank" href="https://zkillboard.com/system/{{solarSystem}}/" class="external-icon zkb"></a>
{%else%}
Unknown
Solar System:
{{solarSystem}}
<a target="_blank" href="{{dotlan_url}}/system/{{solarSystem}}/" class="external-icon dotlan"></a>
<a target="_blank" href="{{zkillboard_url}}/system/{{solarSystem}}/" class="external-icon zkb"></a>
{%endif%}
</span>
</div>
Expand Down Expand Up @@ -55,7 +52,7 @@
<tr class="group" data-group="{{group.id}}">
<td class="number">{{group.count}}</td><td class="name">{{group.name}}</td>
</tr>
{% endfor %}
{% endfor %}
</table>
</div>
</div>
Expand Down Expand Up @@ -86,4 +83,4 @@
})
</script>

{% endblock scripts %}
{% endblock scripts %}
Loading