0.3

Author: ngrislain

README.md

@@ -1,38 +1,65 @@
 # datasets
 
+This helps with the use of standard SQL datasets.
+
+It comes with 4 datasets:
+- 'extract': an extract from 2 simple datasets 'census' (from the US cenus) and 'beacon' (with japanese names and labels).
+- 'financial': from https://relational.fit.cvut.cz/dataset/Financial
+- 'imdb': from https://relational.fit.cvut.cz/dataset/IMDb
+- 'hematitis': from https://relational.fit.cvut.cz/dataset/Hepatitis
+
 ## Instalation
 
-To build the datasets, install the requirements with:
+The package can be installed with:
 ```bash
-poetry shell
+pip install qrlew-datasets
 ```
+The library assumes:
+- either that postgresql is installed,
+- or that docker is installed and can spawn postgresql containers.
 
-You can then build the datasets with:
-```bash
-python -m datasets.build
-```
+### Postgresql in a container
 
-You may need to install the requirements of some drivers such as: https://pypi.org/project/mysqlclient/
+The library automatically spawns containers. There is nothing to do.
 
-## Test in an environment without docker
+### Without docker installed
 
-You can simulate this by trying to run this code inside a container:
-`docker run --name test -d -i -t -v .:/datasets ubuntu:22.04`
-Then run:
-`docker exec -it test bash`
+Setup a `psql` as in https://colab.research.google.com/github/tensorflow/io/blob/master/docs/tutorials/postgresql.ipynb
 
-Then setup a `psql` as in https://colab.research.google.com/github/tensorflow/io/blob/master/docs/tutorials/postgresql.ipynb
+You can set the port to use: here 5433.
 
 ```bash
 # Inspred by https://colab.research.google.com/github/tensorflow/io/blob/master/docs/tutorials/postgresql.ipynb#scrollTo=YUj0878jPyz7
 sudo apt-get -y -qq update
-sudo apt-get -y -qq install postgresql
+sudo apt-get -y -qq install postgresql-14
 # Start postgresql server
-sudo sed -i "s/#port = 5432/port = 5433/g" /etc/postgresql/14/main/postgresql.conf
+# sudo sed -i "s/#port = 5432/port = 5433/g" /etc/postgresql/14/main/postgresql.conf
+sudo sed -i "s/port = 5432/port = 5433/g" /etc/postgresql/14/main/postgresql.conf
 sudo service postgresql start
 # Set password
 sudo -u postgres psql -U postgres -c "ALTER USER postgres PASSWORD 'pyqrlew-db'"
 # Install python packages
+```
+
+#### Testing the absence of docker if docker is installed:
+
+You can simulate the absence of docker by running this code inside a container.
 
+First run:
+`docker run --name test -d -i -t -v .:/datasets ubuntu:22.04`
+Then run:
+`docker exec -it test bash`
+
+## Building the `.sql` dumps
+
+To build the datasets, install the requirements with:
+```bash
+poetry shell
 ```
-<!-- TODO port not working -->
+
+You can then build the datasets with:
+```bash
+python -m datasets.build
+```
+
+You may need to install the requirements of some drivers such as: https://pypi.org/project/mysqlclient/

examples/simple.py

@@ -17,7 +17,7 @@
 #     '--publish', '5432:5432',
 #     'postgres'])
 
-db = PostgreSQL(name=name, user='postgres', password=name, port=5432)
+db = PostgreSQL(name=name, user='postgres', password=name, port=5433)
 db.load('/datasets/datasets/sources/extract/extract.sql')
 db.set_schema('extract')
 db.dump('/tmp/test.dump')

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "qrlew-datasets"
-version = "0.2.6"
+version = "0.3.0"
 description = "Sample SQL datasets"
 authors = ["Nicolas Grislain <[email protected]>"]
 license = "Apache 2.0"