Skip to content

Commit

Permalink
Refactoring/295 remove dist module (#296)
Browse files Browse the repository at this point in the history
* #295: removed dist module
* #295: changed project structure
* #295: changed version to 3.0.1
* #295: upgraded dependencies to latest versions
* #295: fixed broken dependency
* #295: Repaired launch configurations. Added Maven nature to root Eclipse project.
  • Loading branch information
AnastasiiaSergienko authored Jan 10, 2020
1 parent 9d3d0df commit f37bbae
Show file tree
Hide file tree
Showing 259 changed files with 264 additions and 480 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ dependency-reduced-pom.xml
# Others
.DS_Store
*.swp
**/local
local
Scripts
.dbeaver*
**/*.log
File renamed without changes.
File renamed without changes.
File renamed without changes.
4 changes: 4 additions & 0 deletions .settings/org.eclipse.m2e.core.prefs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
activeProfiles=
eclipse.preferences.version=1
resolveWorkspaceProjects=true
version=1
12 changes: 4 additions & 8 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,21 @@ matrix:
include:
- jdk: "openjdk11"


addons:
sonarcloud:
organization: exasol

before_script:
- tools/version.sh verify
- sudo pip install pipenv
- ./jdbc-adapter/tools/version.sh verify


install:
- cd jdbc-adapter/virtualschema-jdbc-adapter
- travis_retry mvn dependency:resolve
- cd ../..

script:
- ./jdbc-adapter/integration-test-data/run_integration_tests.sh
- cd jdbc-adapter
- integration-test-data/run_integration_tests.sh
- mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install sonar:sonar

cache:
directories:
- $HOME/.m2
- "$HOME/.m2"
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@

SonarCloud results:

[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=alert_status)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=alert_status)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)

[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=security_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=reliability_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=sqale_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=sqale_index)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=security_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)
[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=reliability_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)
[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=sqale_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)
[![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=sqale_index)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)

[![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=code_smells)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=coverage)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtualschema-jdbc-adapter-main&metric=ncloc)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtualschema-jdbc-adapter-main)
[![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=code_smells)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=coverage)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)
[![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)
[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Avirtual-schema-jdbc-adapter&metric=ncloc)](https://sonarcloud.io/dashboard?id=com.exasol%3Avirtual-schema-jdbc-adapter)

# Overview

Expand Down
22 changes: 22 additions & 0 deletions assembly/all-dependencies.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
<id>all-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<!-- There is a bug caused by maven assembly. See http://stackoverflow.com/questions/17265002/hadoop-no-filesystem-for-scheme-file
and https://jira.codehaus.org/browse/MASSEMBLY-209 -->
<containerDescriptorHandlers>
<containerDescriptorHandler>
<handlerName>metaInf-services</handlerName>
</containerDescriptorHandler>
</containerDescriptorHandlers>
<dependencySets>
<dependencySet>
<unpack>true</unpack>
<scope>runtime</scope>
<outputDirectory>/</outputDirectory>
</dependencySet>
</dependencySets>
</assembly>
43 changes: 20 additions & 23 deletions doc/development/developing-sql-dialect/developing_a_dialect.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,28 +53,25 @@ If you want to write an SQL dialect, you need to start by implementing the diale

### Project Structure

This repository contains Maven sub-projects that are structured as follows.
This repository is structured as follows.

jdbc-adapter Parent project and integration test framework
virtual-schemas
|
|-- virtualschema-jdbc-adapter The actual implementation files
|
|-- src
| |
| |-- src
| |-- main
| | |
| | |-- main
| | | |
| | | |-- java Productive code
| | | |
| | | '-- resources Productive resources (e.g. service loader configuration)
| | |-- java Productive code
| | |
| | '-- test
| | |
| | |-- java Unit and integration tests
| | |
| | '-- resources Test resources
| | '-- resources Productive resources (e.g. service loader configuration)
| |
| '-- test
| |
| |-- java Unit and integration tests
| |
| '-- resources Test resources
| ...
|
'-- virtualschema-jdbc-adapter-dist Environment for creating the all-in-one adapter JAR

### Package Structure

Expand All @@ -98,18 +95,18 @@ The Java package structure of the `virtualschema-jdbc-adapter` reflects the sepa

| Interface | Implementation | Purpose |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------|----------------------------------------------------------------------------------------|
| [`com.exasol.adapter.dialects.SqlDialect`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/java/com/exasol/adapter/dialects/SqlDialect.java) | mandatory | Define capabilities and which kind of support the dialect has for catalogs and schemas |
| [`com.exasol.adapter.dialects.SqlDialectFactory`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/java/com/exasol/adapter/dialects/SqlDialectFactory.java) | mandatory | Provide a way to instantiate the SQL dialect |
| [`com.exasol.adapter.jdbc.RemoteMetadataReader`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/java/com/exasol/adapter/jdbc/RemoteMetadataReader.java) | optional depending on dialect | Read top-level metadata and find remote tables |
| [`com.exasol.adapter.jdbc.TableMetadataReader`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/java/com/exasol/adapter/jdbc/TableMetadataReader.java) | optional depending on dialect | Decide which tables should be mapped and map data on table level |
| [`com.exasol.adapter.jdbc.ColumnMetadataReader`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/java/com/exasol/adapter/jdbc/ColumnMetadataReader.java) | optional depending on dialect | Map data on column level |
| [`com.exasol.adapter.dialects.QueryRewriter`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/java/com/exasol/adapter/dialects/QueryRewriter.java) | optional depending on dialect | Rewrite the original query into a dialect-specific one |
| [`com.exasol.adapter.dialects.SqlDialect`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/dialects/SqlDialect.java) | mandatory | Define capabilities and which kind of support the dialect has for catalogs and schemas |
| [`com.exasol.adapter.dialects.SqlDialectFactory`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/dialects/SqlDialectFactory.java) | mandatory | Provide a way to instantiate the SQL dialect |
| [`com.exasol.adapter.jdbc.RemoteMetadataReader`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/jdbc/RemoteMetadataReader.java) | optional depending on dialect | Read top-level metadata and find remote tables |
| [`com.exasol.adapter.jdbc.TableMetadataReader`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/jdbc/TableMetadataReader.java) | optional depending on dialect | Decide which tables should be mapped and map data on table level |
| [`com.exasol.adapter.jdbc.ColumnMetadataReader`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/jdbc/ColumnMetadataReader.java) | optional depending on dialect | Map data on column level |
| [`com.exasol.adapter.dialects.QueryRewriter`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/dialects/QueryRewriter.java) | optional depending on dialect | Rewrite the original query into a dialect-specific one |

### Registering the Dialect

The Virtual Schema adapter creates an instance of an SQL dialect on demand. You can pick any dialect that is listed in the `SqlDialects` registry. Each dialect needs a factory that can create an instance of that dialect. That factory must implement the interface 'SqlDialectFactory'.

We use Java's [Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) in order to load the dialect implementation. That means you need to register the factory of your new dialect as a service on the list in [`com.exasol.adapter.dialects.SqlDialectFactory`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/resources/META-INF/services/com.exasol.adapter.dialects.SqlDialectFactory).
We use Java's [Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) in order to load the dialect implementation. That means you need to register the factory of your new dialect as a service on the list in [`com.exasol.adapter.dialects.SqlDialectFactory`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/dialects/SqlDialectFactory.java).

```properties
com.exasol.adapter.dialects.athena.AthenaSqlDialectFactory
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@ And we also need two corresponding test classes:

Each dialect is accompanied by a factory that is responsible for instantiating that dialect.
The [Java Service](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) loader takes care of finding and loading the factory.
It looks up the fully qualified class name of the dialect factories in the file [`com.exasol.adapter.dialects.SqlDialectFactory`](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/main/resources/META-INF/services/com.exasol.adapter.dialects.SqlDialectFactory).
It looks up the fully qualified class name of the dialect factories in the file [`com.exasol.adapter.dialects.SqlDialectFactory`](https://github.com/exasol/virtual-schema-common-jdbc/blob/master/src/main/java/com/exasol/adapter/dialects/SqlDialectFactory.java).

1. Now **create a class for the factory**: `com.exasol.adapter.dialects.athena.AthenaSqlDialectFactory` that **implements** `SqlDialectFactory`.
Let your IDE to generate necessary **overriding methods** for you.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ What you need is, for each source database:

## Preparing Integration Test

1. In order to run the integration test automated, edit the [Travis CI integration test configuration file](../../../jdbc-adapter/integration-test-data/integration-test-travis.yaml) and add your new database.
1. In order to run the integration test automated, edit the [Travis CI integration test configuration file](../../../integration-test-data/integration-test-travis.yaml) and add your new database.
2. Provide a JDBC driver JAR for the source database.
3. Add a new Integration Test class for you database

Expand All @@ -43,21 +43,21 @@ Set the following properties for your database:

### Provide JDBC drivers for the Source Database

The JDBC drivers are automatically deployed during the test. You have to create a directory for the jdbc driver under integration-test-data/drivers. The folder contains the driver jar file(s) and a config file. See the [PostgreSQL config](../../../jdbc-adapter/integration-test-data/drivers/POSTGRESQL/settings.cfg) for an example.
The JDBC drivers are automatically deployed during the test. You have to create a directory for the jdbc driver under integration-test-data/drivers. The folder contains the driver jar file(s) and a config file. See the [PostgreSQL config](../../../integration-test-data/drivers/POSTGRESQL/settings.cfg) for an example.

In order to connect to the source database from your integration test you also have to add the jdbc driver dependency to the [POM](../../../jdbc-adapter/virtualschema-jdbc-adapter/pom.xml) scope verify.
In order to connect to the source database from your integration test you also have to add the jdbc driver dependency to the [POM](../../../pom.xml) scope verify.

### Add a new Integration Test Class

Add a new class that derives from [AbstractIntegrationTest](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/test/java/com/exasol/adapter/dialects/AbstractIntegrationTest.java). This class has to:
Add a new class that derives from [AbstractIntegrationTest](../../../src/test/java/com/exasol/adapter/dialects/AbstractIntegrationTest.java). This class has to:
* Create the test schema in the source database
* Create the virtual schema
* Execute the tests on the virtual schema
See [PostgreSQLDialectIT](../../../jdbc-adapter/virtualschema-jdbc-adapter/src/test/java/com/exasol/adapter/dialects/postgresql/PostgreSQLSqlDialectIT.java) for an example.
See [PostgreSQLDialectIT](../../../src/test/java/com/exasol/adapter/dialects/postgresql/PostgreSQLSqlDialectIT.java) for an example.

## Executing Integration Tests

Executing the integration test is easy, just run the [integration test bash script](../../../jdbc-adapter/integration-test-data/run_integration_tests.sh)
Executing the integration test is easy, just run the [integration test bash script](../../../integration-test-data/run_integration_tests.sh)

# Integration Testing Against a Local Database

Expand Down Expand Up @@ -96,7 +96,7 @@ If BucketFS is new to you, there are nice [training videos on BucketFS](https://

1. Create a dedicated user in the source database that has the necessary access privileges
2. Create credentials for the user under which the integration tests run at the source
3. Make a local copy of the [sample integration test configuration file](../../../jdbc-adapter/integration-test-data/integration-test-sample.yaml) in a place where you don't accidentally check this file in.
3. Make a local copy of the [sample integration test configuration file](../../../integration-test-data/integration-test-sample.yaml) in a place where you don't accidentally check this file in.
4. Edit the credentials information
5. [Deploy the JDBC driver(s)](../../user-guide/deploying_the_virtual_schema_adapter.md#deploying-jdbc-driver-files) to the prepared bucket in Exasol's BucketFS

Expand Down
2 changes: 1 addition & 1 deletion doc/development/remote_debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ You can see the filled out configuration dialog below.

![Eclipse remote debugging settings](../images/Screenshot_Eclipse_remote_debugging.png)

You can find the example configuration above in the collection of launch configurations under [`jdbc-adapter/launch`](../../jdbc-adapter/launch/virtual-schema%20debug%20listen%20on%20port%208000.launch).
You can find the example configuration above in the collection of launch configurations under [`jdbc-adapter/launch`](../../launch/virtual-schema%20debug%20listen%20on%20port%208000.launch).

## Configuring the Virtual Schema for Remote Debugging

Expand Down
2 changes: 1 addition & 1 deletion doc/dialects/athena.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The SQL statement below creates the adapter script, defines the Java class that
```sql
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.JDBC_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/<BFS service>/<bucket>/virtualschema-jdbc-adapter-dist-3.0.0.jar;
%jar /buckets/<BFS service>/<bucket>/virtualschema-jdbc-adapter-dist-3.0.1.jar;
%jar /buckets/<BFS service>/<bucket>/AthenaJDBC42-<JDBC driver version>.jar;
/
;
Expand Down
2 changes: 1 addition & 1 deletion doc/dialects/aurora.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ The SQL statement below creates the adapter script, defines the Java class that
```sql
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.JDBC_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/<BFS service>/<bucket>/virtualschema-jdbc-adapter-dist-3.0.0.jar;
%jar /buckets/<BFS service>/<bucket>/virtualschema-jdbc-adapter-dist-3.0.1.jar;
%jar /buckets/<BFS service>/<bucket>/postgresql-<JDBC driver version>.jar;
/
```
Expand Down
Loading

0 comments on commit f37bbae

Please sign in to comment.