Skip to content

[DRAFT] Adjust timeout behavior of doctests #431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 13 additions & 11 deletions src/crate/client/http.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,42 +22,44 @@ is used::

When using a list of servers, the servers are selected by round-robin::

>>> invalid_host = "invalid_host:9999"
>>> even_more_invalid_host = "even_more_invalid_host:9999"
>>> http_client = HttpClient([crate_host, invalid_host, even_more_invalid_host])
>>> invalid_host = "198.51.100.0:9999"
>>> even_more_invalid_host = "198.51.100.1:9999"
>>> http_client = HttpClient([crate_host, invalid_host, even_more_invalid_host], timeout=0.3)
>>> http_client._get_server()
'127.0.0.1:9295'

>>> http_client._get_server()
'invalid_host:9999'
'198.51.100.0:9999'

>>> http_client._get_server()
'even_more_invalid_host:9999'
'198.51.100.1:9999'

Servers with connection errors will be removed from the active server list::

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host])
>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host], timeout=0.3)
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers
['127.0.0.1:9295']

Inactive servers will be re-added after a given time interval.
To validate this, set the interval very short and sleep for that interval::
To validate this, set the interval and timeout very short and sleep after the first request::

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host], timeout=0.3)
>>> http_client.retry_interval = 1
>>> import time; time.sleep(1)
>>> result = http_client.sql('select name from locations')
>>> import time; time.sleep(1)
>>> server = http_client._get_server()
>>> http_client._active_servers
['invalid_host:9999', 'even_more_invalid_host:9999', '127.0.0.1:9295']
['198.51.100.0:9999', '198.51.100.1:9999', '127.0.0.1:9295']

If no active servers are available and the retry interval is not reached, just use the oldest
inactive one::

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host])
>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host], timeout=0.3)
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers = []
>>> http_client._get_server()
'invalid_host:9999'
'198.51.100.0:9999'


SQL Statements
Expand Down
8 changes: 7 additions & 1 deletion src/crate/testing/layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,19 @@ class CrateLayer(server.ServerLayer, layer.WorkDirectoryLayer):

def __init__(self,
name,
crate_home,
crate_home=None,
crate_config=None,
port=9295,
keepRunning=False,
transport_port=None,
crate_exec=None):
self.keepRunning = keepRunning
if not crate_home:
if not os.environ.has_key('CRATE_HOME'):
raise ValueError("The crate home directory must either be set as the environment"
" variable 'CRATE_HOME' or given as argument 'crate_home'"
)
crate_home = os.environ['CRATE_HOME']
crate_home = os.path.abspath(crate_home)
servers = ['localhost:%s' % port]
self.crate_servers = ['http://localhost:%s' % port]
Expand Down
59 changes: 47 additions & 12 deletions src/crate/testing/layer.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,35 +2,70 @@
Crate Test Layer
================

This layer starts and stops a ``Crate`` instance on a give port and a given crate node name::
This layer starts and stops a ``Crate`` server instance with given node name.

Layer setup
===========

The home directory of the ``Crate`` server is read from the environment variable ``CRATE_HOME``
by default::

>>> from crate.testing.layer import CrateLayer
>>> import random

>>> layer = CrateLayer('crate')

Its also possible to pass the crate home directory via the argument ``crate_home``, this precede
the environment variable::

>>> import os

>>> layer = CrateLayer('crate', os.environ['CRATE_HOME'])

If no ``CRATE_HOME`` environment variable is set and no ``crate_home`` argument is passed, a
``ValueError`` exception is raised::

>>> crate_home_env = os.environ['CRATE_HOME']
>>> del os.environ['CRATE_HOME']
>>> layer = CrateLayer('crate')
Traceback (most recent call last):
...
ValueError: The crate home directory must either be set as the environment variable 'CRATE_HOME' or given as argument 'crate_home'

Re-add the environment variable so following tests will succeed::

>>> os.environ['CRATE_HOME'] = crate_home_env

Using the ``port`` argument a custom port can be defined, otherwise defaults to ``9295``.

>>> port = 9295
>>> layer = CrateLayer('crate', port=port)


>>> layer = CrateLayer('crate',
... crate_home=crate_path(),
... crate_exec=crate_path('bin', 'crate'),
... port=port,)
Startup and Shutdown
====================

Lets start the layer::
They layer starts up using its ``start()`` method::

>>> layer.start()

The layer can be shutdown using its ``stop()`` method::

Now we can access the ``Crate`` instance on the defined port::
>>> layer.stop()

>>> import requests

Accessing the server's HTTP API
===============================

After a successful startup the ``Crate`` instance can be accessed on the defined port::

>>> layer.start()

>>> import requests
>>> stats_uri = "http://127.0.0.1:{0}/_stats".format(port)
>>> response = requests.get(stats_uri)
>>> response.status_code
200


The layer can be shutdown using its ``stop()`` method::

>>> layer.stop()


2 changes: 1 addition & 1 deletion src/crate/testing/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ def crate_path(*parts):


def setUp(test):
test.globs['crate_path'] = crate_path
os.environ['CRATE_HOME'] = crate_path()


def test_suite():
Expand Down