Skip to content

ConcurrentCache documentation

Jump to bottom
n-lagomarsini edited this page Jun 10, 2014 · 3 revisions

The concurrent-tile-cache project contains a new implementation of the JAI TileCache class in order to achieve a better concurrency. The project consists of various files:

  • ConcurrentTileCache.java which contains a new java TileCache implementation using the features of Google Guava Cache. (See this link about google guava cache)
  • CachedTileImpl.java which contains a different implementation(from Sun default implementation) of the java interface CachedTile, used by the above cache.
  • ConcurrentCacheTest.java which executes 10 consecutive test using both the default (Sun implementation) or the new cache. At the end of the test,the throughput is shown for every test.
  • ConcurrentTileCacheMultiMap.java which contains a TileCache implementation similar to the ConcurrentTileCache.java but using two maps instead of a single one.
  • CacheCoberturaTest.java is a simple JUnit test-file that recall the ConcurrentCacheTest.testwriteImageAndWatchFlag(...) method. It is used for checking the code coverage
  • ConcurrentMultiMapTest is another test class for the ConcurrentTileCacheMultiMap.java class.

In ConcurrentTileCache.java the cache is splitted in segments whose number could be defined by the user. WIth this improvement multiple writing thread can access to the cache in the same time without any thread-safety issue because the cache is locked only per-segment and not globally. Of course if the thread number exceeds the segment number, some writers must wait the other for acquiring the lock. From the readers side there is no need for locking, so multiple readers can simultaneously access the cache. This cache is created using the "CacheBuilder" class present in the guava package. The maximum memory capacity is set at the creation time but can change during the process, in such case the cache is flushed and rebuilt. The "memory control" method is not supported because the guava cache uses an inner eviction method based on the tile size. When the maximum capacity is reached, the LSU tiles are evicted from the cache until the memory usage becames lower than the above threshold. For the same reasons even the "comparator" set and get operations are not supported. If diagnostics is enabled, all the cache statistics are easily available with the "CacheStats" class provided by guava and the cache observer are notified when a tile is accessed.

The CachedTileImpl.java is used by the ConcurrentTileCache for storing or retrieving the tile data information and updating the tile status for the observers. The tile key is calculated from the tile indices and from the reference.

ConcurrentTileCacheMultiMap.java is similar to the ConcurrentTileCache but also uses an internal MultiMap for storing the keys of the tiles for each image in a separate Collection in order to improve the TileCache scalability when accessing all the tiles associated to an image.

CacheCoberturaTest permits to test only a single type of TileCache defined by the cacheUsed value.

For testing the various caches you should set this parameters(or use the default):

  • cacheUsed: an Integer that permits to choose which cache you want to use. 0 means that SunTileCache is used. 1 means that ConcurrentTileCache is used.
  • concurrencyLevel: if you are using the ConcurrentXXXCache you have to set the concurrency level of the cache. This parameter change the internal partition of the cache. It must be > 0. Default value is 4.
  • memoryCacheCapacity: this parameter set the cache maximum memory capacity. It must be > 0. Default value is 128 Mb.
  • multipleOp: this boolean allows to have multiple tiles requests on the image per thread. Default value is false. _For now, it slows the test so should not be set to true. _
  • syntheticImage: this boolean permits to choose if you want to use a synthetic image or not. By default the test uses a synthetic image.
  • diagnosticEnabled: this boolean set the cache diagnostic to save statistics about the cache work.
  • path: this last parameter is used if the syntheticImage boolean is false and indicates the real image path to use.

The test file starts his work setting the new tile cache and its concurrency level, if cacheUsed is not 0, or leaving the default cache. Then it sets the memory capacity of the cache and his parallelism(for the inner tile scheduler). After this, it creates a new thread pool for executing the multiple requests to the cache. For using a bigger image, 2 scaling operation are then performed to the initial image. At this point the test enters in a cycle that doubles the number of working threads for every iteration. If the cycle is in the first iteration, a starting thread performs 1000 requests to the cache for improving hotspot code compilation. In this phase another thread check ìs the current memory usage and log the value for testing the functionality of the getCacheMemoryUsage() method. After that, the test starts creating the threads and waits them finishing their work in every cycle. At the end of the single cycle the througput is calculated as total requests executed/sum of all the thread computation time. All the throughput results are stored in a double array and then printed to screen. If multipleOp is set to true the througput is not calculated because it can vary with the number of tiles in cache.

The total request number changes with the number of threads in this way(if multipleOp is set to false):

  • 100 request for 1 thread
  • 200 request for 2 thread(100 request per thread)
  • 400 request for 4 thread(100 request per thread)
  • 400 request for 8 thread(50 request per thread)
  • 400 request for 16 thread(25 request per thread)
  • 400 request for 32 thread(about 13 request per thread)
  • 400 request for 64 thread(about 6 request per thread)

All the results are logged in a file called "ConcurrentCacheTestLog.txt" saved in the "src/test/resources/it/geosolutions/logfiles/" path.

WARNING: This test doesn't use the medialib native acceleration library for elaborating the images and so when the test starts it throws an ClassNotFoundException but the code doesn't stop so this exception can be ignored.

Test results and graph:

ConcurrentMultiMapTest is a simple class which ensures that the ConcurrentTileCacheMultiMap class behaves correctly as cache.

For comparing the performances between the SunTileCache and the ConcurrentTileCacheMultiMap class, we have done various stress tests on GeoServer. For each test we measured the throughput with JMeter software tool. The results of the tests can be seen in the document below:[ConcurrentTileCache with MultiMap Vs SunTileCache] (https://docs.google.com/spreadsheets/d/1Pj9BTlvtBB12ygO8A6nBlY8EWorDmjw_S8NFzMa5NzA/edit?usp=sharing)

Clone this wiki locally