In my last article I laid out common challenges with integration testing. As promised I will now demonstrate how to make your life easier when running integration tests against Exasol’s analytics database with the help of Exasol Testcontainers.
The Testcontainers Framework
Integration regularly requires you to set up services that you need to test against. And those services need something to run on. Exasol, for example, requires a Linux kernel and a good number of system tools as a basis for providing the actual database service. Obviously, you wouldn’t want to setup a bare-metal cluster just to run functional integration tests. It’s cheaper and more convenient to run a single-node Exasol instance inside a Docker container on your development machine or Continuous Integration (CI) server.
And as for functional testing, a single node is enough. Performance testing is another story, but that’s outside of the scope of this blog.
Testcontainers is an OpenSource Framework that reduces the typical boilerplate code necessary to set up an integration test environment to a handful of commands. And Exasol Testcontainers is an extension that adds Exasol-specific functions.
You may like -> Online reservation system development
What do Exasol Testcontainers actually do?
An Exasol test container:
downloads Exasol Docker images onto the test machine
starts the Docker container
waits for the required services to be fully initialized
provides a zero-config database connection for setting up the test database structure and content
lets you create additional connections with other user accounts
reads and parses the cluster configuration so you can access it via objects
provides convenience methods for accessing buckets
gives you access to the cluster logs
Different Testcontainer Ports
The most mature implementation of Testcontainers is the original Java Version. This is also the one we give an Exasol extension to.
We would love to hear from you if you are interested in an extension for one of the ports – like Python Testcontainers, for example. Get in touch by adding a comment on this post or with our contact us form.
Getting the Exasol Testcontainers package
An all-in-one package, including the JDBC driver, is available on the Central Repository (fka. “Maven Central”).
The snippet from the pom.xml below demonstrates how to add the dependency com.exasol/exasol-testcontainers.
<dependency>
<groupId>com.exasol</groupId>
<artifactId>exasol-testcontainers</artifactId>
<version>1.3.0</version>
<scope>test</scope>
</dependency>
As always, please check for the latest available version when you start using this library.
Working with Exasol Testcontainers
In this example I will walk you through setting up and using the Exasol Testcontainer together with JUnit 5. If you haven’t already, check out my post on the benefits of JUnit 5.
At the time of writing this blog, the testcontianers.org website lists integration with the Java test frameworks JUnit 4, JUnit 5 and Spock.
Adding JUnit 5 dependencies
Here’s another snippet from a pom.xml that adds the JUnit part of the test environment to your setup. Again, please check for the latest versions.
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>1.12.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.4.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-runner</artifactId>
<version>1.4.2</version>
<scope>test</scope>
</dependency>
Privileged mode
The Exasol Docker container requires you to run it in ‘privileged mode’, so you need to make sure your Docker environment allow this. In case you work on a machine where the Docker doesn’t support ‘privileged mode’, you can still set up a virtual machine (e.g. with qemu/kvm or VirtualBox) and install the Docker there, allowing privileged mode.
Instantiating the container
Before you can use the Exasol test container you first need to instantiate it. And this is where you can see for the first time just how much of the boilerplate code that you would normally write, isn’t needed.
Let’s take a look at the absolute minimum that you need to get a container for your tests up and running:
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
@Testcontainers
class ExasolContainerTest {
@Container
private static ExasolContainer<? extends ExasolContainer<?>> container = new ExasolContainer<>(
ExasolContainerConstants.EXASOL_DOCKER_IMAGE_REFERENCE);
}
As you can see, you need just two annotations, and one assignment to get a Docker container with Exasol installed, set up and ready-to-use for integration testing. And I think you’ll agree, with the amount of time this will save you, that’s not too shabby, eh?
Please note, you can decide whether you want to assign the container reference to a static variable or an instance variable. Depending on your choice the testcontainers framework either runs a fresh container for every test case or shares one for all test cases in a class. Given that a typical up-start time for an Exasol container is around two minutes, a static variable is probably the better compromise.
I would also recommend adding a logger that follows the container log during instantiation, so that you can see what’s going on.
// ...
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Testcontainers
class ExasolContainerTest {
private static final Logger LOGGER = LoggerFactory.getLogger(ExasolContainerTest.class);
@Container
private static ExasolContainer<? extends ExasolContainer<?>> container = new ExasolContainer<>(
ExasolContainerConstants.EXASOL_DOCKER_IMAGE_REFERENCE)
.withLogConsumer(new Slf4jLogConsumer(LOGGER));
}
Accessing the cluster configuration
It’s helpful to know how the cluster is configured in your integration tests. And since parsing the cluster configuration file would be boilerplate code again, we’ve taken that burden off your shoulders too.
In the example below, you can see how to get the HTTP port of the default BucketFS service from the cluster configuration.
final String serviceName = "bfsdefault";
final ClusterConfiguration configuration = container.getClusterConfiguration();
final int port = configuration.getBucketFsServiceConfiguration(serviceName).getHttpPort();
The method getClusterConfiguration() gives you a ClusterConfiguration object that contains the configuration of the Exasol cluster as a hierarchically structured tree of objects.
Getting database connections
Arguably the most important feature of the Exasol test container is that you can create JDBC connections without a lot of fuss. A default connection is always available since the test container logic internally needs it to check whether the container is ready to be used. You can either get a fresh connection for the default user with createConnection() or you can get one for a user of your choice.
In the example below I demonstrate how to get a connection for the user TONY.
try (final Connection connection = container.createConnectionForUser(
"TONY", "Tony's super secret passphrase")) {
// ...
}
The ability to get connections for multiple users is important in cases where you run to integration test case that verify access restricted actions.
Working with BucketFS
Bucket access directly via the test container comes in really handy, especially if you plan to integration test User Defined Functions (UDFs). For these, you often need libraries in your UDFs that you then need to upload into a bucket before testing the UDF itself.
Here are some methods for typical BucketFS tasks that you encounter during integration testing:
listContents() and listContents(String path) — read the contents of a bucket or a “path” in the bucket — we will discuss later why path is set in quotes here
uploadFile(Path localPath, String pathInBucket) — upload a file from a locally accessible file system into a bucket
uploadStringContent(String content, String pathInBucket) — upload the contents of a string to a text file in a bucket
For your convenience the list methods simulate how paths usually work on a file system. In Exasol’s analytics database up to 6.2 objects in a bucket aren’t really organized in paths. But they can contain slashes in their names which at first glance looks similar. The test containers abstract that behaviour so that the list methods work like most users would expect it.
If you are wondering why there is a method called uploadString(...), that is a convenience method that allows you to get dynamically created configuration files into a bucket. If your UDF can be parameterized through configuration in buckets or needs credentials, you can create those on-the-fly in your test and put it into the bucket.
Here’s an example of uploading string contents to the default bucket.
final String content = "Hello BucketFS!";
final String pathInBucket = "string-uploaded.txt";
container.getDefaultBucket().uploadStringContent(content, pathInBucket);
Conclusion
If you plan to run functional integration tests against Exasol’s analytics database, we recommend using Exasol Testcontainers. You save a lot of effort, get cleaner test code and you can concentrate on writing the actual test cases. We currently offer a test container for Java. If you are interested in test containers for other languages, please get in touch
Read more:
Comentarios