In this article, let’s explore the Hazelcast Ringbuffer with the help of a simple example.
What is Hazelcast Ringbuffer?
Ringbuffer is one of the distributed data structures of Hazelcast. It is a distributed version of a Circular Buffer (or Circular Queue) having a fixed capacity.
Each Ringbuffer has a head and a tail. When we add an entry to the Ringbuffer, it adds the item to the tail and increments it.
On the other hand, the head always points to the oldest entry. Therefore, when we add an entry beyond the capacity, both the tail and the head move ahead.
Moreover, currently, Hazelcast Ringbuffer is not a partitioned data structure; its data is stored in a single partition and the replicas are stored in another partition. Therefore, we should create a Ringbuffer that can safely fit in a single cluster member.
Interestingly, Hazelcast itself uses Ringbuffer as a building block for the Reliable Topic data structure.
The Ringbuffer Example
Let’s consider a simple example to understand the Ringbuffer. We’ll use the Ringbuffer to store the recent searches for an application. Ideally, we should store the recent searches for each user. However, we’ll keep it simple in this example.
We’ll create a simple Spring Boot application. The application will expose an API for search and another API for fetching the recently searched strings. We’ll store the search strings in the Hazelcast Ringbuffer.
For simplicity, let’s say that the capacity of the Ringbuffer is 16 as shown in the figure below.
Since the Ringbuffer has a fixed capacity of 16, it’ll store the first 16 search strings with the head pointing to the first search string and the tell pointing to the 16th search string.
Now, if we add the 17th search string, it’ll override the first search string at the head. Consequently, the head will point to the second search string and the tail to the 17th one.
Hazelcast Ringbuffer in Action
To start with, we’ll create a Spring Boot web application using the Spring Initializr.
Hazelcast Configuration
Second, we’ll add the Hazelcast dependency to the pom.xml:
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-spring</artifactId>
<version>5.3.6</version>
</dependency>Third, let’s create a HazelcastConfig class and define a bean for HazelcastInstance with a minimal configuration for the Hazelcast Ringbuffer:
@Configuration
public class HazelcastConfig {
@Value("${recentSearch.ringBuffer.name}")
private String ringBufferName;
@Value("${recentSearch.ringBuffer.capacity}")
private int ringBufferCapacity;
@Bean
public HazelcastInstance hazelcast() {
Config config = new Config();
// Configure the ring buffer
RingbufferConfig ringbufferConfig = new RingbufferConfig(ringBufferName);
ringbufferConfig.setCapacity(ringBufferCapacity);
config.addRingBufferConfig(ringbufferConfig);
return Hazelcast.newHazelcastInstance(config);
}
}
After that, we’ll define the name and the capacity of the Ringbuffer in application.yml:
recentSearch:
ringBuffer:
name: "recentSearch"
capacity: 16In general, for a production-ready code, we should also consider other configurations of Ringbuffer. Some examples are – backup count, time to live, and split-brain protection.
Developing the Search API
For this example, we’ll implement the following two REST APIs:
/search?query=searchString – The API for search which will save the search string into the Ringbuffer
/search/recent – The API to get the recent search strings
First, we’ll create the SearchController class:
@RestController
@RequestMapping("/search")
public class SearchController {
private final SearchService service;
@Autowired
public SearchController(SearchService service) {
this.service = service;
}
@GetMapping
public String searchForQuery(@RequestParam String query) {
return service.searchForQuery(query);
}
@GetMapping("/recent")
public Collection<String> getAllRecentSearches() {
try {
return service.getAllRecentSearches();
} catch (InterruptedException e) { // The readOne methods of Ringbuffer may throw InterruptedException
throw new RuntimeException(e);
}
}
}
Then, let’s create the SearchService class:
@Service
public class SearchService {
Logger logger = LoggerFactory.getLogger(SearchService.class);
private final HazelcastInstance hzInstance;
private final String ringBufferName;
public SearchService(
HazelcastInstance hzInstance,
@Value("${recentSearch.ringBuffer.name}") String ringBufferName) {
this.hzInstance = hzInstance;
this.ringBufferName = ringBufferName;
}
/**
* Search for the specified query
* @param query the search query
* @return search result
*/
public String searchForQuery(String query) {
Ringbuffer<String> ringBuffer = hzInstance.getRingbuffer(ringBufferName);
long sequence = ringBuffer.add(query);
logger.info("Added query {} to recent search. Sequence {}", query, sequence);
return "You searched for %s".formatted(query);
}
/**
* @return recent search queries based on configured capacity
*/
public Collection<String> getAllRecentSearches() throws InterruptedException {
Ringbuffer<String> ringBuffer = hzInstance.getRingbuffer(ringBufferName);
long headSequence = ringBuffer.headSequence();
long tailSequence = ringBuffer.tailSequence();
logger.info("Head sequence {} Tail sequence {}", headSequence, tailSequence);
List<String> recentSearches = new ArrayList<>();
while(headSequence <= tailSequence){
String searchQuery = ringBuffer.readOne(headSequence);
recentSearches.add(searchQuery);
headSequence++;
}
return recentSearches;
}
}
As we can see, we’ve injected the HazelcastInstance in the SearchService. When we search for a string, the searchForQuery method adds the search query to the Ringbuffer.
On the other hand, the getAllRecentSearches method returns the recent searches. Here, we simply read the head and tail sequences from the Ringbuffer. Then, we iterate and add all the elements to the list of recent searches.
And, that’s all we’ve to do to show the recent searches! Simple, isn’t it?
Testing the Search API
We can run the application at this point and test using a browser or Postman. Also, to see the Ringbuffer in action in a distributed setting, we can run multiple instances of our application (by changing the port).
However, for this example, we’ll create a test to verify our implementation.
So, we’ll create the SearchControllerTest class:
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
class SearchControllerTest {
@LocalServerPort
private int port;
@Autowired
private TestRestTemplate restTemplate;
@Value("${recentSearch.ringBuffer.capacity}")
private int ringBufferCapacity;
@Test
void recentSearchesShouldContainOnlyMaxAllowedResults() {
IntStream
.range(0, 20)
.forEach(i ->
assertThat(
this.restTemplate.getForObject(
"http://localhost:%d/search?query=searchString%d".formatted(port, i),
String.class
)
).contains("You searched for")
);
@SuppressWarnings("unchecked")
Collection<String> recentSearches =
this.restTemplate.getForObject(
"http://localhost:%d/search/recent".formatted(port),
Collection.class
);
assertThat(recentSearches.size()).isEqualTo(ringBufferCapacity);
}
}
Here, we’re calling the search API 20 times. Then, we’re validating that the recent search API returns the results with a size equal to the Ringbuffer capacity, i.e. 16.
Furthermore, we can also check the logs to see what is happening:
Added query searchString0 to recent search. Sequence 0 Added query searchString1 to recent search. Sequence 1 Added query searchString2 to recent search. Sequence 2 Added query searchString3 to recent search. Sequence 3 Added query searchString4 to recent search. Sequence 4 Added query searchString5 to recent search. Sequence 5 Added query searchString6 to recent search. Sequence 6 Added query searchString7 to recent search. Sequence 7 Added query searchString8 to recent search. Sequence 8 Added query searchString9 to recent search. Sequence 9 Added query searchString10 to recent search. Sequence 10 Added query searchString11 to recent search. Sequence 11 Added query searchString12 to recent search. Sequence 12 Added query searchString13 to recent search. Sequence 13 Added query searchString14 to recent search. Sequence 14 Added query searchString15 to recent search. Sequence 15 Added query searchString16 to recent search. Sequence 16 Added query searchString17 to recent search. Sequence 17 Added query searchString18 to recent search. Sequence 18 Added query searchString19 to recent search. Sequence 19 Head sequence 4 Tail sequence 19
Notably, we can see in the logs, the Ringbuffer added the first 16 strings from sequence 0 to sequence 15. Since we’ve called the search API 20 times in our test, the rest 4 elements were added from sequence 0 to 3. So, finally, the head sequence is 4 and the tail sequence is 19.
Summary
In conclusion, we’ve talked about the Hazelcast Ringbuffer in this article. Also, we’ve used the Ringbuffer in a Spring Boot application.
We can also explore some other features of the Ringbuffer like batching for improving performance, asynchronous methods that return the CompletionStage, and a RingBuffer store to back the Ringbuffer by a central data store.
Nice explanation in simple words along with example/sample code. Thanks Kamlesh
Thank you, Umesh. I am glad you liked it.
Thanks for blog Kamlesh and providing with simple exmple. Could you please also create a blog for hbase and how to integrate both?
Thank you Jagan for your comment. I am glad you liked it. I do not have experience working on hbase. However, I will surely check.