Whole-system view of railways from Resonate on Redis offers progress

by Joseph K. Clark

The railway industry is moving from moving metal to taking a whole-system view that also considers passengers. Although not as well-known as its principal customer, Network Rail, Resonate, a software supplier to the railways, plays a role in that transition, says Daren Wood, its vision director.

And Jon Gregory, the application architect, and his technical team deliver an essential piece. They have built an event-driven microservice application with Redis Enterprise at its core, from key-value, in-memory open-source database supplier Redis Labs – now called Redis. The application uses Redis Streams and Java Microservices to provide real-time processing of train movements to a browser-based client using SSE connections to push the updates.

Resonate provides traffic management technology for the railways. Wood says its technology is “all about how to operate the railways more efficiently and deliver better customer service”.

He adds: “We deliver systems that take data on planned activity and real-time activity on the network, present it to Network Rail’s operational teams, so they can make better decisions, improve the quality of their plans, [in the context of] a lot of changes.”

Wood says the railway industry is very fragmented, with the train operators on the one hand and Network Rail on the other. “We’re doing work over this year to join up systems across the industry,” he says. “For the first time, some of these people will have shared views of all those resources – tracks, rolling stock, crews. The project began in March 2020 and is ongoing.”

Redis offers progress

As the lead application architect, Gregory, says: “We receive data from many different systems that weren’t always designed to work together, so we need to combine it before we can present it to the users [at Network Rail].

“That drove the need for real-time, in-memory processing. But alongside that, we also needed the system to be hosted anywhere, on-premise, and [with AWS] in the cloud. We needed high availability and replication, where Redis stands out against many other in-memory databases.”

He says The project required real-time processing and the ability to “go back and replay the data, replay multiple streams of that data so you can compare different scenarios on the railways”.

“And so we are using Redis Streams because that creates an immutable record of events or things that happened. As we were going down that path, we realized that with Redis Enterprise, we have one tool that solves many problems: streaming, database performance, replication, and a database that we could install anywhere. We have now a system where we’ve got not only the real-time view of what’s happened on the rail infrastructure but also what has happened in the last 30 days.” The technical team also looked at [messaging system] Kafka but decided it was “not the right fit for this problem, mainly because of the complexity”.

Gregory said the ability to keep the system compatible with running on-premise was also important. “We have a lot of data on the railways. And if you want to save 30 days available for replay, you would soon get into a difficult situation [with public cloud providers] because you can only hold 10 or 14 days within a cloud provider. After all, they apply restrictions.

“We were trying to avoid a situation where we put a ceiling or constraint on the system. If you were to build a system dependent on Azure or AWS, providing 30 days on-stream, but then you’re too small to influence their decision if they drop down to 15 days, it would essentially break your system.”

The system being replaced is about 25 years old. Wood says: “The first phase of this project is the more or less like-for-like replacement in terms of functionality for the visibility that they’ve currently got across their network. But with the challenges the rail industry’s got emerging from the pandemic, there is that need to be more flexible and get more out of the [rail] network, responding to demand.

“So, we’ve put in place a whole set of infrastructure in the broadest sense that can enable all sorts of reuse of these data streams. We’re just starting with this relatively simple visualization of what’s going on in the network, but we can now build in other data streams and a much more complete picture of what the real-time railway is doing. Potentially, we can start building in all sorts of predictive analytics and other tooling.”

Wood adds: “I think in the short term, these products are all about improving the efficiency of the operation, improving customer service, particularly in terms of train performance. But then there is the next evolution in building a set of tools fit for a 21st-century railway, to be slightly corny about it.

“There is a big role for technology to support that refocusing away from managing the physical set of assets to managing customer service. And Redis and AWS are the key strategic technologies supporting this platform.”

Related Posts