How do you secure your data in memory?

October 19, 2015

Cyber security is costing Britain’s businesses an estimated £34 billion a year as they seek to repair damage caused by security breaches and to beef up security to prevent attacks. This latest research by the Centre for Economics and Business Research and US application security firm Veracode found that just over half of the total cost comes from lost revenues from successful attacks, with the remaining £16 billion representing companies’ preventative spending.

While the idea of such attacks is not new, the stakes are rising as more and more financial transactions move away from traditional routes of commerce to digital formats. As a result, the many of layers of security and access, including special servers and encryption, put in place over the past decade to prevent database breaches are no longer adequate to police the in-memory space. And with more and more data moving from databases to in-memory, it is now time to consider what options are available to protect that data from unwanted access and provide accurate audit data in the event that it is accessed.

In the on-line trading world data needs to be accessed securely at low and predictable latencies. To achieve these low latencies more and more systems are opting for in-memory data grids (IMDG) instead of more traditional relational databases (RDBMS) that store and query data from disk. Clients need to be able to execute trades on distributed eTrading platforms and banks need to ensure that the transaction is accurately executed after appropriate checks are performed. To enable an eTrading application to do this, it needs to store details ranging from credit limits and pricing, through to sensitive client details and authentication within the same memory. This poses a challenge around securing the data and ensuring it isn’t accessed or modified maliciously or otherwise, except for by those with the required authorisation.

Most IMDG products have security integration features. However, a lot of business applications built on IMDG still rely on simple home-grown user/password tables stored in clear text and without encryption. Another issue is that many of these home-grown security solutions secure just the application API layer and not the underlying IMDG, making it relatively easy in these cases to hack directly against the IMDG. The lack of corporate security integration in these systems could be due to the fact that IMDG infrastructure is still very much controlled by the development teams. Many organizations and their central operations teams still don’t understand  or control central IMDG infrastructure, in the same way we might see a central DBA team control production databases.

Let’s take a look at the various layers of security available within IMDG solutions, using a look at a concrete set of examples from the market-leading open source IMDG, Hazelcast (

Firstly, as an IMDG is a distributed system we need to make sure that all communications between processes over the network are secure.  Most IMDG products provide Transport Layer Security (TLS) that encrypts this on the wire communication between cluster members.  Hazelcast provides the ability to secure network traffic using TLS between cluster members, from client to cluster and finally between interconnected clusters over wide area networks.

The second layer we’ll describe as “process security”.  Unlike traditional RDBMS which remain static, it is common for processes to be started and stopped frequently within a production cluster. This is how an IMDG provides elastic scalability. By adding extra processes into the running cluster we can increase the memory and computer power available. This in itself presents a stability problem and in this environment we need to ensure that rogue processes do not accidentally or deliberately join the wrong cluster. It would be potentially disastrous for a production cluster to have a process from a developers test cluster join together. This is an accidental issue, but it would be just as easy for a hacker to insert another process into the cluster and potentially read sensitive data. To prevent this we need “process security”, which usually takes the form of a mechanism that allows the cluster to check any new process joining the cluster.  The check is entirely bespoke and would usually take the form of exchanging secure certificates.  Hazelcast provides this feature in the form of a “SocketInterceptor” callback, within which the developer can code their security check.

With communication and process security in place, we can now start to think about how to secure the data and also how we audit data access.  It is at this level of security that we need to integrate with Corporate Information Security (CIS) systems.  These CIS systems usually store application security meta-data backed by data stores such as LDAP or Active Directory. It would usually be the responsibility of the CIS system to authenticate a user and then to provide a set of roles for that user against a business system.  Hazelcast has the ability to integrate with CIS systems via its JAAS (Java Authentication and Authorization Service) framework.  JAAS is a standard framework that can be used to authenticate and authorise a user’s session against the IMDG platform.  When a client of the IMDG platform makes an initial connection attempt, Hazelcast can take authentication tokens from the client connect and pass these to the CIS system.  Once authorised as a valid user, the CIS system would then return a set of roles back to Hazelcast.  These roles map to a set of data structures and allowed operations within Hazelcast, for example an “admin” role may allow updates to certain trade or counter-party data whilst a “read-only” role will, as the name suggests, allow only reads.

As we can see, this check provides a security barrier against entire data structures in memory based on client membership of a role.  But what if we want to allow access to just a few elements or rows of a data structure?  For this we’ll need a more finely grained set of security meta-data and checks.  Some, but not all IMDG products provide this entry or row level security check.  Hazelcast provides a callback mechanism called a “SecurityInterceptor”.  This provides the application developer with the relevant information required to make a data entry access decision. On request, the pre-authorised client credentials along with the data row requested can be verified in real time against the CIS system. The CIS system would store finer business meta-data to determine access rights to an entry.  For example, using an inclusion or exclusion mask a client could be allowed access to certain counter-party entries and not others.

Finally, we are discovering an ever increasing requirement for data access audit within systems. This presents challenges to latency requirements for many systems as auditing access should not slow down general operations within the system.  Hazelcast again provides a callback mechanism which allows bespoke integration, which can take the form of connections to a central audit system or at its simplest form writing out to a file system.

Become a Brickendon Change Leader

What can we help you achieve?