SAP HANA Native Storage Extension (NSE) Q&A: Questions and Answers

by | 6 February 2020

What is the SAP HANA Native Storage Extension (NSE)?

 

SAP HANA Native Storage Extension (NSE) is a storage technology that integrates the following media: Disk / Flash Drive / In-Memory. This optimises the use of in-memory while maintaining integrated management in HANA. As described below, this corresponds to "Warm Data" storage. Data that is accessed less frequently, and mainly read, can be stored without being loaded completely into memory.

https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.04/en-US/4efaa94f8057425c8c7021da6fc2ddf5.html

Figure 1: NSE and Buffer Cache

 

Any "Warm" data stored on disk via the NSE does not require "In-Memory" and is therefore not subject to licensing. However, in order to be used, this "Warm" data goes into the "Buffer Cache", which is an allocation of "In-Memory" space of around 10% of the "In-Memory" size of the HANA database. As we will see later, this 10% is not used exclusively for the Buffer Cache if the NSE data is not queried.

Some elements found in the document below attest that the consumption of In-Memory in the context of NSE data is limited to the dedicated space in the Buffer Cache: http: //www.vldb.org/pvldb/vol12/p2047-sherkat.pdf

6. HANA Buffer Cache: With this purpose, we have introduced a buffer cache which is maintained separately from the resource manager to manage the page resources within a memory limit.
6.2 Hot Buffer Retention and Page Stealing: Although the goal is to keep the memory footprint low by limiting usage through the limited buffer cache, the overall performance of the system should remain on par with that of the in-memory system. To achieve this, it is essential to reduce I/O by retaining frequently used buffers while maintaining enough free buffers for future usage
6.4 Out-of-buffers and Emergency Pool: Queries fail if the buffer cache fails to find a free buffer despite a reasonable number of retries. In this case, the user is alerted to increase the buffer caches capacity. But some critical tasks like undo of a transaction, crash recovery, or continuous log replay on a secondary site, cannot fail as their failure would leave the database in an inconsistent state.

 

What applications are compatible with the NSE?

 

This new "Warm" data management mode is applicable in the following cases:

1. HANA Native (using the base without motorising an application)

  • HANA 2.0 SP4 and later

2. HANA when it powers the following applications:

  • S/4HANA
  • BW/4HANA

What are the different types of storage in HANA?

 

  • "Hot Data

This type of data is used for: critical operational activities, real time, and analytics. The data is always stored in HANA memory for best performance. The cost in terms of TCO is the highest

  • "Warm Data

This refers to data that is only read infrequently. It is not necessary to store this data permanently in the HANA memory, but it is accessed in an integrated and transparent manner with the hot data via the buffer cache. The TCO is optimised by the use of disk.

  • "Cold Data

Only for read data, very infrequent access, with storage that is not integrated with the HANA database but accessible via HANA's federated database capabilities.

Figure2: SAP HANA Tiering

 

What are the benefits in terms of TCO?

 

In-memory storage is expensive because of the following points:

  • Licences: if you operate a Full-Use database for which the licence cost is directly linked to the size of the In-Memory database (only). The purchase of the licences is a CAPEX cost (once only) for 64Gb (size of a standard HANA unit). Then annually, maintenance is due, between 17 and 22% of the acquisition cost.

 

  • Managed services costs: Servers have to respect a ratio between CPU and amount of "In-Memory" memory and are configured with very high specifications, so the increase of the server's memory leads to a mechanical increase of the server's monthly cost.

So data storage mechanisms that use a technology other than In-Memory, even partially such as NSE, bring a TCO gain in terms of licences and management costs.

 

Does the implementation of the NSE have an impact on my architecture?

 

No. And this is an excellent advantage of this technology over other technologies such as Extended Nodes.

The growth of a database can be done in two ways:

  • "Scale Out: Adding additional servers that will run in parallel to add storage space and distribute the load. In the past this solution was recommended and brought performance gains, although it comes at a cost as some of the in-memory space is used to run the cluster and so allocation is less efficient. In addition, the hardware cost is much higher and it seems to me that "Scale Out" is no longer recommended by SAP below a certain size limit (4Tb?)

 

  • "Scale Up: Add memory and CPU to the existing server.

Scale Up | Scale Out

https://www.HANAtutorials.com/p/scale-up-or-scale-out-HANA-configuration.html

The advantage of the NSE is that it can and should be configured in "Scale Up", i.e. without having to add several servers, offering TCO optimisation without investment in additional servers.

On the other hand, the implementation of the "Extended Node", which is an older technology, requires at least two nodes to be in place. TCO optimisation when the architecture only includes one HANA node is therefore much more degraded because two additional nodes (1 Worker + 1 Extended Node) must be installed.

 

Is NSE possible when HANA is configured as "Scale Out"?

 

No. NSE cannot be implemented when the architecture is configured as Scale-Out. The use of "Extended Nodes" should be considered in this situation.

 

Does the NSE have an impact on performance?

 

Yes, but only for data that is stored on the NSE. This is because the HANA database only accesses the data stored on disk in the NSE via the Buffer (of 10% of the In-Memory database), and depending on the size of the data being requested, it is necessary to make multiple calls to the data on disk to refresh the buffer.

It is possible to optimize the buffer size by reading the following document: SAP HANA Administration Guide for SAP HANA Platform.

 

Does the implementation of the NSE, and the Buffer Cache reduce my In-Memory Hot Data Allocation?

 

No, but... To function, the NSE requires a buffer that allows the pages stored on disk to be loaded into "In-Memory" memory. By default the buffer cache is activated, whether the NSE is implemented or not. This buffer is sized as 10% of the size of the "In-Memory" memory.

This memory is free when NSE data is not accessed and can be used for other "Hot Data", however when NSE data is accessed, this buffer is reserved for it and HANA will first free up space in this buffer for its operation. (Illustration Figure 1).

https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.04/en-US/8a35ce565c594c11bb785bea607213d8.html

 

Does the NSE have a size limit?

 

No. According to SAP Note 2771956, there is technically no limit to the size of the NSE database:

The NSE data size per HANA system and tenant database is not limited by technical enforcement. Consider the following when storing large data sets in NSE on servers with limited memory capacity

 

However, in the blog (https://blogs.sap.com/2019/07/22/increase-HANA-data-capacity-with-sap-HANA-native-storage-extensionnse) there are recommendations on the ratio between what is stored in NSE and the Buffer with the ratio 1/8. However, I cannot find this information in SAP notes.

If we keep these orders of magnitude, (which may change), i.e. 10% for the Buffer Cache and 1/8 for the NSE, we obtain the following figures from 1TB of In-Memory:

  • In-Memory: 1000 GB
  • Buffer Cache: 100 GB or 10%.
  • SAP NSE: 800 GB or 8x the buffer

Given the limitations that Buffer Cache size can pose, is it worthwhile when you want to store more than 800 GB in the above example to use Dynamic Tiering rather than NSE?
In other words, wouldn't disk access be faster than the NSE optimization mechanism via the Buffer Cache when the usage is expected to be well over 8x buffer?

Is rollback possible when tables are stored in the NSE?

 

Yes, it is always possible to go back when tables are put on the NSE. See SAP Note 2799997

 

For further information

 

The following two tabs change content below.

Jérôme Blanc

After graduating from the Ecole Centrale d'Électronique, Jérôme specialised in SAP BI technologies. Before founding Bilink, Jérôme spent several years in London and Chicago with a company specialising in SAP BI solutions. Jérôme wishes to bring to our firm the best practices that he was able to appreciate during his experiences abroad while ensuring that our own values are respected. Jérôme's role today is to guarantee the smooth running of our clients' BI transformation projects
Share This