Hugepages in oracle database

HugePages are supported in many Operating systems. We are considering the Linux operating system since it is the most widely used in the market.

Let's understand Hugepages step by step.

I.   What are Hugepages and why they are used

II.  Hugepages advantages

III. Conditions to use Hugepages

IV. How to use them.

I. what are Hugepages

In any operating system, the memory is allocated in term of pages. The normal default page size in Linux is 4KB and these pages are managed by using the page table. so if your oracle database SGA size is say 5gb, it would be allocated in 5gb/4kb = 1.3 million pages, so that means 1.3 million records in the page table to do the mapping. that would an extensive overhead on the OS to manage it.

So to overcome this issue, Hugepages were introduced. They are basically large memory pages with default size of 2MB. so if your database sga size is 5gb, you would be 5gb/2mb=2560 pages. so with this, the OS will need 2560 records in the page table to manage these pages and thus the overhead on the OS will be drastically reduced which will lead to better overall performance.

II. Hugepages advantages

1. Contiguous pages are preallocated and cannot be used for anything else but for System shared memory (for example, SGA)
2. Hugepages are locked in memory and never swapped out, which forces the SGA to always stay in physical memory.
3. Lesser page table entries -> Lesser memory requirements for storing the page table -> lesser overhead on the OS to manage the pages.

III. Conditions to use Hugepages

1. If the SGA size is >= 5GB, it is strongly recommended to configure HugePages at the OS level.

2. Hugepagesize can vary depending on the kernel and hardware architecture of the server.

3. Oracle large pages support began in Oracle 10g Release 1. So Hugepages can be used in Oracle Database 10gR1 and later versions.

4. Automatic Memory Management (AMM) is not compatible with Linux HugePages i.e. MEMORY_TARGET cannot be used with HugePages, Instead, Automatic Shared Memory Management ( sga_target) and Automatic PGA Management ( pga_aggregate_target) should be used as they are compatible with HugePages.


5. Disable Transparent HugePages
Transparent HugePages memory is enabled by default with Red Hat Enterprise Linux 6, Oracle Linux 6 etc. Transparent HugePages memory differs from standard HugePages memory because the kernel khugepaged thread allocates memory dynamically during runtime. Standard HugePages memory is pre-allocated at startup, and does not change during runtime. It degrade the database performance many times so Oracle strongly recommends that you disable Transparent HugePages and use standard Hugepages on all Oracle Database servers.


To check if Transparent HugePages is enabled run one of the following commands as the root user: Red Hat Enterprise Linux kernels:

# cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
Other kernels:
# cat /sys/kernel/mm/transparent_hugepage/enabled

The following is a sample output that shows Transparent HugePages is being used as the [always] flag is enabled.

[always] never

Note:

If Transparent HugePages are not being used then the /sys/kernel/mm/transparent_hugepage or /sys/kernel/mm/redhat_transparent_hugepage files do not exist.

To disable Transparent HugePages perform the following steps. DO THIS STEP ONLY IF YOU FIND TRANSPARENT HUGEPAGES already enabled IN YOUR OS.

Add the following entry to the kernel boot line in the /etc/grub.conf file:

transparent_hugepage=never

IV. How to use them.

1. Check current status of Hugepages. Run the following command to determine the current HugePage usage. 

$ grep Huge /proc/meminfo

AnonHugePages:         0 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

$

Clearly in above output, no value in HugePages_Total, HugePages_Free that means currently hugepages are not being used.


2. Edit /etc/sysctl.conf file.

The below script is provided by Oracle to calculate the value of no of hugepages. Copy the below file onto your server, give it execute permissions.

Make sure that the oracle database is running and AMM is not enabled i.e. MEMORY_TARGET is not being used.

Run this script.

hugepages_setting.sh

It will give the output like below:

$ ./hugepages_setting.sh 

Recommended setting: vm.nr_hugepages = 306

$

Edit the "/etc/sysctl.conf" file from "root" user and add the following entry in it.

vm.nr_hugepages=306

Run the following command as the "root" user.

# sysctl -p

3. Edit /etc/security/limits.conf file. 

Check the following entries into the "/etc/security/limits.conf". 

* soft memlock 626688

* hard memlock 626688

Both the values should be equals to the total Hugepages memory i.e. HugePages * Hugepagesize so in our case it will be 306*2048=626688. If the values are found lower, then edit the file and change the values.


Limits.conf file is used to limits the system resources. The memlock option is used to set the maximum amount of locked-in-memory in kilobytes. This is memory that will not be paged out and always remain present in physical memory. So by putting these values, we are enabling the hugepages to always remain the resident of the physical memory and are never pages out by the OS.

4. Stop all the database services and restart the server.


5. Check the HugePages information again.

$ grep Huge /proc/meminfo

AnonHugePages:         0 kB

HugePages_Total:     306

HugePages_Free:       98

HugePages_Rsvd:       93

HugePages_Surp:        0

Hugepagesize:       2048 kB

$

Clearly, the hugepages are being used at OS level.

 
6. Enable database to use large pages. 

Lets enable hugepages at database level by setting the USE_LARGE_PAGES parameter to ONLY. Note that this parameter was introduces in 11.2.0.2 so please skip this step if your database version is older than 11.2.0.2.

ALTER SYSTEM SET use_large_pages=only SCOPE=SPFILE;

SHUTDOWN IMMEDIATE;

STARTUP;

Below are the available values of this parameter:

Values:

1. TRUE ( it is its DEFAULT value)

In Oracle Database 11.2.0.2, if there are not enough large pages configured on the system, then regular sized pages will be used to allocate SGA memory.

In Oracle Database 11.2.0.3 and later releases, Oracle allocates as much of the SGA as it can in large pages, and if it runs out, it will allocate the rest of the SGA using regular sized pages. In this supported mixed page mode allocation, the database will exhaust the available large pages before switching to regular sized pages.

2. FALSE
Specifies that the instance will not use large pages.

3. ONLY

Specifies that the instance will fail to start if large pages cannot be used for the entire SGA memory.

It is recommended by Oracle to always set this parameter to a value of ONLY if you are going to use large pages (hugepages).