Shared memory and semaphores explained
Oracle database uses shared memory for its internal memory structure. This Article describes the Solaris operating system 10 kernel parameters which affects the allocation and usage of shared memory and semaphores. Then an example using Oracle database 10g and Solaris operating system 10 shows how to configure them using projects and same command to trace shared memory allocation.Since Solaris operating system 10 the kernel tunables which where used in /etc/system are obsolete. They are replaced by a resource control. But one can still set the parameters with the old /etc/system tunables. List 1 show the most important parameters with its new resource control and the default value:
| Affects | Resource Control | Obsolete /etc/system tunable | Default value |
| Shared Memory | project.max-shm-memory | shmsys:shminfo_shmmax | 1/4 of physical RAM |
| Shared Memory | project.max-shm-ids | shmsys:shminfo_shmmni | 128 |
| Semaphores | project.max-sem-ids | semsys:seminfo_semmni | 128 |
| Semaphores | process.max-sem-nsems | semsys:seminfo_semmsl | 512 |
| Semaphores | process.max-sem-ops | semsys:seminfo_semopm | 512 |
Shared memory is a facility for programms to share physical RAM between different processes without the need to transport or copy one processes address space to another. To use shared memory the kernel needs to load two modules, the interprocess communication IPC facilities and the shmsys module. You can see them with the following command:
bash:~>modinfo |grep shmsys 133 78344000 2945 52 1 shmsys (System V shared memory) bash:~>modinfo |grep ipc 134 78199c85 3dc - 1 ipc (common ipc code)In Solaris 10 os there are two relevant parameters. They have different meaning depending on if they are used in /etc/system or in a project. In a project project.max-shm-memory configures how much shared memory in sum the programms which run under the project can use. project.max-shm-ids defines how much different shared memory segments can be allocated under the project. When configuring the parameters in /etc/system they apply to every user with or without profile. shmsys:shminfo_shmmax is a upper border for the maximum size of a single shared memory segment and shmsys:shminfo_shmmni is the maximum limit for the number of shared memory segments.
Semaphores
Semaphores are like traffic lights for a shared resource here the shared memory. Only if the light shows green to you the resource can be accessed. Every process which can't access the the shared memory because it is busy has to wait until it is available. It increments and decrements the semaphore value. The unix system V semaphores works with semaphore sets. One set can contain multiple semaphores. How much semaphores they have is determined at creation time. There are comfort functions witch does operations on every semaphore of a semaphore set. The parameter semsys:seminfo_semmni determines the maximum number of semaphore sets system wide. Where every semaphore set can have up to semsys:seminfo_semmsl semaphores. The parameter semsys:seminfo_semopm establishes the maximum number of operations per semaphore set. You should set this parameter equal to semsys:seminfo_semmsl so you can perform operations on every semaphore of a semaphore set. Using the parameters in a project does not restrict them systemwide. They are only restricted for the project they are configured in.
Example with Oracle database 10g running on Solaris OS 10
On a machine with the Solaris operating system we investigate which shared memory is currently allocated and how much physical RAM the machine has:
-bash-3.00$ /usr/sbin/prtconf |grep Mem Memory size: 16376 Megabytes -bash-3.00$ ipcs -pma IPC status from as of Mon Feb 11 21:23:01 CET 2008 T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ Shared Memory: m 1 0x3cb26b68 --rw-r----- oracle oinstall oracle oinstall 26 817897472The first command shows us there are 16 GB physical memory. The second command shows that user oracle has a shared memory segment of 780MB (SEGSZ) most likely a database. So we have enough free physical memory for our example. We try to start an instance with a little bit more than 4GB RAM: 4347483648 bytes. For this test we dont need a full database it is enough to start an instance with the nomount option and Oracle database will allocate the shared memory. To reach this we need a pfile with the following parameters and save it as $ORACLE_HOME/dbs/initoratest1.ora
DB_NAME=oratest1 sga_max_size=4347483648 sga_target=4347483648 # Processes and Sessions PROCESSES=500 sessions=555 compatible='10.2.0.1.0'Now lets try to start the instance with no parameters in /etc/system and no project configured for the user:
-bash-3.00$ export ORACLE_SID=oratest1 -bash-3.00$ sqlplus / as sysdba SQL*Plus: Release 10.2.0.2.0 - Production on Mon Feb 11 21:21:54 2008 Copyright (c) 1982, 2005, Oracle. All Rights Reserved. Connected to an idle instance. SQL> startup nomount ORA-27102: out of memory SVR4 Error: 22: Invalid argument SQL>It can't start despite we have enough physical memory because we don't have enough shared memory configured. As we have no parameters configured in /etc/system and no project for the user the default maximum shared memory for this machine is 16 GB / 4 = 4 GB of shared memory. But our example uses a little more then 4 GB in it's parameter file for the sga_max_size and sga_target parameters. We configure a project so that the user is able to use more shared memory. We first check the current project configuration:
-bash-3.00$ id -p uid=701(oracle) gid=700(oinstall) projid=3(default) -bash-3.00$ cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10::::We can see the user has the default project assigned which in this case has no special configuration for shared memory or semaphores and also in the /etc/system no parameters configured. As root we add a new project for the user oracle with the following command:
-bash-3.00$ projadd -c "Oracles" 'user.oracle'It is important to know the the new projects only affects users which are logged on after the profile is established. So we logout and login again:
-bash-3.00$ id -p uid=701(oracle) gid=700(oinstall) projid=3(default) -bash-3.00$ exit logout -bash-3.00$ su - oracle Password: You have new mail. -bash-3.00$ id -p uid=701(oracle) gid=700(oinstall) projid=100(user.oracle) -bash-3.00$Now as user root we modify the profile so that our test instance is able to start. This modifications are stored in the /etc/projects file. They can survife a restart of the operating system. In this example I configure 8 GB shared memory for the user oracle and more then 3000 semaphores. That would be enough for one big or several smaller databases:
root # projmod -s -K "project.max-shm-memory=(privileged,8GB,deny)" 'user.oracle' root # projmod -s -K "project.max-shm-ids=(privileged,1024,deny)" 'user.oracle' root # projmod -s -K "process.max-sem-nsems=(privileged,3096,deny)" 'user.oracle' root # projmod -s -K "project.max-sem-ids=(privileged,200,deny)" 'user.oracle' root # projmod -s -K "process.max-sem-ops=(privileged,3096,deny)" 'user.oracle'Now we start the instance. Check the allocated shared memory and take a look into the /etc/project file.
-bash-3.00$ su - oracle
Password:
You have new mail.
-bash-3.00$ export ORACLE_SID=oratest1
-bash-3.00$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.2.0 - Production on Mon Feb 11 21:31:58 2008
Copyright (c) 1982, 2005, Oracle. All Rights Reserved.
Connected to an idle instance.
SQL> startup nomount
ORACLE instance started.
Total System Global Area 4362076160 bytes
Fixed Size 2034576 bytes
Variable Size 1006638192 bytes
Database Buffers 3338665984 bytes
Redo Buffers 14737408 bytes
SQL>exit
-bash-3.00$ ipcs -pma
IPC status from as of Mon Feb 11 21:32:46 CET 2008
T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ
Shared Memory:
m 70 0xe8a693dc --rw-r----- oracle oinstall oracle oinstall 13 4362092544
m 1 0x3cb26b68 --rw-r----- oracle oinstall oracle oinstall 26 817897472
-bash-3.00$
-bash-3.00$ cat /etc/project
system:0::::
user.root:1::::
noproject:2::::
default:3::::
group.staff:10::::
user.oracle:100:Oracle:::process.max-sem-nsems=(privileged,3096,deny); /
process.max-sem-ops=(privileged,3096,deny);project.max-sem-ids=(privileged,200,deny); /
project.max-shm-ids=(privileged,1024,deny);project.max-shm-memory=(privileged,8589934592,deny)
root # projects -l
system
projid : 0
comment: ""
users : (none)
groups : (none)
attribs:
user.root
projid : 1
comment: ""
users : (none)
groups : (none)
attribs:
noproject
projid : 2
comment: ""
users : (none)
groups : (none)
attribs:
default
projid : 3
comment: ""
users : (none)
groups : (none)
attribs:
group.staff
projid : 10
comment: ""
users : (none)
groups : (none)
attribs:
user.oracle
projid : 100
comment: "Oracle"
users : (none)
groups : (none)
attribs: process.max-sem-nsems=(privileged,3096,deny)
process.max-sem-ops=(privileged,3096,deny)
project.max-sem-ids=(privileged,3096,deny)
project.max-shm-ids=(privileged,200,deny)
project.max-shm-memory=(privileged,8589934592,deny)
As we can see the instance started and another shared memory segment with 4362092544
is shown by the icps command. The new parameters are saved in the /etc/project file
so that they can survife a reboot. The new max limit of 8GB is large enough to start
the instance while the other 780MB instance is still sunning. Finally the projects -l
command shows us the new parameters for the oracle user.
| This Article applies to: | Oracle Database (any Version) |
| Initial published: | 17.03.2008 |
| Last change: | 10.04.2008 |
