Libtune API documentation

Version 0.3
2006/10/06

Nadia Derbey
Nadia.Derbey@bull.net

1. Overview
2. Supported kernels and distros
3. The libtune API interfaces
    3.1. Getting information (tun_get)
        3.1.1. Description
        3.1.2. Parameters
        3.1.3. Returned values
    3.2. Setting information (tun_set)
        3.2.1. Description
        3.2.2. Parameters
        3.2.3. Returned values
    3.3. Locating information (tun_locate)
        3.3.1. Description
        3.3.2. Parameters
        3.3.3. Returned values
    3.4. Getting the key word for a location (tun_get_kwd)
        3.4.1. Description
        3.4.2. Parameters
        3.4.3. Returned values
    3.5. Getting all the valid keywords (tun_lst_all)
         3.5.1. Description
         3.5.2. Parameters
       3.5.3. Returned values
    3.6. Getting help information (tun_help)
         3.6.1. Description
         3.6.2 Parameters
         3.6.3. Returned values
4. Supported tunables
    4.1. Red Hat specific tunables
    4.2. SuSE specific tunables
5. Manual howto
    5.1. How to add support for a new kernel release
    5.2. How to add support for a new distribution family
    5.3. How to add support for a new tunable
         5.3.1. Kernel specific tunable
         5.3.2. Distro-dependent tunable
6. "Automatic" howto
    6.1. Adding support for a new kernel release
        6.1.1. In the source tree
        6.1.2. In the installed files
    6.2. Adding support for a new distro family
        6.2.1. In the source tree
        6.2.2. In the installed files
    6.3. Adding support for a new tunable
    6.4. Adding support for an existing tunable
    6.5. Removing a tunable support
    6.6. Getting the list of supported tunables
    6.7. Note
End of document

1. Overview

Accessing kernel tunables, system information and resource consumptions is needed during the whole life cycle of an application, starting from its installation. This access is usually implemented through installation and supervision scripts. Unfortunately, the following issues have been identified:

These scripts are rarely portable. since they require to get, set and change values that are represented by objects that may change from distribution to distribution, or even from one release to the other inside the same distribution.
There are quite multiple ways of accessing the kernel configuration and tunables: procfs, sysfs, existing syscalls or library routines, etc...

This raises the need for a standard, well defined API to manipulate the kernel configuration and tunables for software products to relay on.

The goal of the libtune API is to unify the various ways Linux developers have to access kernel tunables, system information, resource consumptions. The libtune API is built on top of the existing mechanisms, instead of replacing them, in order to maintain backward compatibility.

In the following chapters, we first present the supported kernel and distros supported today. Then, the API interfaces are presented. Follows a chapter about the tunables that are supported today.
Finally, a howto is given:

how to manually add support for a new kernel, how to add support for a new distro, how to add support for a new tunable.
how to automatically add support for the same features.

2. Supported kernels and distros

`kernel`	`Distributions`
	`Novell`		`Red Hat`
	`Family`	`Family release`	`Family`	`Family release`
`2.6.5`	`SLES (SuSE Linux Enterprise Server)`	`9`	`FC (Fedora Core)`	`2`
`2.6.9`			`FC (Fedora Core)`	`3`
`2.6.9`			`RHEL (Red Hat Enterprise Linux)`	`4`
`2.6.11`	`(Open) SuSE`	`9.3`	`FC (Fedora Core)`	`4`
`2.6.16`	`SLES (SuSE Linux Enterprise Server)`	`10`

3. The libtune API interfaces

3.1. Getting information (tun_get)

#include <libtune.h>
size_t tun_get(int keyword, void *identifier, char **out_buff, size_t *out_sz)

3.1.1. Description

This routine is used to get a tunable value: given a keyword that uniquely identifies a tunable, it returns its value (as read from the underlying /proc or /sys file). See next chapters to have a list of the supported tunables.
The value is returned as a buffer of characters (out_buff), whose size is out_sz.
For the tunables that depend on an identifier (pid, CPU id, etc), that identifier can be provided through the identifier parameter.

3.1.2. Parameters

This routine takes the following parameters:

keyword: this is the value that uniquely identifies the tunable
identifier: this is the identifier of the object we want to get information for. Depending on what the tunable is related to, the identfier may be one of:
- a NULL pointer if the tunable is system wide
- a pointer to a pid. In that case the convention is the following:
- a pointer to a CPU id.
- a string that identifies a network interface
out_buff: this is a pointer to the output buffer. It may be one of:
- a pointer to a NULL buffer. In that case, tun_get() will allocate a buffer large enough to contain the output data. This buffer should then be freed by the calling application.
- a pointer to a previously malloc-ated buffer. In that case, its size is provided through *out_sz. If that buffer is not large enough to contain all the output data, tun_get() will realloc-ate it, updating *out_buff and *out_sz as necessary.
The output data is provided in a string format. The buffer is null-terminated and does not include the last EOF.
out_sz: this is a pointer to the size of the output buffer, including the last NULL character.

3.1.3. Returned values

On success, tun_get() returns the number of characters read, including the terminating null character, but not including the EOF character. This value can be used to handle embedded null characters in the data read.
tun_get() returns -1 on failure, and errno is set:

EINVAL	keyword is invalid
	identifier is a NULL pointer, while the tunable asked for needs an identifier
	identifier is an empty string, while the tunable asked for needs an identifier
	outbuf is a NULL pointer
	out_sz is a NULL pointer
ENODATA	The value looked for could not be found
ESRCH	identifier points to an invalid pid
EPERM	the calling application does not have the neccessary rights to get the needed tunable value
ENOENT	the underlying file does not exist.
ENOMEM	if the API failed to allocate memory

3.2. Setting information (tun_set)

#include <libtune.h>
size_t tun_set(int keyword, void *identifier, char *in_buff, size_t in_sz, char **out_buff, size_t *out_sz)

3.2.1. Description

This routine is used to set a tunable value: given a keyword that uniquely identifies a tunable, it sets its value to the string contained in in_buff, whose size is in_sz (by writing that value into the underlying /proc or /sys file). See next chapters to have a list of the supported tunables.
The previous tunable value is returned as a buffer of characters (out_buff), whose size is out_sz.
For the tunables that depend on an identifier (pid, CPU id, etc), that identifier can be provided through the identifier parameter.

3.2.2. Parameters

This routine takes the following parameters:

keyword: this is the value that uniquely identifies the tunable to set
identifier: this is the identifier of the object we want to set information for. Depending on what the tunable is related to, the indentifier may be one of:

a NULL point if the tunable is system wide
a pointer to a pid. In that case the convention is the following:

pid = -1 identifies the calling process
pid > 0 identifies a process id

a pointer to a CPU id.
a string that identifies a network interface

in_buff: this is the input buffer. It should contain a null-terminated string with the value to set.
in_sz: this is the size of the input buffer, not including the last NULL character.
out_buff: this is a pointer to the output buffer. It contains the old setting of the changed object. It may be one of:
- a pointer to a NULL buffer. In that case, tun_set() will allocate a buffer large enough to contain the output data. This buffer should then be freed by the calling application.
- a pointer to a previously malloc-ated buffer. In that case, its size is provided through *out_sz. If that buffer is not large enough to contain all the output data, tun_set() will realloc-ate it, updating *out_buff and *out_sz as necessary.
The output data is provided in a string format. The buffer is null-terminated and does not include the last EOF.
If, for some reason an error occured while getting the old tunable setting, tun_set() will fail.
out_sz: this is a pointer to the size of the output buffer, including the last NULL character.

3.2.3. Returned values

On success, tun_set() returns the number of output characters, including the terminating null character, but not including the EOF character. This value can be used to handle embedded null characters in the data read.
tun_set() returns -1 on failure, and errno is set:

EINVAL	keyword is invalid
	identifier is a NULL pointer, while the tunable asked for needs an identifier
	identifier is an empty string, while the tunable asked for needs an identifier
	in_buff is a NULL pointer
	in_sz has a 0 value
	outbuf is a NULL pointer
	out_sz is a NULL pointer
	keyword is one of TUN_H_OP_MAX, TUN_S_OP_MAX, TUN_H_CHILD_MAX, TUN_S_CHILD_MAX, TUN_S_STK_MAX, TUN_S_DATA_MAX, TUN_S_VM_MAX and the input buffer is not a string of digits
ESRCH	identifier points to an invalid pid
EPERM	the calling application does not have the neccessary rights to set the needed tunable value
ENOENT	the underlying file does not exist.
ENOMEM	if the API failed to allocate memory
ERANGE	keyword is one of TUN_H_OP_MAX, TUN_S_OP_MAX, TUN_H_CHILD_MAX, TUN_S_CHILD_MAX, TUN_S_STK_MAX, TUN_S_DATA_MAX, TUN_S_VM_MAX and the input buffer contains a value that is out of range

3.3. Locating information (tun_locate)

#include <libtune.h>
int tun_locate(int keyword, char **location, int *loc_sz)

3.3.1. Description

This routine can be used to locate the underlying pseudo-file of a tunable: given a keyword, it returns in the string pointed to by location the path to the underlying pseudo-file. The path size is stored into loc_sz.
This routine is only meaningful for indexes that correspond to information that is managed through pseudo files.

3.3.2. Parameters

This routine takes the following parameters:

keyword: this is the value that uniquely identifies the tunable to get the location for.
location: this is a pointer to the output buffer. It may be one of:
- a pointer to a NULL buffer. In that case, tun_locate() will allocate a buffer large enough to contain the output data. This buffer should then be freed by the calling application.
- a pointer to a previously malloc-ated buffer. In that case, its size is provided through *loc_sz. If that buffer is not large enough to contain all the output data, tun_locate() will realloc-ate it, updating *location and *loc_sz as necessary.
The output data is provided in a string format. The buffer is null-terminated.
loc_sz: this is the size of the output buffer, not including the last NULL character.

3.3.3. Returned values

0 on success
1 if the index corresponds to a tunable that is not managed through pseudo files.
-1 on failure, and errno is set:

EINVAL	keyword is invalid
	outbuf is a NULL pointer
	out_sz is a NULL pointer
ENOMEM	if the API failed to allocate memory

3.4. Getting the key word for a location (tun_get_kwd)

#include <libtune.h>
char *tun_get_kwd(char *location, int *keyword)

3.4.1. Description

This is the reverse operation of the preceding one: given a location, it returns the associated keyword (to use for example in a set / get operation). It also returns a string that gives the corresponding constant name. Actually, since many tunables may have the same location, it is recommended to call this routine several times in order to get back all the possible keywords and constant names that correspond to the same location.
Example of tunables that have the same underlying location:

entire /proc/meminfo file
total memory space
size of a huge page
total swap space
free swap space, ...

3.4.2. Parameters

This routine takes the following parameters:

location: this is a null-terminated string that contains the file location to look for. Files locations that contain a variable part should be given with the printf format:

integer variable parts should be replaced by a '%d'
string varaible parts should be replaced by a '%s'

keyword: this is a pointer to an integer used:

in input to know where the search will begin from:

when tun_get_kwd() is first called, *keyword should be set to -1 (so the location will be looked for, starting from keyword 0)
subsequent calls should be done to tun_get_kwd() without changing *keyword's value (so the location will be looked for, starting from <*keyword + 1>

in ouput to store the next keyword associated to the specified location.

3.4.3. Returned values

If successful, tun_get_kwd() returns a pointer to the next constant name for the specified location (this string is malloc'ed by the library and should then be freed by the calling application), or NULL if the location is not found anymore (in that case, *keyword is not updated and errno is set to ENOENT).
Note: if the location corresponds to a keyword that has been generated by the daemon, an empty string is returned ("").
On failure, ((char *) -1) is returned, *keyword is not updated and errno is set:

EINVAL	location is a NULL pointer
	keyword points to an invalid value
	location is an empty string
	keyword is a NULL pointer
ENOMEM	if the API failed to allocate memory

3.5. Getting all the valid key words (tun_lst_all)

#include <libtune.h>
int tun_lst_all(int *keyword)

3.5.1. Description

This is an interface that can be used to get all the valid tunables present in the TUNDB database: given a keyword, it returns the next valid one in the supported tunables. This interface, combined with tun_help() (see below) is useful to know what are the tunables covered by the libtune API.

3.5.2. Parameters

This routine takes the following parameters:

keyword: this is a pointer to an integer used:

in input to know where the search will begin from:

when tun_lst_all() is first called, *keyword should be set to -1 (so the first valid tunable will be looked for, starting from keyword 0)
subsequent calls should be done to tun_lst_all() without changing *keyword's value (so the next valid tunable will be looked for, starting from keyword <*keyword + 1>

in ouput to store the next valid keyword.

3.5.3. Returned values

If successful, tun_lst_all() returns 0 and *keyword is filled with the next valid tunable value.
If there ane nomore valid tunables, TUN_ILLEGAL is returned.
On failure, -1 is returned, *keyword is not updated and errno is set:

EINVAL	keyword is a NULL pointer
	keyword points to an invalid value

3.6. Getting help information (tun_help)

#include <libtune.h>
int tun_help(int keyword, char **help, size_t *help_sz)

3.6.1. Description

This routine can be used, given a keyword value, to return the corresponding help string into the string pointed to by help. The string size is stored into help_sz.

3.6.2. Parameters

This routine takes the following parameters:

keyword: this is the keyword to get help for
help: this is a pointer to the output buffer. It may be one of:
- a pointer to a NULL buffer. In that case, tun_help() will allocate a buffer large enough to contain the output data. This buffer should then be freed by the calling application.
- a pointer to a previously malloc-ated buffer. In that case, its size is provided through *help_sz. If that buffer is not large enough to contain all the output data, tun_help() will realloc-ate it, updating *help and *help_sz as necessary.
The output data is provided in a string format. The buffer is null-terminated.
help_sz: this is the size of the output buffer, not including the last NULL character.

3.6.3. Returned values

0 on success
1 if the index corresponds to information that does not have an associate help string (this is true for files detected by the daemon).
-1 on failure, errno is set:

EINVAL	keyword is invalid
	help is a NULL pointer
	help_sz is a NULL pointer
ENOMEM	if the API failed to allocate memory

4. Supported tunables

`Constant name`	`Tunable meaning`	`id parameter in the API interfaces`	`location`
`TUN_SHMMIN`	`Minimum shm segment size (in bytes)`		`N/A`
`TUN_H_OP_MAX`	`Max number of open fds (hard limit)`		`N/A`
`TUN_S_OP_MAX`	`Max number of open fds (soft limit)`		`N/A`
`TUN_H_CHILD_MAX`	`Max number of processes (hard limit)`		`N/A`
`TUN_S_CHILD_MAX`	`Max number of processes (soft limit)`		`N/A`
`TUN_S_STK_MAX`	`stack size (soft limit)`		`N/A`
`TUN_S_DATA_MAX`	`size of data segment (soft limit)`		`N/A`
`TUN_S_VM_MAX`	`size of virtual memory (soft limit)`		`N/A`
`TUN_SEMMSL`	`Max number of semaphores per id`		`/proc/sys/kernel/sem`
`TUN_SEMMNS`	`Max number of semaphores in the system`		`/proc/sys/kernel/sem`
`TUN_SEMOPM`	`Max number of operations per semop call`		`/proc/sys/kernel/sem`
`TUN_SEMMNI`	`Max number of semaphore identifiers`		`/proc/sys/kernel/sem`
`TUN_REC_PACKS`	`Number of receive packets for <interface>`	`Network interface (string)`	`/proc/net/dev`
`TUN_MEMSPACE`	`Total memory space`		`/proc/meminfo`
`TUN_SWAPTOT`	`Total swap space`		`/proc/meminfo`
`TUN_SWAPFREE`	`Free swap area`		`/proc/meminfo`
`TUN_HPGTOT`	`Total number of huge pages`		`/proc/meminfo`
`TUN_HPGFREE`	`Number of free huge pages`		`/proc/meminfo`
`TUN_HPGSZ`	`Huge page size`		`/proc/meminfo`
`TUN_CTX_SW`	`Number of context switches`		`/proc/stat`
`TUN_CPUPHYSID`	`Physical id for <cpuid>`	`CPU id (integer)`	`/proc/cpuinfo`
`TUN_SIBLINGS`	`Number of sibling CPUs for CPU <cpuid>`	`CPU id (integer)`	`/proc/cpuinfo`
`TUN_SHMMNI`	`Maximum number of shm segment ids`		`/proc/sys/kernel/shmmni`
`TUN_SEM`	`All semaphore parameters`		`/proc/sys/kernel/sem`
`TUN_CMDLINE`	`Complete command line for process <pid>`	`Process id (integer)`	`/proc/<PID>/cmdline`
`TUN_STAT`	`Statistics for process <pid>`	`Process id (integer)`	`/proc/<PID>/stat`
`TUN_MEMINFO`	`Memory statistics`		`/proc/meminfo`
`TUN_SLEEPSTATE`	`Sleep state`		`/sys/power/state`
`TUN_CPULIMITS`	`limits for CPU <cpuid>`	`CPU id (integer)`	`/proc/acpi/processor/CPU<CPUID>/limit`
`TUN_FILEMAX`	`Max number of files`		`/proc/sys/fs/file-max`
`TUN_CPUINFO`	`CPU info`		`/proc/cpuinfo`
`TUN_FIN_TIMEOUT`	`Timeout for a FIN packet before the socket is forcibly closed`		`/proc/sys/net/ipv4/tcp_fin_timeout`
`TUN_MAX_BACKLOG`	`Maximum # of queued connection requests which have still not received an ACK from the connecting client`		`/proc/sys/net/ipv4/tcp_max_syn_backlog`
`TUN_TW_REUSE`	`Allow to reuse TIME-WAIT sockets for new connections`		`/proc/sys/net/ipv4/tcp_tw_reuse`
`TUN_LOC_P_RANGE`	`Local port range used by TCP and UDP`		`/proc/sys/net/ipv4/ip_local_port_range`
`TUN_IPV4_RMEM`	`Min-default-max memory size of the TCP receive buffers`		`/proc/sys/net/ipv4/tcp_rmem`
`TUN_IPV4_WMEM`	`Min-default-max memory size of the TCP send buffers`		`/proc/sys/net/ipv4/tcp_wmem`
`TUN_WIN_SCALING`	`Enable TCP to negociate the use of window scaling during connection set up`		`/proc/sys/net/ipv4/tcp_window_scaling`
`TUN_MTU_SIZE`	`MTU size for <interface>`	`Network interface (string)`	`/proc/sys/net/ipv6/conf/<interface>/mtu`
`TUN_PARTITIONS`	`Disks info`		`/proc/partitions`
`TUN_NET_DEV`	`Network device status info`		`/proc/net/dev`
`TUN_RMEM_MAX`	`Max receive buffer size`		`/proc/sys/net/core/rmem_max`
`TUN_WMEM_MAX`	`Max send buffer size`		`/proc/sys/net/core/wmem_max`
`TUN_NET_BACKLOG`	`Max # of received buffers processed before congestion`		`/proc/sys/net/core/netdev_max_backlog`
`TUN_OVC_MEM`	`Kernel policy for memory allocation`		`/proc/sys/vm/overcommit_memory`
`TUN_OVC_RATIO`	`Percentage of memory added to the swap to get the max address space allowed to be committed`		`/proc/sys/vm/overcommit_ratio`
`TUN_SWAPPINESS`	`Swappiness`		`/proc/sys/vm/swappiness`
`TUN_NR_HUGE_PG`	`Number of configured huge pages`		`/proc/sys/vm/nr_hugepages`
`TUN_P_STAT`	`Kernel and system statistics`		`/proc/stat`
`TUN_SHMALL`	`Max # of shm segment pages system wide`		`/proc/sys/kernel/shmall`
`TUN_SHMMAX`	`Max size, in bytes, of a shm segment`		`/proc/sys/kernel/shmmax`
`TUN_MSGMNI`	`Max # of msg queue ids`		`/proc/sys/kernel/msgmni`
`TUN_THREADS_MAX`	`Max # of threads`		`/proc/sys/kernel/threads-max`
`TUN_MIN_ADV_MSS`	`Min MSS value`		`/proc/sys/net/ipv4/route/min_adv_mss`

4.1. RedHat specific tunables

`Constant name`	`Tunable meaning`	`location`	`>= FC4 location`
`TUN_EXEC_SHIELD`	`Security level for the exec-shield functionality - Red Hat specific`	`/proc/sys/kernel/exec-shield`	`/proc/sys/kernel/exec-shield`
`TUN_EXEC_SHIELD_R`	`Controls whether to randomize VM mappings - Red Hat specific`	`/proc/sys/kernel/exec-shield-randomize`	`/proc/sys/kernel/randomize_va_space`

4.2. SuSE specific tunables

`Constant name`	`Tunable meaning`	`id parameter in the API interfaces`	`location`
`TUN_MAPPED_BASE`	`Shared memory address map for <pid> - SuSE specific`	`Process id (integer)`	`/proc/<PID>/mapped_base`

5. "Manual" howto

In the following chapters, we explain how to manually enhance the source tree in order to support more kernel releases, distros, distro families.

5.1. How to add support for a new kernel release

Assume you want to add suport for the 2.6.Z kernel release

Create the directory db/include/base-2.6.Z
Create the db/include/base-2.6.Z/libtuninitdb_base.h include file.
This file should contain the database definition (TUNDB_K[] array) for the available tunables in 2.6.Z kernel release.

the underlying pseudo-file name attribute OR'd with the underlying pseudo-file contents attribute.
the index of the strategy routine that should be used to get / set the tunable value
the path to the underlying pseudo-file
the help string
the constant string

Create the distro specific include file (see the Howto for a new distribution).

5.2. How to add support for a new distribution family

Assume you want to add support for Fedora Core X under the 2.6.Z kernel release.

Create the directory db/include/base-2.6.Z/distro-FC_X
Create the db/include/base-2.6.Z/distro-FC_X/libtuninitdb_distro.h include file.
This file should contain the database definition (TUNDB_D[] array) for the available tunables in Fedora Core X family. It should be copied from templates/distro/libtuninitdb_distro.h and enhanced if needed.
Each TUNDB_D[] index corresponds to a supported tunable (defined in include/libtune.h) and contains the following information:

the underlying pseudo-file name attribute OR'd with the underlying pseudo-file contents attribute.
the index of the strategy routine that should be used to get / set the tunable value
the path to the underlying pseudo-file
the help string
the constant string

If a family other than the ones already supported is to be supported, scripts/get_distro.sh should be changed to detect it and output a pseudo for that distro family. In that case, the <family_pseudo>_<family_release> directory should be created under the appropriate kernel release, <family_pseudo> being the result of get_distro.sh -f and <family_release> being the result of get_distro.sh -r:

# cd scripts # F=`./get_distro.sh -f` # R=`./get_distro.sh -r` # cd .. # mkdir -p db/include/base-2.6.Z/distro-${F}_${R}

Today, the values that can be returned by scripts/get_distro.sh for ${R} are:

Distro Distro family Pseudo ($F)

RedHat Fedora Core FC

Red Hat Linux RH

Red Hat Enterprise Linux RHEL

SuSE SuSE Linux SL

SuSE Linux Enterprise Server SLES

5.3. How to add support for a tunable

5.3.1. Kernel specific tunable

Assume you want to add the tunable TUN_BLAH_BLAH for the 2.6.Z kernel

if this is a brand new tunable (i.e. it was not yet supported), define the tunable constant value into include/libtune.h, applying the following rule:

take TUN_NEXT_K value for TUN_BLAH_BLAH constant value
increment TUN_NEXT_K (this is important for the automatic scripts)

(optional) define the following values:

H_BLAH_BLAH: help string for TUN_BLAH_BLAH into db/include/libtune_helps.h
C_BLAH_BLAH: constant string for TUN_BLAH_BLAH ("TUN_BLAH_BLAH") into db/include/libtune_const.h
P_BLAH_BLAH: location path for the underlying pseudo-file into db/include/libtune_paths.h. The following writing rules apply for the location paths:

if the pseudo-file has a system-wide name (ala /proc/meminfo), its full name should be specified
if the pseudo-file has a name with a variable part:

a process id should be replaced by %d: "/proc/%d/cmdline"
a CPU id should be replaced by %d: "/proc/acpi/processor/CPU%d/limit"
a network interface should be replaced by %s: "/proc/sys/net/ipv6/conf/%s/mtu"
a PCI bus should be replaced by %s: "/sys/bus/pci/devices/%s/class"

add the tune_t structure for TUN_BLAH_BLAH at the appropriate index of TUNDB_K[] array into db/include/base-2.6.Z/libtuninitdb_base.h:

attribute: define it by OR'ing the underlying file name attribute and the content attribute

pseudo-file name attribute:

`Name category (see point #2 in the list)`	`How to set the name attribute`
`2.1`	`ATTR_NAME_SYS_WIDE`
`2.2.a`	`ATTR_NAME_PER_PROC`
`2.2.b`	`ATTR_NAME_PER_CPU`
`2.2.c`	`ATTR_NAME_PER_NETINT`
`2.2.d`	`ATTR_NAME_PER_PCI`

content attribute:
the following file contents are currently supported:

the tunable has no underlying pseudo-file
the tunable has an underlying pseudo-file made of a single line, with several fields (ala /proc/sys/kernel/sem)
the tunable has an underlying pseudo-file made of several lines, each line made of several fields and dedicated to one object (ala /proc/net/dev)
the tunable has an underlying pseudo-file made of a single block of lines, each line identified by a fixed string (ala /proc/meminfo)
the tunable has an underlying pseudo-file made of a several blocks of lines, each line identified by a fixed string, each block of lines dedicated to one object (ala /proc/cpuinfo)
if the tunable has an underlying pseudo-file whatever its contents

`Contents category`	`How to set the content attribute`
`3.a`	`Depends on the tunable`
`3.b`	`ATTR_CONT_SYS_WIDE`
`3.c`	`ATTR_CONT_PER_* (see include/libtune_priv.h for the possible values)`
`3.d`	`ATTR_CONT_SYS_WIDE`
`3.e`	`ATTR_CONT_PER_* (see include/libtune_priv.h for the possible values)`
`3.f`	`ATTR_CONT_SYS_WIDE`

strategy routines: the strategy routines depend on the underlying pseudo-file contents (refer to the previous point). They are actually represented by their index into the array STRATEGY_ROUTINES[] (see lib/libtundb.c):

`Contents category`	`Index to use`	`Get strategy routine`	`Set strategy routine`
`3.a`	`N/A`	`Depends on the tunable`	`Depends on the tunable`
`3.b`	`2`	`strat_get_subfile_line`	`strat_set_subfile_line`
`3.c`	`3`	`strat_get_subfile_lines`	`strat_set_subfile_lines`
`3.d`	`4`	`strat_get_subfile_block`	`strat_set_subfile_block`
`3.e`	`5`	`strat_get_subfile_blocks`	`strat_set_subfile_block`
`3.f`	`6`	`strat_get_file`	`strat_set_file`

location path: P_BLAH_BLAH or NULL if the tunable belongs to the 3.a category
help string: H_BLAH_BLAH
constant string: C_BLAH_BLAH
field number (beginning from 1): should be provided only for the 3.b and 3.c tunables categories
field name: should be provided only for the 3.d and 3.e tunables categories

Note: As many "empty" tune_t strtuctures as needed should be added between the last defined index and the new one into TUNDB_K[].

5.3.2. Distro-dependent tunable

Assume you want to add the tunable TUN_BLAH_BLAH for the Red Hat Fedora Core X distribution (2.6.Z kernel release):

define the tunable value into include/libtune.h, applying the following rule:

take TUN_NEXT_D value for TUN_BLAH_BLAH constant value
increment TUN_NEXT_D (this is important for the automatic scripts)

(optional) define the following values:

H_BLAH_BLAH: help string for TUN_BLAH_BLAH into db/include/libtune_helps.h
C_BLAH_BLAH: constant string for TUN_BLAH_BLAH ("TUN_BLAH_BLAH") into db/include/libtune_const.h
P_BLAH_BLAH: location path for the underlying pseudo-file into db/include/libtune_paths.h (see point #2 in the previous chapter for the writing rules)

add the tune_t structure for TUN_BLAH_BLAH at the appropriate index of TUNDB_D[] array into db/include/base-2.6.Z/distro-FC_X/libtuninitdb_distro.h (see previous chapter for the rules to apply)

6. "Automatic" howto

A set of scripts has been recently added to the delivery. With the help of these scripts it is now possible to automatically do all the operations described in the previous chapter (add support for a new kernel release, a new distro family within a given kernel release, or a new tunable).

6.1. Adding support for a new kernel release (add_kernel.sh)

6.1.1. In the source tree

This is done by calling the script add_kernel.sh. If you want to add suport for the 2.6.30 kernel release:

# cd scripts # ./add_support.sh 2.6.30 #

The effect of this command is to create the file db/include/base-2.6.30/libtuninitdb_base.h as described in chapter 5.1 .

6.1.2. In the installed files

This is done by calling the script add_kernel.sh. If you want to add suport for the 2.6.30 kernel release:

# cd /var/lib/tundb/bin # ./add_kernel.sh 2.6.30 #

The effect of this command is to create the following 2 files:

db/include/base-2.6.30/libtuninitdb_base.h (source of the database definition)
/var/lib/tundb/base-2.6.30/tundb_kernel (binary database definition)

6.2. Adding support for a new distro family (add_distro.sh)

Support is always added for a distro family within an existing kernel release.

6.2.1. In the source tree

This is done by calling the script add_distro.sh. The script then asks you to enter an already supported kernel release.
If you want to add support for the Fedora Core 20 distro family within the newly supported 2.6.30 kernel release (the characters in bold are those typed by the user, while the others are those output by the tool):

# cd scripts
# ./add_distro.sh FC 20 Enter an existing kernel release: 1 ) 2.6.9 2 ) 2.6.17 3 ) 2.6.5 4 ) 2.6.16 5 ) 2.6.11 6 ) 2.6.30 Answer: 6 #../db/include/base-2.6.30/distro-FC_20/libtuninitdb_distro.h has been successfully created

The effect of this command is to create the file db/include/base-2.6.30/distro-FC_20/libtuninit_distro.h as described in chapter 5.2.

6.2.2. In the installed files

# cd /var/lib/tundb
# ./add_distro.sh FC 20 Enter an existing kernel release: 1 ) 2.6.9 2 ) 2.6.17 3 ) 2.6.5 4 ) 2.6.16 5 ) 2.6.11 6 ) 2.6.30 Answer: 6 # /var/lib/tundb/base-2.6.30/distro-FC_20/libtuninitdb_distro.h has been successfully created /var/lib/tundb/base-2.6.30/distro-FC_20/tundb_distro has been successfully created

The effect of this command is to create the following 2 files:

db/include/base-2.6.30/distro-FC_20/libtuninitdb_distro.h (source of the distro-dependent database definition)
/var/lib/tundb/base-2.6.30/distro-FC_20/tundb_distro (binary distro-dependent database definition)

6.3. Adding support for a new tunable (new_tunable.sh)

As for the previous commands, support can be added into one of:

the source tree (you should then call the script from the "scripts" directory)
the installed files (you should the call the script from the /var/lib/tundb/bin directory)

The effect of the script is to update libtune.h (actually include/libtune.h if changing the source tree or /usr/include/libtune.h if changing the installed files). This script only needs to know if the tunable is kernel or distro-dependent (see 5.3).

If you want to add the kernel-dependent TUN_BLAH_BLAH_K tunable in the installed files (the characters in bold are those typed by the user, while the others are those output by the tool):

# cd /var/lib/tundb/bin # ./new_tunable.sh BLAH_BLAH_K
*************** DEPENDENCY *************** Is the tunable

1 ) kernel dependent
2 ) distro dependent

Answer: 1 TUN_BLAH_BLAH_K has been correctly added to /usr/include/libtune.h Original /usr/include/libtune.h has been saved into /usr/include/libtune.h_ref_24580 # diff /usr/include/libtune.h /usr/include/libtune.h_ref_24580 105a106 > #define TUN_BLAH_BLAH_K 0x00036 107c108 < #define TUN_NEXT_K 0x00036 --- > #define TUN_NEXT_K 0x00037

6.4. Adding support for an existing tunable

Again, support can be added into one of:

the source tree (you should then call the script from the "scripts" directory)
the installed files (you should the call the script from the /var/lib/tundb/bin directory). When updating the installed file, you can choose to update only the database source file or the database binary file or both.

The effect of the script is to update (see 5.3):

for a kernel dependent tunable:

libtuninitdb_base.h (in the source tree or in the installed files).
tundb_base (in the installed files)

for a distro dependent tunable:

libtuninitdb_distro.h (in the source tree or in the installed files).
tundb_distro (in the installed files)

Support can be added for a tunable only after it has been defined into libtune.h (i.e. new_tunable.sh has been called - see 6.3). This is the only way for the automatic scripts to add the tunable description structure at the right index into the tunables database.
Support for a tunable may be added within an existing kernel release or distro. This is a first information that will be asked for by the script.
The next questions are common whether we are adding a kernel dependent tunable or a distro-dependent tunable. The only thing that differs is the changed files (see before).

If you want to add the kernel-dependent TUN_BLAH_BLAH_K tunable within the newly supported 2.6.30 kernel release (the characters in bold are those typed by the user, while the others are those output by the tool):

# ./add_tunable.sh -t BLAH_BLAH_K TUN_BLAH_BLAH_K goes to kernel DB Enter an existing kernel release:1 ) 2.6.9 2 ) 2.6.17 3 ) 2.6.5 4 ) 2.6.16 5 ) 2.6.11 6 ) 2.6.30 Answer: 6 What file do you want to update 1 ) database binary file 2 ) database header file 3 ) both files Answer: 3 The following files will be updated /var/lib/tundb/base-2.6.30/libtuninitdb_base.h /var/lib/tundb/base-2.6.30/tundb_kernel ************* TUNABLE CHARACTERISTICS *************** *************** STRATEGY ROUTINE ***************1 ) Get/set a given field in a file made of a single line 2 ) Get/set a given field within a given line in a file made of several lines 3 ) Get/set a value in a file made of a single block. This block is made of several lines that contain an identifier and the actual value. 4 ) Get/set a value in a file made of several blocks. Each block is identified by a unique identifier, and it is made of several lines that contain an identifier and the actual value. 5 ) Get/set the entire file contents 6 ) Other (no pseudo-file support) Answer: 1 Enter the field number within a line to access the tunable (should start at 0): 1 *************** FILE NAME ATTRIBUTE ***************1 ) The pathname is a fixed string 2 ) Per process pathname (a pid is needed to build the path) 3 ) Per CPU pathname (a CPU id is needed to build the path) 4 ) Per IRQ pathname (an IRQ number is needed to build the path) 5 ) Per PCI bus pathname (a PCI bus number is needed to build the path) 6 ) Per network pathname (a network interface name is needed to build the path) 7 ) Per user pathname (a uid is needed to build the path) Answer: 1
*************** LOCATION PATH *************** Enter the absolute path to the pseudo-file (printf conversion sequences - %. - are recognized):/proc/blah_blah_k************** FILE CONTENTS ATTRIBUTE ************** 1 ) System wide setting 2 ) Per process setting 3 ) Per CPU setting 4 ) Per IRQ setting 5 ) Per PCI bus setting 6 ) Per network interface setting 7 ) Per user setting Answer: 1
*************** HELP STRING *************** Enter the help string in a single line format (escape sequences are recognized and not interpreted): help string for BLAH_BLAH_K TUN_BLAH_BLAH_K has been correctly added to /var/lib/tundb/base-2.6.30/tundb_kernel Original /var/lib/tundb/base-2.6.30/tundb_kernel has been saved into /var/lib/tundb/base-2.6.30/tundb_kernel_ref_24666 TUN_BLAH_BLAH_K has been correctly added to /var/lib/tundb/base-2.6.30/libtuninitdb_base.h Original /var/lib/tundb/base-2.6.30/libtuninitdb_base.h has been saved into /var/lib/tundb/base-2.6.30/libtuninitdb_base.h_ref_24666 # diff/var/lib/tundb/base-2.6.30/libtuninitdb_base.h_ref_24666 \/var/lib/tundb/base-2.6.30/libtuninitdb_base.h 142a143,146 > {ATTR_NAME_SYS_WIDE|ATTR_CONT_SYS_WIDE, 6, > "/proc/blah_blah_k", > "help string for blah_blah_k", > "TUN_BLAH_BLAH_K"}, #

6.5. Removing a tunable support

Again, support can be removed from one of:

the source tree (you should then call the script from the "scripts" directory)
the installed files (you should the call the script from the /var/lib/tundb/bin directory). When updating the installed file, you can choose to update only the database source file or the database binary file or both.

The effect of the script is to update (see 5.3):

for a kernel dependent tunable:

libtuninitdb_base.h (in the source tree or in the installed files).
tundb_base (in the installed files)

for a distro dependent tunable:

libtuninitdb_distro.h (in the source tree or in the installed files).
tundb_distro (in the installed files)

Support for a tunable may be removed from an existing kernel release or distro. This is a first information that will be asked for by the script.
The next questions are common whether we are removing a kernel dependent tunable or a distro-dependent tunable. The only thing that differs is the changed files (see before).

If you want to remove the kernel-dependent TUN_BLAH_BLAH_K tunable from the newly supported 2.6.30 kernel release (the characters in bold are those typed by the user, while the others are those output by the tool):

# ./remove_tunable.sh BLAH_BLAH_K TUN_BLAH_BLAH_K will be removed from kernel DB Enter an existing kernel release: 1 ) 2.6.9 2 ) 2.6.17 3 ) 2.6.5 4 ) 2.6.16 5 ) 2.6.11 6 ) 2.6.30 Answer: 6 What file do you want to update 1 ) database binary file 2 ) database header file 3 ) both files Answer: 3 The following files will be updated /var/lib/tundb/base-2.6.30/libtuninitdb_base.h /var/lib/tundb/base-2.6.30/tundb_kernel TUN_BLAH_BLAH_K has been correctly removed from /var/lib/tundb/base-2.6.30/tundb_kernel Original /var/lib/tundb/base-2.6.30/tundb_kernel has been saved into /var/lib/tundb/base-2.6.30/tundb_kernel_ref_24845 TUN_BLAH_BLAH_K has been correctly removed from /var/lib/tundb/base-2.6.30/libtuninitdb_base.h Original /var/lib/tundb/base-2.6.30/libtuninitdb_base.h has been saved into /var/lib/tundb/base-2.6.30/libtuninitdb_base.h_ref_24845 # diff /var/lib/tundb/base-2.6.30/libtuninitdb_base.h_ref_24845 \ /var/lib/tundb/base-2.6.30/libtuninitdb_base.h 143,146c143,144 < {ATTR_NAME_SYS_WIDE|ATTR_CONT_SYS_WIDE, 6, < "/proc/blah_blah_k", < "help string for blah_blah_k", < "TUN_BLAH_BLAH_K"}, --- > {ATTR_NONE, 0, "", "", ""}, #

6.6. Getting the list of supported tunables

The script get_tunedb scans the installed binary database and outputs the list of supported tunables.
If called without option, it ouput both the kernel-dependent and distro-dependent tunable.
If called with the -k option, it only outputs the kernel-dependent tunables
If called with the -d option, it only outputs the distro-dependent tunables.
For each tunable, the following values are output:

Tunable constant name
Tunable constant value
Location path
Help string

Here is the ouput for the distro-dependent tunables of a database installed on SLES 10 family:

# cd /var/lib/tundb/bin
# ./get_tunedb -d

TUN_MAPPED_BASE      (0xC02)
        /proc/%d/mapped_base
        Shared memory address map for <pid>

6.7. Note

All the automatically generated files are copied from template files (under the template directory). The template files, as well as the generated ones should carefully be changed in order for the script to remain working correctly.

`Distro`	`Distro family`	`Pseudo ($F)`
`RedHat`	`Fedora Core`	`FC`
	`Red Hat Linux`	`RH`
	`Red Hat Enterprise Linux`	`RHEL`
`SuSE`	`SuSE Linux`	`SL`
`SuSE`	`SuSE Linux Enterprise Server`	`SLES`