Copyright © 2008 ASIPTO SRL
Abstract
This book documents the internal architecture of OPENSER SIP Server, providing the details useful to develop extensions in the core or as a module.
The focus will be on major components of the SIP server, such as memory manager, locking system, parser, database API, configuration file, MI commands, pseudo-variables and module interface.
Examples, API and the architecture are based on current development version of OPENSER - 1.4.0 - at April 2, 2008. Some may change until 1.4.0 is released as stable. The document will be timely updated accordingly, check for updates at www.asipto.com or www.openser.org.
This document is free to use for anybody. The authors are not liable in any way for the consequences you may get due to usage of this document.
Table of Contents
- 1. Introduction
- 2. OPENSER Architecture
- 3. Locking system
- 4. Memory Manager
- 5. Data structures
- 6. SIP Parser
- 7. Transport Layer
- 8. Extending configuration file
- 9. Database API
- 9.1. DB API Structure
- 9.2. DB API Functions
- 9.2.1. Function init(...)
- 9.2.2. Function close(...)
- 9.2.3. Function use_table(...)
- 9.2.4. Function query(...)
- 9.2.5. Function fetch_result(...)
- 9.2.6. Function raw_query(...)
- 9.2.7. Function free_result(...)
- 9.2.8. Function insert(...)
- 9.2.9. Function delete(...)
- 9.2.10. Function update(...)
- 9.2.11. Function replace(...)
- 9.2.12. Function last_inserted_id(...)
- 9.2.13. Function insert_update(...)
- 9.3. DB API Data Types
- 9.4. Macros
- 9.5. Example of usage
- 10. Management Interface
- 11. Pseudo-variables
- 12. Transformations
- 13. Statistics
- 14. Data Lumps
- 15. Timer
- 16. Module Development
- 16.1. module_exports type
- 16.2. cmd_export_t type
- 16.3. param_export_t type
- 16.4. proc_export_t type
- 16.5. stat_export_t type
- 16.6. mi_export_t type
- 16.7. pv_export_t
- 16.8. Functions Types
- 16.9. Command Functions
- 16.10. Developing a new module
- 16.10.1. Naming the module
- 16.10.2. Makefile
- 16.10.3. Main File
- 16.10.4. Add Module Parameter
- 16.10.5. Module Init Function
- 16.10.6. Module Child Init Function
- 16.10.7. Module Destroy Function
- 16.10.8. Add Command Function
- 16.10.9. Add Pseudo-Variable
- 16.10.10. Add MI Command
- 16.10.11. Add Extra Process
- 16.10.12. CFGUTILS module_exports
- 17. Licensing
- 18. References
- 19. Contact Details
List of Figures
In June 2005, OPENSER was born as a split from SIP Express Router project of FhG FOKUS Institute, Berlin, Germany. The newly created project was aiming to create an open development environment to build robust and scalable open source SIP server.
The website of the project is http://www.openser.org. The source code is hosted on sourceforge.net SVN repository.
Today OPENSER SIP Server is a reference implementation, featuring hundreds of VoIP services world wide, being developed by people around the world. It is included in official distributions of several Linux and BSD flavors.
The number of registered developers and packagers exceeded 25. The level of contributions and the amount of contributors has an important impact on the evolution of the project. The book tries to ease the understanding of OPENSER from a developer point of view, giving the kick start knowledge, it does not intend to be a cookbook. Efforts to improve the documentation in the sources is undertaken and make it doxygen compliant, making a good developer documentation out of there.
Daniel-Constantin Mierla is one of the co-founders of OPENSER SIP Server project. He is involved in VoIP and SIP since beginning of 2002, at FhG FOKUS Institute, Berlin, Germany, being involved as core developer in the SIP Express Router project. Currently he is employed by ASIPTO, an OPENSER-focused company. Daniel is an active OPENSER developer and member of management board.
Elena-Ramona Modroiu is one of the co-founder of OPENSER SIP Server project. She got involved in VoIP and SIP while working at her graduation thesis within SIP Express Router project at FhG FOKUS Institute. She completed studies at Polytechnic University of Valencia and Siemens Germany, working now at ASIPTO, being an active developer and member of management board of OPENSER.
The two authored many online tutorials about OPENSER that include OPENSER Core Cookbook, OPENSER Transformations, OPENSER Pseudo-variables, OPENSER and Asterisk Integration, OPENSER and FreeRADIUS.
This document is focusing only to OPENSER specific API, has no intention to teach C programing for Linux and Networking. You, as a reader, should have already the basic knowledge of C programming.
Do not contact the authors to ask about standard C functions or variables.
There are many references to parts of code in the source tree. You must be familiar with the directory structure of OPENSER. It is not our intention to explain how something was implemented, but how to use existing code to extend OPENSER easily.
The source code remains the best reference for developers. In the last time, the comments around the important functions in OPENSER have been improved and converted to doxygen format. You should double-check the source code if the prototype of the functions presented in this document are still valid.
Daily updated doxygen documentation is available at http://devel.openser.org/doxygen/.
OPENSER has a modular architecture. As a big picture, there are two main categories:
the core - it is the component that provides the low-level functionalities for OPENSER
the modules - are the components that provides the most of the functionalities that make OPENSER powerful in real world deployments.
The core includes:
memory manager
SIP message parser
locking system
DNS and transport layer management (UDP, TCP, TLS, SCTP)
configuration file parser and interpreter
database abstraction layer (DB API)
management interface (MI) API
stateless forwarding
pseudo-variables and transformations engines
statistics engine
timer API
With the modules you can get functionalities such as:
user location management
accounting, authorization and authentication
text and regular expression operations
stateless replying
stateful processing
instant messaging and presence extensions
RADIUS support
database connectors
MI transports
CPL interpreter
sms and xmpp gateways
NAT traversal
Perl and Java SIP Servlet extensions
The execution of OPENSER configuration file is triggered when receiving a SIP message from the network. The processing flow is different for a SIP request or a SIP reply.
The document contains a chapter dedicated to Configuration File that explains its structure and the types of routing blocks.
OPENSER provides a custom locking system which has a simple interface for development. Its root element is a mutex semaphore, that can be set (locked) or unset (unlocked). The rest of synchronization mechanisms available in SysV and POSIX not being needed.
The locks can be used as simple variables or lock sets (array of simple locks). To improve the speed, behind the locks is, by default, machine-specific code. If the architecture of the machine is unknown, OPENSER will use SysV semaphores.
Basically, to create a lock, you have to define a variable of type gen_lock_t, allocate it in shared memory and initialize it. Then you can perform set/unset operations, destroying the variable when you finish using it.
To use the locking system in your C code you have to include the headers file: locking.h.
Data types and available functions to do operations with locks are described in the next sections.
It is the type to define a lock variable. It must be allocated in shared memory, to be available across OPENSER processes.
Allocates a lock in shared memory. If your gen_lock_t is not part of a structure allocated in share memory, you have to use this function to properly create the lock.
It returns the pointer to share memory structure.
Example 3.3. Example of usage
...
#include "locking.h"
...
gen_lock_t *lock;
lock = lock_alloc();
if(lock==NULL)
{
LM_ERR("cannot allocate the lock\n");
exit;
}
...
Free the shared memory allocated for lock.
The parameter is a variable returned by lock_alloc(...).
Example 3.5. Example of usage
...
#include "locking.h"
...
gen_lock_t *lock;
lock = lock_alloc();
if(lock==NULL)
{
LM_ERR("cannot allocate the lock\n");
exit;
}
/* make use of lock */
...
lock_dealloc(lock);
...
Initialize the lock. You must call this function before fist set operation on the lock.
It returns the parameter if there is no error, NULL if error occurred.
Example 3.7. Example of usage
...
#include "locking.h"
...
gen_lock_t *lock;
lock = lock_alloc();
if(lock==NULL)
{
LM_ERR("cannot allocate the lock\n");
exit;
}
if(lock_init(lock)==NULL)
{
LM_ERR("cannot init the lock\n");
lock_dealloc(lock);
exit;
}
/* make use of lock */
...
lock_dealloc(lock);
...
Destroy internal attributes of gen_lock_t. You must call it before deallocating a lock to ensure proper clean up (if SysV is used, it calls the functions to remove the semaphores).
The parameter is a lock initialized with lock_init(...).
Example 3.9. Example of usage
...
#include "locking.h"
...
gen_lock_t *lock;
lock = lock_alloc();
if(lock==NULL)
{
LM_ERR("cannot allocate the lock\n");
exit;
}
if(lock_init(lock)==NULL)
{
LM_ERR("cannot init the lock\n");
lock_dealloc(lock);
exit;
}
/* make use of lock */
...
lock_destroy(lock);
lock_dealloc(lock);
...
Perform set operation on a lock. If the lock is already set, the function waits until the lock is unset.
The parameter must be initialized with lock_init(...) before calling this function.
Example 3.11. Example of usage
...
#include "locking.h"
...
gen_lock_t *lock;
lock = lock_alloc();
if(lock==NULL)
{
LM_ERR("cannot allocate the lock\n");
exit;
}
if(lock_init(lock)==NULL)
{
LM_ERR("cannot init the lock\n");
lock_dealloc(lock);
exit;
}
/* make use of lock */
lock_get(lock);
/* under lock protection */
...
lock_destroy(lock);
lock_dealloc(lock);
...
Perform unset operation on a lock. If the lock is set, it will unblock it. You should call it after lock_set(...), after finishing the operations that need synchronization and racing protection.
The parameter must be initialized before calling this function.
Example 3.13. Example of usage
...
#include "locking.h"
...
gen_lock_t *lock;
lock = lock_alloc();
if(lock==NULL)
{
LM_ERR("cannot allocate the lock\n");
exit;
}
if(lock_init(lock)==NULL)
{
LM_ERR("cannot init the lock\n");
lock_dealloc(lock);
exit;
}
/* make use of lock */
lock_get(lock);
/* under lock protection */
...
lock_release(lock);
...
lock_destroy(lock);
lock_dealloc(lock);
...
The lock set is an array of gen_lock_t. To create a lock set, you have to define a variable of type gen_lock_set_t, allocate it in shared memory specifying the number of locks in the set, then initialize it. You can perform set/unset operations, providing the lock set variable and destroying the variable when you finish using it.
To use the lock sets in your C code you have to include the headers file: locking.h.
Data types and available functions to do operations with lock sets are described in the next sections.
It is the type to define a lock set variable. It must be allocated in shared memory, to be available across OPENSER processes.
Allocates a lock set in shared memory.
The parameter n specifies the number of locks in the set. Return pointer to the lock set, or NULL if the set couldn't be allocated.
Example 3.16. Example of usage
...
#include "locking.h"
...
gen_lock_set_t *set;
set = lock_set_alloc(16);
if(set==NULL)
{
LM_ERR("cannot allocate the lock set\n");
exit;
}
...
Free the memory allocated for a lock set.
The parameter has to be a lock set allocated with lock_set_alloc(...).
Example 3.18. Example of usage
...
#include "locking.h"
...
gen_lock_set_t *set;
set = lock_set_alloc(16);
if(set==NULL)
{
LM_ERR("cannot allocate the lock set\n");
exit;
}
...
lock_set_dealloc(set);
...
Initialized the internal attributes of a lock set. The lock set has to be allocated with lock_set_alloc(...). This function must be called before performing any set/unset operation on lock set.
The parameter is an allocated lock set. It returns NULL if the lock set couldn't be initialized, otherwise returns the pointer to the lock set.
Example 3.20. Example of usage
...
#include "locking.h"
...
gen_lock_set_t *set;
set = lock_set_alloc(16);
if(set==NULL)
{
LM_ERR("cannot allocate the lock set\n");
exit;
}
if(lock_set_init(set)==NULL)
{
LM_ERR("cannot initialize the lock set'n");
lock_set_dealloc(set);
exit;
}
/* make usage of lock set */
...
lock_set_dealloc(set);
...
Destroy the internal structure of the lock set. You have to call it once you finished to use the lock set. The function must be called after lock_set_init(...)
The parameter is an initialized lock set. After calling this function you should not perform anymore set/unset operations on lock set.
Example 3.22. Example of usage
...
#include "locking.h"
...
gen_lock_set_t *set;
set = lock_set_alloc(16);
if(set==NULL)
{
LM_ERR("cannot allocate the lock set\n");
exit;
}
if(lock_set_init(set)==NULL)
{
LM_ERR("cannot initialize the lock set'n");
lock_set_dealloc(set);
exit;
}
/* make usage of lock set */
...
lock_set_destroy(set);
lock_set_dealloc(set);
...
Set (block) a lock in the lock set. You should call this function after the lock set has been initialized. If the lock is already set, the function waits until that lock is unset (unblocked).
First parameter is the lock set. The second is the index withing the set of the lock to be set. First lock in set has index 0. The index parameter must be between 0 and n-1 (see lock_set_alloc(...).
Example 3.24. Example of usage
...
#include "locking.h"
...
gen_lock_set_t *set;
set = lock_set_alloc(16);
if(set==NULL)
{
LM_ERR("cannot allocate the lock set\n");
exit;
}
if(lock_set_init(set)==NULL)
{
LM_ERR("cannot initialize the lock set'n");
lock_set_dealloc(set);
exit;
}
/* make usage of lock set */
lock_set_get(set, 8);
/* under lock protection */
...
lock_set_destroy(set);
lock_set_dealloc(set);
...
Unset (unblock) a lock in the lock set.
First parameter is the lock set. The second is the index withing the set of the lock to be unset. First lock in set has index 0. The index parameter must be between 0 and n-1 (see lock_set_alloc(...).
Example 3.26. Example of usage
...
#include "locking.h"
...
gen_lock_set_t *set;
set = lock_set_alloc(16);
if(set==NULL)
{
LM_ERR("cannot allocate the lock set\n");
exit;
}
if(lock_set_init(set)==NULL)
{
LM_ERR("cannot initialize the lock set'n");
lock_set_dealloc(set);
exit;
}
/* make usage of lock set */
lock_set_get(set, 8);
/* under lock protection */
...
lock_set_release(set, 8);
...
lock_set_destroy(set);
lock_set_dealloc(set);
...
A clear sign of issues with the locking is that one or more OPENSER processes eat lot of CPU. If the traffic load does not justify such behavior and no more SIP messages are processed, the only solution is to troubleshoot and fix the locking error. The problem is that a lock is set but never unset. A typical case is when returning due to an error and forgetting to release a previously lock set.
To troubleshoot a solution is to use gdb, attach to the process that eats lot of CPU and get the backtrace. You need to get the PID of that OPENSER process - top or ps tools can be used.
... # gdb /path/to/openser PID ...
From the backtrace you should get to the lock that is set and not released. From there you should start the investigation - what are the cases to set that lock and in which circumstances it does not get unset.
OPENSER is a multi-process application, usage of shared memory is required many times. The memory manager tries to simplify the work with shared and private memory, providing a very simple programming interface. The internal design took in consideration speed optimizations, something very important for a real-time communication server.
The manager is initialized at start-up, creating the chunks for private and shared memory. It is important to know that the shared memory is not available during configuration parsing, that includes the setting of module parameters.
When the own memory manager cannot be used, OPENSER falls back to SysV shared memory system.
Shortly, the manager reserves a big chunk of system memory for itself at start-up, then it allocates parts inside the chunk.
This type of memory is specific per process, no synchronization is needed to access structures allocated in it. It should be used for variables that do not need to be visible in other OPENSER processes or for temporary operations.
To store static values in private memory and have it in all processes without the need to synchronize for accessing it, you must create it before OPENSER forks.
To use the private memory manager you have to include the file: mem/mem.h.
Allocates space in private memory.
The parameter specifies the size of memory space to be allocated. Returns the pointer to memory if the the operation succeeds, NULL otherwise.
Example 4.2. Example of usage
...
#include "mem/mem.h"
...
char *p;
p = (char*)pkg_malloc(8*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot allocate pkg memory\n");
exit;
}
...
Free allocated private memory.
The parameter is the pointer to the memory to be freed.
Example 4.4. Example of usage
...
#include "mem/mem.h"
...
char *p;
p = (char*)pkg_malloc(8*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot allocate pkg memory\n");
exit;
}
strcpy(p, "openser");
LM_DBG("string value at %p is [%s]\n", p, p);
pkg_free(p);
...
Realloc a previously allocated memory chunk. It copies the content of the old memory chunk to the new one. If the space after the old chunk is free and large enough to scale to the new size, OPENSER memory manager will set the size of the old chunk to the new size, marking properly the memory zone, in this way, the copy operation is skipped.
The first parameter is the pointer to the memory space that needs to be re-sized. The second parameter is the new size in bytes. The function return the pointer to the new memory space, or NULL if an error occurred. Beware that the returned pointer may be different than the old pointer.
Example 4.6. Example of usage
...
#include "mem/mem.h"
...
char *p;
p = (char*)pkg_malloc(8*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot allocate pkg memory\n");
exit;
}
strcpy(p, "openser");
LM_DBG("string value at %p is [%s]\n", p, p);
p = (char*)pkg_realloc(p, 16*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot re-allocate pkg memory\n");
exit;
}
strcat(p, " server");
LM_DBG("string value at %p is [%s]\n", p, p);
pkg_free(p);
...
The data stored in shared memory is visible in all OPENSER modules. It is the space where are stored user location records, the TM structures for stateful processing, routing rules for dispatcher or lcr module, and many more.
The shared memory is initialized after the config file is parsed, because it need to know the user and group OPENSER is running under, for the case when the memory manger uses SysV operations.
To use share memory functions in your C code you need to include the file: mem/shm_mem.h. When accessing share memory data, you make sure don't have a race between OPENSER processes or protect via a lock.
Allocates space in shared memory.
The parameter specifies the size in bytes of the desired shared memory space. It returns the pointer to shared memory in case of success, or NULL if an error occurred.
Example 4.8. Example of usage
...
#include "mem/shm_mem.h"
...
char *p;
p = (char*)shm_malloc(8*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot allocate shm memory\n");
exit;
}
...
Free a shared memory space previously allocated with shm_share(...).
The parameter is the pointer to the shared memory space to be freed.
Example 4.10. Example of usage
...
#include "mem/shm_mem.h"
...
char *p;
p = (char*)shm_malloc(8*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot allocate shm memory\n");
exit;
}
strcpy(p, "openser");
LM_DBG("string value at %p is [%s]\n", p, p);
shm_free(p);
...
Realloc a previously allocated shared memory chunk. It copies the content of the old memory chunk to the new one.
The first parameter is the pointer to the memory space that needs to be re-sized. The second parameter is the new size in bytes. The function return the pointer to the new memory space, or NULL if an error occurred. Beware that the returned pointer may be different than the old pointer.
Example 4.12. Example of usage
...
#include "mem/shm_mem.h"
...
char *p;
p = (char*)shm_malloc(8*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot allocate shm memory\n");
exit;
}
strcpy(p, "openser");
LM_DBG("string value at %p is [%s]\n", p, p);
p = (char*)shm_realloc(p, 16*sizeof(char));
if(p==NULL)
{
LM_ERR("cannot re-allocate shm memory\n");
exit;
}
strcat(p, " server");
LM_DBG("string value at %p is [%s]\n", p, p);
shm_free(p);
...
There are two cases of memory problems:
memory leak - allocating memory at runtime and don't free when no longer needing. It results in out of memory messages. Note that such messages might be because of too small size of the memory for the traffic, data or number of subscribers OPENSER has to handle.
memory overwriting - writing more than was allocated for that structure. It results in segmentation fault, crashing OPENSER.
OPENSER has an internal debugger for memory - it is able to show the chunks allocated in private or shared memory and the file and line from where it was allocated. In debugging mode it prints all calls for allocations and freeing memory.
To enable the memory debugger you have to recompile OPENSER. Before that you have to edit the file Makefile.defs and remove the define F_MALLOC (-DF_MALLOC) and add the define DBG_QM_MALLOC (-DDBG_QM_MALLOC).
Once compiled and installed with memory debugging you have to set memlog to a value lower than debug in configuration file. You can start OPENSER and try to reproduce the errors again. Once memory leak errors are printed you can either issue a signal USR1 to the process that printed the messages or stop OPENSER. You get in the syslog file the status of the memory. If you see memory allocation done from the same place in the sources, many times, at runtime, it is a memory leak. If not, increase the memory size to fit your load needs and run again -- if you don't get the memory leak errors it was the case of insufficient memory allocated for OPENSER.
For memory overwriting a core should be generated. If yes, you can investigate it with gdb.
... # gdb /path/to/openser corefile ...
From the backtrace you should get the file and line where the overwriting happened. In case a core is not generated, check the messages in the syslog. Look for BUG and error, for head or tail of a memory chunk being overwriting.
In this chapter we focus on most used data structures inside OPENSER sources. Most of them relate to SIP message structure. Other important data structures are explained in the chapters detailing specific components or functionalities -- for example, see Database API or Pseudo-variables chapters.
SIP is a text-based protocol, therefore lot of operations resume to text manipulation. OPENSER uses references in the SIP message body most of the time, doing it via an anchor pointer and the length. For that it uses the str structure.
The str structure is defined in file str.h.
Example 5.1. Definition
...
struct _str{
char* s; /* pointer to the beginning of string (char array) */
int len; /* string length */
};
typedef struct _str str;
...
Example 5.2. Example of usage
...
#include "str.h"
...
str s;
s.s = "openser";
s.len = strlen(s.len);
LM_DBG("the string is [%.*s]\n", s.len, s.s);
...
This is the structure holding a parsed SIP URI. You can fill it by calling parse_uri(...) function.
The structure is defined in file parser/msg_parser.h.
Example 5.3. Definition
...
struct sip_uri {
str user; /* Username */
str passwd; /* Password */
str host; /* Host name */
str port; /* Port number */
str params; /* URI Parameters */
str headers; /* URI Headers */
unsigned short port_no; /* Port number r*/
unsigned short proto; /* Transport protocol */
uri_type type; /* URI scheme */
/* parameters */
str transport; /* transport parameter */
str ttl; /* ttl parameter */
str user_param; /* user parameter */
str maddr; /* maddr parameter */
str method; /* method parameter */
str lr; /* lr parameter */
str r2; /* specific rr parameter */
/* values */
str transport_val; /* value of transport parameter */
str ttl_val; /* value of ttl parameter */
str user_param_val; /* value of user parameter */
str maddr_val; /* value of maddr parameter */
str method_val; /* value of method parameter */
str lr_val; /* value of lr parameter */
str r2_val; /* value of r2 parameter */
};
...
Members of the structure corresponds to a part of a SIP URI. To get details about the format of SIP URI read RFC3261. Example of SIP URI:
sip:alice@sipserver.org:5060;transport=tcp
This is the main structure related to a SIP message. When a SIP message is received from the network, it is parsed in such structure. The pointer to this structure is given as parameter to all functions exported by modules to be used in the configuration file.
The structure is defined in file parser/msg_parser.h.
Example 5.4. Definition
...
struct sip_msg {
unsigned int id; /* message id, unique/process*/
struct msg_start first_line; /* Message first line */
struct via_body* via1; /* The first via */
struct via_body* via2; /* The second via */
struct hdr_field* headers; /* All the parsed headers*/
struct hdr_field* last_header; /* Pointer to the last parsed header*/
hdr_flags_t parsed_flag; /* Already parsed header field types */
/* Via, To, CSeq, Call-Id, From, end of header*/
/* pointers to the first occurrences of these headers;
* everything is also saved in 'headers' (see above)
*/
/* shorcuts to known headers */
struct hdr_field* h_via1;
struct hdr_field* h_via2;
struct hdr_field* callid;
struct hdr_field* to;
struct hdr_field* cseq;
struct hdr_field* from;
struct hdr_field* contact;
struct hdr_field* maxforwards;
struct hdr_field* route;
struct hdr_field* record_route;
struct hdr_field* path;
struct hdr_field* content_type;
struct hdr_field* content_length;
struct hdr_field* authorization;
struct hdr_field* expires;
struct hdr_field* proxy_auth;
struct hdr_field* supported;
struct hdr_field* proxy_require;
struct hdr_field* unsupported;
struct hdr_field* allow;
struct hdr_field* event;
struct hdr_field* accept;
struct hdr_field* accept_language;
struct hdr_field* organization;
struct hdr_field* priority;
struct hdr_field* subject;
struct hdr_field* user_agent;
struct hdr_field* content_disposition;
struct hdr_field* accept_disposition;
struct hdr_field* diversion;
struct hdr_field* rpid;
struct hdr_field* refer_to;
struct hdr_field* session_expires;
struct hdr_field* min_se;
struct hdr_field* ppi;
struct hdr_field* pai;
struct hdr_field* privacy;
struct sdp_info* sdp; /* parsed SDP body */
char* eoh; /* pointer to the end of header (if found) or null */
char* unparsed; /* here we stopped parsing*/
struct receive_info rcv; /* source and dest ip, ports, proto a.s.o*/
char* buf; /* scratch pad, holds a unmodified message,
* via, etc. point into it */
unsigned int len; /* message len (orig) */
/* modifications */
str new_uri; /* changed first line uri, when you change this
* don't forget to set parsed_uri_ok to 0 */
str dst_uri; /* Destination URI, must be forwarded to this URI if len!=0 */
/* current uri */
int parsed_uri_ok; /* 1 if parsed_uri is valid, 0 if not, set it to 0
if you modify the uri (e.g change new_uri)*/
struct sip_uri parsed_uri; /* speed-up > keep here the parsed uri*/
/* the same for original uri */
int parsed_orig_ruri_ok;
struct sip_uri parsed_orig_ruri;
struct lump* add_rm; /* used for all the forwarded requests/replies */
struct lump* body_lumps; /* Lumps that update Content-Length */
struct lump_rpl *reply_lump; /* only for localy generated replies !!!*/
/* whatever whoever want to append to branch comes here */
char add_to_branch_s[MAX_BRANCH_PARAM_LEN];
int add_to_branch_len;
/* index to TM hash table; stored in core to avoid
* unnecessary calculations */
unsigned int hash_index;
/* flags used from script */
flag_t flags;
/* flags used by core - allows to set various flags on the message; may
* be used for simple inter-module communication or remembering
* processing state reached */
unsigned int msg_flags;
str set_global_address;
str set_global_port;
/* force sending on this socket */
struct socket_info* force_send_socket;
/* create a route HF out of this path vector */
str path_vec;
};
...
To fill such structure you can use function parse_msg(...) giving a buffer containing raw text of a SIP message. Most of the attributes in this structure point directly inside the SIP message buffer.
Example of a SIP message:
... REGISTER sip:sip.test.com SIP/2.0 Via: SIP/2.0/UDP 192.168.1.3:5061;branch=z9hG4bK-d663b80b Max-Forwards: 70 From: user <sip:u123@sip.test.com>;tag=ea8cef4b108a99bco1 To: user <sip:u123@sip.test.com> Call-ID: b96fead3-f03493d4@xyz CSeq: 3720 REGISTER Contact: user <sip:u123@192.168.1.3:5061>;expires=3600 User-Agent: Linksys/RT31P2-2.0.10(LIc) Content-Length: 0 Allow: ACK, BYE, CANCEL, INFO, INVITE, NOTIFY, OPTIONS, REFER Supported: x-sipura ...
The structure corresponds to a parsed representation of the first line in a SIP message. It is defined in file parser/parse_fline.h.
Example 5.5. Definition
...
struct msg_start {
int type; /* Type of the Message - Request or Response (Reply) */
int len; /* length including delimiter */
union {
struct {
str method; /* Method string */
str uri; /* Request URI as raw string */
str version; /* SIP version */
int method_value; /* Internal integer representation of SIP method */
} request;
struct {
str version; /* SIP version */
str status; /* Reply status */
str reason; /* Reply reason phrase */
unsigned int statuscode; /* Integer representation of reply status */
} reply;
}u;
};
...
To parse a buffer containing the first line of a SIP message you have to use the function parse_fline(...).
The structure holding a parsed SIP header. It is defined in file parser/hf.h.
Example 5.6. Definition
...
struct hdr_field {
hdr_types_t type; /* Header field type */
str name; /* Header field name */
str body; /* Header field body (may not include CRLF) */
int len; /* length from hdr start until EoHF (incl.CRLF) */
void* parsed; /* Parsed data structures */
struct hdr_field* next; /* Next header field in the list */
struct hdr_field* sibling; /* Next header of same type */
};
...
To parse specific headers in a SIP message you have to use the function parse_headers(...). The function takes as parameter a bitmask flag that can specify what headers you need to be parsed. For example, to parse the From and To headers:
parse_headers(msg, HDR_FROM_F|HDR_TO_F, 0);
To optimize the operations with headers, an integer value is assigned to most used headers. This value is stored in attribute type. Here is the list with the values for header type:
...
enum _hdr_types_t {
HDR_ERROR_T = -1 /* Error while parsing */,
HDR_OTHER_T = 0 /* Some other header field */,
HDR_VIA_T = 1 /* Via header field */,
HDR_VIA1_T = 1 /* First Via header field */,
HDR_VIA2_T = 2 /* only used as flag */,
HDR_TO_T /* To header field */,
HDR_FROM_T /* From header field */,
HDR_CSEQ_T /* CSeq header field */,
HDR_CALLID_T /* Call-Id header field */,
HDR_CONTACT_T /* Contact header field */,
HDR_MAXFORWARDS_T /* MaxForwards header field */,
HDR_ROUTE_T /* Route header field */,
HDR_RECORDROUTE_T /* Record-Route header field */,
HDR_PATH_T /* Path header fiels */,
HDR_CONTENTTYPE_T /* Content-Type header field */,
HDR_CONTENTLENGTH_T /* Content-Length header field */,
HDR_AUTHORIZATION_T /* Authorization header field */,
HDR_EXPIRES_T /* Expires header field */,
HDR_PROXYAUTH_T /* Proxy-Authorization hdr field */,
HDR_SUPPORTED_T /* Supported header field */,
HDR_PROXYREQUIRE_T /* Proxy-Require header field */,
HDR_UNSUPPORTED_T /* Unsupported header field */,
HDR_ALLOW_T /* Allow header field */,
HDR_EVENT_T /* Event header field */,
HDR_ACCEPT_T /* Accept header field */,
HDR_ACCEPTLANGUAGE_T /* Accept-Language header field */,
HDR_ORGANIZATION_T /* Organization header field */,
HDR_PRIORITY_T /* Priority header field */,
HDR_SUBJECT_T /* Subject header field */,
HDR_USERAGENT_T /* User-Agent header field */,
HDR_ACCEPTDISPOSITION_T /* Accept-Disposition hdr field */,
HDR_CONTENTDISPOSITION_T /* Content-Disposition hdr field */,
HDR_DIVERSION_T /* Diversion header field */,
HDR_RPID_T /* Remote-Party-ID header field */,
HDR_REFER_TO_T /* Refer-To header fiels */,
HDR_SESSION_EXPIRES_T /* Session-Expires header field */,
HDR_MIN_SE_T /* Min-SE header field */,
HDR_PPI_T /* P-Preferred-Identity header field */,
HDR_PAI_T /* P-Asserted-Identity header field */,
HDR_PRIVACY_T /* Privacy header field */,
HDR_RETRY_AFTER_T /* Retry-After header field */,
HDR_EOH_T /* Some other header field */
};
...
If the type of hdr_field structure is HDR_TO_T it is the parsed To header.
The attribute parsed may hold the parsed representation of the header body. For example, for Content-Lenght header it contains the content length value as integer.
The structure holds a parsed To header. Same structure is used for From header and the other headers that have same structure conform to IETF RFCs. The structure is defined in file parser/parse_to.h.
Example 5.7. Definition
...
struct to_body{
int error; /* Error code */
str body; /* The whole header field body */
str uri; /* URI withing the body of the header */
str display; /* Display Name */
str tag_value; /* Value of tag parameter*/
struct sip_uri parsed_uri; /* Parsed URI */
struct to_param *param_lst; /* Linked list of parameters */
struct to_param *last_param; /* Last parameter in the list */
};
...
To parse a To header you have to use function parse_to(...).
The structure holds a parsed Via header. It is defined in file parse_via.h.
Example 5.8. Definition
...
struct via_body {
int error; /* set if an error occurred during parsing */
str hdr; /* header name "Via" or "v" */
str name; /* protocol name */
str version; /* protocol version */
str transport; /* transport protocol */
str host; /* host part of Via header */
unsigned short proto; /* transport protocol as integer*/
unsigned short port; /* port number as integer */
str port_str; /* port number as string*/
str params; /* parameters */
str comment; /* comment */
unsigned int bsize; /* body size, not including hdr */
struct via_param* param_lst; /* list of parameters*/
struct via_param* last_param; /*last via parameter, internal use*/
/* shortcuts to "important" params*/
struct via_param* branch; /* branch parameter */
str tid; /* transaction id, part of branch */
struct via_param* received; /* received parameter */
struct via_param* rport; /* rport parameter */
struct via_param* i; /* i parameter */
struct via_param* alias; /* alias see draft-ietf-sip-connect-reuse-00 */
struct via_param* maddr; /* maddr parameter */
struct via_body* next; /* pointer to next via body string if compact Via or null */
};
...
The str attributes in the structure are referenced to SIP message buffer. To parse a Via header you have to use the function parse_via(...).
OPENSER includes its own implementation of SIP parser. It is known as lazy or incremental parser. That means it parses until it founds the required elements or encounters ends of SIP message.
All parsing functions and data structures are in the files from the directory parser. The main file for SIP message parsing is parser/msg_parser.c with the corresponding header file parser/msg_parser.h.
It does not parse entirely the parts of the SIP message. For most of the SIP headers, it identifies the name and body, it does not parse the content of header's body. It may happen that a header is malformed and OPENSER does not report any error as there was no request to parse that header body. However, most used headers are parsed entirely by default. Such headers are top most Via, To, CSeq, Content-Lenght.
The parser does not duplicate the values, it makes references inside the SIP message buffer it parses. For the parsed structures it allocates private memory. It keeps the state of parsing, meaning that it has an anchor to the beginning of unparsed part of the SIP message and stores a bitmask of flags with parsed known headers. If the function to parse a special header is called twice, the second time will return immediately as it finds in the bitmask that the header was already parsed.
This chapter in not intended to present all parsing functions, you have to check the files in the directory parser. There is kind of naming convention, so if you need the function to parse the header XYZ, look for the files parser/parse_XYZ.{c,h}. If you don't find it, second try is to use ctags to locate a parse_XYZ(...) function. If no luck, then ask on OPENSER development mainling list devel@lists.openser.org. Final solution in case on negative answer is to implement it yourself. For example, CSeq header parser is in parser/parse_cseq.{c,h}.
The next sections will present the parsing functions that give access to the most important parts of a SIP message.
A developer does not interfere too much with this function as it is called automatically by OPENSER when a SIP message is received from the network.
You can use it if you load the content of SIP message from a file or database, or you received on different channels, up to your extension implementation. You should be aware that it is not enough to call this function and then run the actions from the configuration file. There are some attributes in the structure sip_msg that are specific to the environment: received socket, source IP and port, ...
Return 0 if parsing was OK, >0 if error occurred.
Example 6.4. Example of usage
...
str msg_buf;
struct sip_msg msg;
...
msg_buf.s = "INVITE sip:user@sipserver.com SIP/2.0\r\nVia: ...";
msg_buf.len = strlen(msg_buf.s);
if (parse_msg(buf,len, &msg)!=0) {
LM_ERR("parse_msg failed\n");
return -1;
}
if(msg.first_line.type==SIP_REQUEST)
LM_DBG("SIP method is [%.*s]\n", msg.first_line.u.request.method.len,
msg.first_line.u.request.method.s);
...
Parse the SIP message until the headers specified by parameter flags flags are found. The parameter next can be used when a header can occur many times in a SIP message, to continue parsing until a new header of that type is found.
The values that can be used for flags are defined in parser/hf.h.
When searching to get a specific header, all the headers encountered during parsing are hooked in the structure sip_msg.
Return 0 if parsing was sucessful, >0 in case of error.
Example 6.6. Example of usage
...
if(parse_headers(msg, HDR_CALLID_F, 0)!=0)
{
LM_ERR("error parsing CallID header\n");
return -1;
}
...
Parse a buffer that contains the body of a To header. The function is defined in parser/parse_to.h.
Return 0 in case of success, >0 in case of error.
The next example shows the parse_from(...) function that makes use of parse_to(...) to parse the body of header From. The function is located in file parser/parse_from.c.
The structure filled at parsing is hooked in the structure sip_msg, inside the attribute from, which is the shortcut to the header From.
Example 6.8. Example of usage
...
int parse_from_header( struct sip_msg *msg)
{
struct to_body* from_b;
if ( !msg->from && ( parse_headers(msg,HDR_FROM_F,0)==-1 || !msg->from)) {
LM_ERR("bad msg or missing FROM header\n");
goto error;
}
/* maybe the header is already parsed! */
if (msg->from->parsed)
return 0;
/* first, get some memory */
from_b = pkg_malloc(sizeof(struct to_body));
if (from_b == 0) {
LM_ERR("out of pkg_memory\n");
goto error;
}
/* now parse it!! */
memset(from_b, 0, sizeof(struct to_body));
parse_to(msg->from->body.s,msg->from->body.s+msg->from->body.len+1,from_b);
if (from_b->error == PARSE_ERROR) {
LM_ERR("bad from header\n");
pkg_free(from_b);
set_err_info(OSER_EC_PARSER, OSER_EL_MEDIUM, "error parsing From");
set_err_reply(400, "bad From header");
goto error;
}
msg->from->parsed = from_b;
return 0;
error:
return -1;
}
...
Next example shows how to get the content of SIP message body in a 'str' variable.
Example 6.9. Example of usage
...
int get_msg_body(struct sip_msg *msg, str *body)
{
/* 'msg' is a pointer to a valid struct sip_msg */
/* get message body
- after that whole SIP MESSAGE is parsed
- calls internally parse_headers(msg, HDR_EOH_F, 0)
*/
body->s = get_body( msg );
if (body->s==0)
{
LM_ERR("cannot extract body from msg\n");
return -1;
}
/* content-length (if present) must be already parsed */
if (!msg->content_length)
{
LM_ERR("no Content-Length header found!\n");
return -1;
}
body->len = get_content_length( msg );
return 0;
}
...
You can see in the next example how to access the body of header Call-ID.
Example 6.10. Example of usage
...
void print_callid_header(struct sip_msg *msg)
{
if(msg==NULL)
return;
if(parse_headers(msg, HDR_CALLID_F, 0)!=0)
{
LM_ERR("error parsing CallID header\n");
return;
}
if(msg->callid==NULL || msg->callid->body.s==NULL)
{
LM_ER("NULL call-id header\n");
return;
}
LM_INFO("Call-ID: %.*s\n", msg->callid->body.len, msg->callid->body.s);
}
...
The section gives the guidelines to add a new function for parsing a header.
add source and header files in directory parser naming them parse_hdrname.{c,h}.
if the header is used very often, consider doing speed optimization by allocating a header type and flag. That will allow to identify the header via integer comparison after the header was parsed and added in headers list in the structure sip_msg.
make sure you add the code to properly clean up the header structure when the structure sip_msg is freed.
make sure that the tm module properly clones the header or it resets the pointers to the header when copying the structure sip_msg in shared memory.
As a developer, the interaction with the transport layer is lower and lower. It is already implemented the support for UDP, TCP, TLS and SCTP. From the modules, you can use the API exported by sl and tm modules to send stateless replies, or to send stateful requests/replies. Sending stateless requests can be done with the functions from core, exported in file forward.h.
The core takes care of receiving the messages from the network, the basic validation for them, preparing the environment for a higher level processing of SIP messages. When developing new extensions, you don't have to care about reading/writing from/to network.
If you want to investigate the implementation of transport layers, you can start with:
udp_*.{c,h} for UDP
tcp_*.{c,h} for TCP
tls/*.{c,h} for TLS
sctp_*.{c,h} for SCTP
flex and bison are used to parse the configuration file and build the actions tree that are executed at run time for each SIP message. Bison is the GNU implementation compatible with Yacc (Yet Another Compiler Compiler), but also Yacc or Byacc can be used instead of it.
Extending the configuration file can be done by adding a new core parameter or a new core functions. Apart of these, one can add new routing blocks, keywords or init and runtime instructions.
The config file include two main categories of statements:
init statements - this category includes setting the global parameters, loading modules and setting module parameters. These statements are executed only one time, at startup.
runtime statements - this category includes the actions executed many times, after OPENSER initialization. These statements are grouped in route blocks, there being different route types that get executed for certain events.
route - is executed when a SIP request is received
onreply_route - is executed when a SIP reply is received
error_route - is executed when some errors (mainly related to message parsing) are encountered
failure_route - is executed when a negative reply was received for a transaction that armed the handler for the failure event.
branch_route - is executed for each destination branch of the transaction that armed the handler for it.
In the next section we will present how to add a core parameter and add a routing action -- core function. You need to have knowledge of flex and bison.
Some of the core parameters correspond to global variables in OPENSER sources. Others induce actions to be taken during statup.
Let's follow step by step the definition and usage of the core parameter log_name. It is a string parameter that specifies the value to be printed as application name in syslog messages.
First is the declaration of the variable in the C code. The log_name is defined in main.c and initialized to 0 (when set to o, it is printed the OPENSER name (including path) in the syslog).
... char *log_name = 0; ...
Next is to define the token in the flex file: cfg.lex.
... LOGNAME log_name ...
The association of a token ID and extending the grammar of the configuration file is done in the bison file: cfg.y.
...
%token LOGNAME
...
assign_stm: ...
| LOGNAME EQUAL STRING { log_name=$3; }
| LOGNAME EQUAL error { yyerror("string value expected"); }
...
The grammar was extended with a new assign statement, that allows to write in the configuration file an expression like:
... log_name = "openser123" ...
When having a line like above one in the configuration file, the variable log_name in C code will be initialized to the string in the right side of the equal operator.
To introduce new functions in OPENSER core that are exported to the configuration file the grammar have to be extended (bison and flex files), the interpreter must be enhanced to be able to run the new actions.
Behind each core function resides an action structure. This data type is defined in route_struct.h:
...
typedef struct action_elem_ {
int type; /* type of the element */
union {
long number; /* element is a long number */
char* string; /* element is a null-terminated string */
void* data; /* element is pointer to a custom type data */
str s; /* element is str value */
pv_spec_t* item; /* element is a pseudo-variable */
} u;
} action_elem_t, *action_elem_p;
/* increase MAX_ACTION_ELEMS to support more module function parameters
if you change this define, you need also to change the assignment in
the action.c file */
#define MAX_ACTION_ELEMS 7
struct action{
int type; /* type of action: forward, drop, log, send ...*/
action_elem_t elem[MAX_ACTION_ELEMS]; /* the elements of this action: parameters, etc. */
int line; /* line in cofig file where the action is used */
struct action* next; /* next action in config file */
};
...
Each action is identified by a type. The types of actions are defined in same header file. For example, the strip(...) function has the type STRIP_T, the functions exported by modules have the type MODULE_T.
To each action may be given a set of parameters, so called action elements. In case of functions exported by modules, the first element is the pointer to the function, next are the parameters given to that function in configuration file.
For debugging and error detection, the action keeps the line number in configuration file where it is used.
Next we discuss how setflag(...) config function was implemented.
Define the token in flex file: cfg.lex.
... SETFLAG "setflag" ...
Assign a token ID and extend the bison grammar.
...
%token SETFLAG
...
cmd: ...
| SETFLAG LPAREN NUMBER RPAREN {mk_action2($$, SETFLAG_T, NUMBER_ST, 0,
(void *)$3, 0 ); }
| SETFLAG error { $$=0; yyerror("missing '(' or ')'?"); }
...
First grammar specification says that setflag(...) can have one parameter of type number. The other rule for grammar is to detect error cases.
First step is to add a new action type in route_struct.h.
Then add a new case in the switch of action types, file action.c, function
... case SETFLAG_T: ret = setflag( msg, a->elem[0].u.number ); break; ...
The C function setflag(...) is defined and implemented in flags.{c,h}. It simply sets the bit in flags attribute of sip_msg at the position given by the parameter.
...
int setflag( struct sip_msg* msg, flag_t flag ) {
msg->flags |= 1 << flag;
return 1;
}
...
We are not done yet. OPENSER does a checking of the actions tree after all configuration file was loaded. It does sanity checks and optimization for run time. For our case, it does a double-check that the parameter is a number and it is in the range of 0...31 to fit in the bits size of an integer value. See function fix_actions(...) in route.c.
...
case SETFLAG_T:
case RESETFLAG_T:
case ISFLAGSET_T:
if (t->elem[0].type!=NUMBER_ST) {
LM_CRIT("bad xxxflag() type %d\n", t->elem[0].type );
ret=E_BUG;
goto error;
}
if (!flag_in_range( t->elem[0].u.number )) {
ret=E_CFG;
goto error;
}
break;
...
Last thing you have to add is to complete the function print(action(...) with a new case for your action that will be used to print the actions tree -- for debugging purposes. See it in file route_struct.c.
...
case SETFLAG_T:
LM_DBG("setflag(");
break;
...
From now on, you can use in your configuration file the function setflag(_number_).
Don't forget to add documentation in OpenSER Core Cookbook.
Internally, OPENSER uses a reduced set of SQL operations to access the records on the storage system. This allowed to write DB driver modules for non-SQL storage systems, such as db_text -- a tiny DB engine using text files.
Therefore, the interface provides data types and functions that are independent of underlying DB storage. A DB driver module has to implement the functions specified by the interface and provide a function named db_bind_api(...) to link to the interface functions.
The DB interface is implemented in the db directory. To use it, one has to include the file db/db.h.
It is the structure that gets filled when binding to a DB driver module. It links to the interface functions implemented in the module.
Example 9.1. Definition
...
typedef struct db_func {
unsigned int cap; /* Capability vector of the database transport */
db_use_table_f use_table; /* Specify table name */
db_init_f init; /* Initialize database connection */
db_close_f close; /* Close database connection */
db_query_f query; /* Query a table */
db_fetch_result_f fetch_result; /* Fetch result */
db_raw_query_f raw_query; /* Raw query - SQL */
db_free_result_f free_result; /* Free a query result */
db_insert_f insert; /* Insert into table */
db_delete_f delete; /* Delete from table */
db_update_f update; /* Update table */
db_replace_f replace; /* Replace row in a table */
db_last_inserted_id_f last_inserted_id; /* Retrieve the last inserted ID
in a table */
db_insert_update_f insert_update; /* Insert into table, update on duplicate key */
} db_func_t;
...
The attribute cap is a bitmask of implemented functions, making easy to detect the capabilities of the DB driver module. A module using the DB API should check at startup that the DB driver configured to be used has the required capabilities. For example, msilo module need select, delete and insert capabilities. The flags for capabilities are enumerated in the next figure (located in db/db_cap.h).
...
typedef enum db_cap {
DB_CAP_QUERY = 1 << 0, /* driver can perform queries */
DB_CAP_RAW_QUERY = 1 << 1, /* driver can perform raw queries */
DB_CAP_INSERT = 1 << 2, /* driver can insert data */
DB_CAP_DELETE = 1 << 3, /* driver can delete data */
DB_CAP_UPDATE = 1 << 4, /* driver can update data */
DB_CAP_REPLACE = 1 << 5, /* driver can replace (also known as INSERT OR UPDATE) data */
DB_CAP_FETCH = 1 << 6, /* driver supports fetch result queries */
DB_CAP_LAST_INSERTED_ID = 1 << 7, /* driver can return the ID of the last insert operation */
DB_CAP_INSERT_UPDATE = 1 << 8 /* driver can insert data into database and update on duplicate */
} db_cap_t;
...
Parse the DB URL and open a new connection to database.
Parameters:
_url - database URL. Its format depends on DB driver. For an SQL server like MySQL has to be: mysql://username:password@server:port/database. For db_text has to be: text:///path/to/db/directory.
The function returns pointer to db_con_t* representing the connection structure or NULL in case of error.
The function closes previously open connection and frees all previously allocated memory. The function db_close must be the very last function called.
Parameters:
_h - db_con_t structure representing the database connection.
The function returns nothing.
Specify table name that will be used for subsequent operations (insert, delete, update, query).
Parameters:
_h - database connection handle.
_t - table name.
The function 0 if everything is OK, otherwise returns value negative value.
Query table for specified rows. This function implements the SELECT SQL directive.
Example 9.5. Function type
... typedef int (*db_query_f) (const db_con_t* _h, const db_key_t* _k, const db_op_t* _op, const db_val_t* _v, const db_key_t* _c, const int _n, const int _nc, const db_key_t _o, db_res_t** _r); ...
Parameters:
_h - database connection handle.
_k - array of column names that will be compared and their values must match.
_op - array of operators to be used with key-value pairs.
_v - array of values, columns specified in _k parameter must match these values.
_c - array of column names that you are interested in.
_n - number of key-value pairs to match in _k and _v parameters.
_nc - number of columns in _c parameter.
_o - order by statement for query.
_r - address of variable where pointer to the result will be stored.
The function 0 if everything is OK, otherwise returns value negative value.
Note: If _k and _v parameters are NULL and _n is zero, you will get the whole table. If _c is NULL and _nc is zero, you will get all table columns in the result. Parameter _r will point to a dynamically allocated structure, it is neccessary to call db_free_result function once you are finished with the result. If _op is 0, equal (=) will be used for all key-value pairs comparisons. Strings in the result are not duplicated, they will be discarded if you call. Make a copy of db_free_result if you need to keep it after db_free_result. You must call db_free_result before you can call db_query again!
Fetch a number of rows from a result.
Example 9.6. Function type
... typedef int (*db_fetch_result_f) (const db_con_t* _h, db_res_t** _r, const int _n); ...
Parameters:
_h - database connection handle.
_r - structure for the result.
_n - the number of rows that should be fetched.
The function 0 if everything is OK, otherwise returns value negative value.
This function can be used to do database specific queries. Please use this function only if needed, as this creates portability issues for the different databases. Also keep in mind that you need to escape all external data sources that you use. You could use the escape_common and unescape_common functions in the core for this task.
Example 9.7. Function type
... typedef int (*db_raw_query_f) (const db_con_t* _h, const str* _s, db_res_t** _r); ...
Parameters:
_h - database connection handle.
_s - the SQL query.
_r - structure for the result.
The function 0 if everything is OK, otherwise returns value negative value.
Free a result allocated by db_query.
Parameters:
_h - database connection handle.
_r - pointer to db_res_t structure to destroy.
The function 0 if everything is OK, otherwise returns value negative value.
Insert a row into the specified table.
Example 9.9. Function type
... typedef int (*db_insert_f) (const db_con_t* _h, const db_key_t* _k, const db_val_t* _v, const int _n); ...
Parameters:
_h - database connection handle.
_k - array of keys (column names).
_v - array of values for keys specified in _k parameter.
_n - number of keys-value pairs int _k and _v parameters.
The function 0 if everything is OK, otherwise returns value negative value.
Delete a row from the specified table.
Example 9.10. Function type
... typedef int (*db_delete_f) (const db_con_t* _h, const db_key_t* _k, const db_op_t* _o, const db_val_t* _v, const int _n); ...
Parameters:
_h - database connection handle.
_k - array of keys (column names) that will be matched.
_o - array of operators to be used with key-value pairs.
_v - array of values that the row must match to be deleted.
_n - number of keys-value pairs int _k and _v parameters.
The function 0 if everything is OK, otherwise returns value negative value.
Update some rows in the specified table.
Example 9.11. Function type
... typedef int (*db_update_f) (const db_con_t* _h, const db_key_t* _k, const db_op_t* _o, const db_val_t* _v, const db_key_t* _uk, const db_val_t* _uv, const int _n, const int _un); ...
Parameters:
_h - database connection handle.
_k - array of keys (column names) that will be matched.
_o - array of operators to be used with key-value pairs.
_v - array of values that the row must match to be modified.
_uk - array of keys (column names) that will be modified.
_uv - new values for keys specified in _k parameter.
_n - number of key-value pairs in _v parameters.
_un - number of key-value pairs in _uv parameters.
The function 0 if everything is OK, otherwise returns value negative value.
Insert a row and replace if one already exists.
Example 9.12. Function type
... typedef int (*db_replace_f) (const db_con_t* handle, const db_key_t* keys, const db_val_t* vals, const int n); ...
Parameters:
_h - database connection handle.
_k - array of keys (column names).
_v - array of values for keys specified in _k parameter.
_n - number of keys-value pairs int _k and _v parameters.
The function 0 if everything is OK, otherwise returns value negative value.
Retrieve the last inserted ID in a table.
Parameters:
_h - structure representing database connection
The function returns the ID as integer or returns 0 if the previous statement does not use an AUTO_INCREMENT value.
Insert a row into specified table, update on duplicate key.
Example 9.14. Function type
... typedef int (*db_insert_update_f) (const db_con_t* _h, const db_key_t* _k, const db_val_t* _v, const int _n); ...
Parameters:
_h - database connection handle.
_k - array of keys (column names).
_v - array of values for keys specified in _k parameter.
_n - number of keys-value pairs int _k and _v parameters.
The function 0 if everything is OK, otherwise returns value negative value.
This type represents a database key (column). Every time you need to specify a key value, this type should be used.
This type represents an expression operator uses for SQL queries.
Each cell in a database table can be of a different type. To distinguish among these types, the db_type_t enumeration is used. Every value of the enumeration represents one datatype that is recognized by the database API.
Example 9.17. Definition
...
typedef enum {
DB_INT, /* represents an 32 bit integer number */
DB_DOUBLE, /* represents a floating point number */
DB_STRING, /* represents a zero terminated const char* */
DB_STR, /* represents a string of 'str' type */
DB_DATETIME, /* represents date and time */
DB_BLOB, /* represents a large binary object */
DB_BITMAP /* an one-dimensional array of 32 flags */
} db_type_t;
...
This structure represents a value in the database. Several datatypes are recognized and converted by the database API. These datatypes are automatically recognized, converted from internal database representation and stored in the variable of corresponding type. Modules that want to use this values needs to copy them to another memory location, because after the call to free_result there are not more available. If the structure holds a pointer to a string value that needs to be freed because the module allocated new memory for it then the free flag must be set to a non-zero value. A free flag of zero means that the string data must be freed internally by the database driver.
Example 9.18. Definition
...
typedef struct {
db_type_t type; /* Type of the value */
int nul; /* Means that the column in database has no value */
int free; /* Means that the value should be freed */
/** Column value structure that holds the actual data in a union. */
union {
int int_val; /* integer value */
double double_val; /* double value */
time_t time_val; /* unix time_t value */
const char* string_val; /* zero terminated string */
str str_val; /* str type string value */
str blob_val; /* binary object data */
unsigned int bitmap_val; /* Bitmap data type */
} val;
} db_val_t;
...
This structure represents a database connection, pointer to this structure are used as a connection handle from modules uses the db API.
Example 9.19. Definition
...
typedef struct {
const str* table; /* Default table that should be used */
unsigned long tail; /* Variable length tail, database module specific */
} db_con_t;
...
Structure holding the result of a query table function. It represents one row in a database table. In other words, the row is an array of db_val_t variables, where each db_val_t variable represents exactly one cell in the table.
Example 9.20. Definition
...
typedef struct db_row {
db_val_t* values; /* Columns in the row */
int n; /* Number of columns in the row */
} db_row_t;
...
This type represents a result returned by db_query function (see below). The result can consist of zero or more rows (see db_row_t description).
Note: A variable of type db_res_t returned by db_query function uses dynamically allocated memory, don't forget to call db_free_result if you don't need the variable anymore. You will encounter memory leaks if you fail to do this! In addition to zero or more rows, each db_res_t object contains also an array of db_key_t objects. The objects represent keys (names of columns).
Example 9.21. Definition
...
typedef struct db_res {
struct {
db_key_t* names; /* Column names */
db_type_t* types; /* Column types */
int n; /* Number of columns */
} col;
struct db_row* rows; /* Rows */
int n; /* Number of rows in current fetch */
int res_rows; /* Number of total rows in query */
int last_row; /* Last row */
} db_res_t;
...
The DB API offers a set of macros to make easier to access the attributes in various data structures.
Macros for db_res_t:
RES_NAMES(res) - get the pointer to column names
RES_COL_N(res) - get the number of columns
RES_ROWS(res) - get the pointer to rows
RES_ROW_N(res) - get the number of ro