Under the process model of nginx,Similar traffic statistics, flow control, data sharing, etc. require multiple workers to work together to complete the task.Shared memory is an important process communication scheme.This article describes the functions related to shared memory in nginx code.Including the use and precautions of ngx_shmem and ngx_slab,It does not include the memory management algorithms implemented in ngx_slab.

Uses of ngx_shmem

The ngx_shmem.c/h file is simply a wrapper around the mmap ()/munmap () system call or shmget ()/shmdt ().Implemented the ngx-style base library,Can apply for and release a continuous shared memory space.Generally used for fixed-length shared data,The fixed data length during use will not expand or contract.

typedef struct {
  u_char * addr;
  size_t size;
} ngx_shm_t;
ngx_int_t ngx_shm_alloc (ngx_shm_t * shm);
void ngx_shm_free (ngx_shm_t * shm);

The process of using shared memory in ngxin,Generally created by the master process,The worker process obtains the memory pointer through inheritance.

Regarding the use of ngx_shmem, you can refer to some snippets in ngx_event_module_init ().This part of the code creates several variables in shared memory,Used to record the number of requests for each state (accepted/reading/writing ...),Several key event entries in the ngx_event_module are used to add and subtract statistics on these variables.Achieve statistics on the current request status of all worker processes.

ngx_str_set (&shm.name, "nginx_shared_zone");
if (ngx_shm_alloc (&shm)!=ngx_ok) {
  return ngx_error;
ngx_stat_accepted=(ngx_atomic_t *) (shared + 3 * cl);
ngx_stat_handled=(ngx_atomic_t *) (shared + 4 * cl);
ngx_stat_requests=(ngx_atomic_t *) (shared + 5 * cl);
ngx_stat_active=(ngx_atomic_t *) (shared + 6 * cl);
ngx_stat_reading=(ngx_atomic_t *) (shared + 7 * cl);
ngx_stat_writing=(ngx_atomic_t *) (shared + 8 * cl);
ngx_stat_waiting=(ngx_atomic_t *) (shared + 9 * cl);

More details about this feature,You can view the ngx_stat_stub macro definition related code and ngx_http_stub_status_module in the code.

Use of ngx_slab

ngx_shmem is a minimalistic package,Implemented the basic functions of shared memory.But most scenes in our program share data without a fixed-size structure.And more are data structures such as ngx_array, ngx_list, ngx_queue, ngx_rbtree that can vary in size.

We expect to have a memory pool that can dynamically apply to free up space like ngx_pool_t.ngx_slab is just such a structure,In principle, the familiarity with the malloc () of the system is the application and release of a piece of memory segment through a series of algorithms.It is just that ngx_slab operates on shared memory based on ngx_shmem.

First look at the interface of ngx_slab

typedef struct {
  ngx_shmtx_t mutex;
  void * data;/* Generally stores the root data address obtained from the pool application (the first application data interface in the pool) * /
  void * addr;/* The shared memory base address obtained using ngx_shmem application * /
} ngx_slab_pool_t;
void ngx_slab_init (ngx_slab_pool_t * pool);
void * ngx_slab_alloc (ngx_slab_pool_t * pool, size_t size);
void * ngx_slab_alloc_locked (ngx_slab_pool_t * pool, size_t size);
void * ngx_slab_calloc (ngx_slab_pool_t * pool, size_t size);
void * ngx_slab_calloc_locked (ngx_slab_pool_t * pool, size_t size);
void ngx_slab_free (ngx_slab_pool_t * pool, void * p);
void ngx_slab_free_locked (ngx_slab_pool_t * pool, void * p);

You can see that the interface is not complicated,The difference between alloc and calloc is whether to clear the memory segment obtained by the application.The interface ending with _locked indicates that the operation pool has acquired the lock.The ngx_slab_pool_t structure has a ngx_shmtx_t mutex used to synchronize multiple processes to access the pool concurrently.Note that ngx_slab_alloc () will acquire the lock first, then apply for space, and finally release the lock.While ngx_slab_alloc_locked () applies for space directly,Think of the program as acquiring locks in other logic.

The use of ngx_shmem in the development of nginx generally requires the following initialization process:

The module calls the ngx_shared_memory_add () interface during the configuration parsing process to register a section of shared memory.Provide callback function for shared memory size and memory initialization. The framework uses ngx_shmem to apply for memory in ngx_init_cycle (),And initialize ngx_slab, and then call back the initialization function registered by the module Module uses ngx_slab's application/whether interface

In this process,It involves the ngx_shared_memory_add () interface and the corresponding ngx_shm_zone_t structure.

struct ngx_shm_zone_s {
  void * data;
  ngx_shm_t shm;
  ngx_shm_zone_init_pt init;
  void * tag;
  void * sync;
  ngx_uint_t noreuse;/* unsigned noreuse:1;* /
ngx_shm_zone_t * ngx_shared_memory_add (ngx_conf_t * cf, ngx_str_t * name,  size_t size, void * tag);

It is worth mentioning the noreuse attribute, which controls whether the shared memory will be reapplied during the reload process of nginx.

Since the ngx_init_cycle () function is longer,This process can look up the relevant code by looking for/* create shared memory */this comment or cycle->shared_memory object.

For more details on the use of ngx_slab,Suggestions can refer to ngx_http_limit_conn_module, which is a module that implements limit on the number of connections through shared memory.The complexity of the module,Is a good reference example.

Understanding nginx (2nd Edition)


  • Previous Detailed explanation of SpringCloud Ribbon load balancing
  • Next ZooKeeper installation and deployment tutorial