Shared memory

Shared memory and message passing can be combined to provide IPC that offers:

• Very high performance (shared memory)
• Synchronization (message passing)
• Network transparency (message passing)

Using message passing, a client sends a request to a server and blocks. The server receives the messages in priority order from clients, processes them, and replies when it can satisfy a request. At this point, the client is unblocked and continues. The very act of sending messages provides natural synchronization between the client and the server. Rather than copy all the data through the message pass, the message can contain a reference to a shared-memory region, so the server could read or write the data directly. This is best explained with a simple example.

Let's assume a graphics server accepts draw image requests from clients and renders them into a frame buffer on a graphics card. Using message passing alone, the client would send a message containing the image data to the server. This would result in a copy of the image data from the client's address space to the server's address space. The server would then render the image and issue a short reply.

If the client didn't send the image data inline with the message, but instead sent a reference to a shared-memory region that contained the image data, then the server could access the client's data directly.

Since the client is blocked on the server as a result of sending it a message, the server knows that the data in shared memory is stable and does not change until the server replies. This combination of message passing and shared memory achieves natural synchronization and very high performance.

This model of operation can also be reversed—the server can generate data and give it to a client. For example, suppose a client sends a message to a server that reads video data directly from a CD-ROM into a shared memory buffer provided by the client. The client is blocked on the server while the shared memory is being changed. When the server replies and the client continues, the shared memory is stable for the client to access. This type of design can be pipelined using more than one shared-memory region.

Simple shared memory can't be used between processes on different computers connected via a network. Message passing, on the other hand, is network transparent. A server could use shared memory for local clients and full message passing of the data for remote clients. This allows you to provide a high-performance server that is also network transparent.

In practice, the message-passing primitives are more than fast enough for the majority of IPC needs. The added complexity of a combined approach need only be considered for special applications with very high bandwidth.

Multiple threads within a process share the memory of that process. To share memory between processes, you must first create a shared-memory region and then map that region into your process's address space. Shared-memory regions are created and manipulated using the following calls:

POSIX shared memory is implemented in the BlackBerry 10 OS via the process manager (procnto). The above calls are implemented as messages to procnto (see Process manager).

The shm_open() function takes the same arguments as open() and returns a file descriptor to the object. As with a regular file, this function lets you create a new shared-memory object or open an existing shared-memory object.

When a new shared-memory object is created, the size of the object is set to zero. To set the size, you use ftruncate() —the very same function used to set the size of a file —or shm_ctl().

When you have a file descriptor to a shared-memory object, you use the mmap() function to map the object, or part of it, into your process's address space. The mmap() function is the cornerstone of memory management within BlackBerry 10 OS and deserves a detailed discussion of its capabilities.

The mmap() function is defined as follows:

void * mmap( void *where_i_want_it,
             size_t length,
             int memory_protections,
             int mapping_flags,
             int fd,
             off_t offset_within_shared_memory );

In simple terms this says: "Map in length bytes of shared memory at offset_within_shared_memory in the shared-memory object associated with fd."

The mmap() function tries to place the memory at the address where_i_want_it in your address space. The memory is given the protections specified by memory_protections and the mapping is done according to the mapping_flags.

The three arguments fd, offset_within_shared_memory, and length define a portion of a particular shared object to be mapped in. It's common to map in an entire shared object, in which case the offset is zero and the length is the size of the shared object in bytes. On an Intel processor, the length is a multiple of the page size, which is 4096 bytes.

The return value of mmap() is the address in your process's address space where the object was mapped. The argument where_i_want_it is used as a hint by the system to where you want the object placed. If possible, the object is placed at the address requested. Most applications specify an address of zero, which gives the system free rein to place the object where it wants.

The following protection types may be specified for memory_protections:

You should use the PROT_NOCACHE manifest when you're using a shared-memory region to gain access to dual-ported memory that may be modified by hardware (for example, a video frame buffer or a memory-mapped network or communications board). Without this manifest, the processor may return stale data from a previously cached read.

The mapping_flags determine how the memory is mapped. These flags are broken down into two parts—the first part is a type and must be specified as one of the following:

The MAP_SHARED type is the one to use for setting up shared memory between processes; MAP_PRIVATE has more specialized uses.

You can OR a number of flags into the above type to further define the mapping. These are described in detail in mmap() . A few of the more interesting flags are:

MAP_ANON

Map anonymous memory that isn't associated with any file descriptor; you must set the fd parameter to NOFD. The mmap() function allocates the memory, and by default, fills the allocated memory with zeros; see Initializing allocated memory.

You commonly use MAP_ANON with MAP_PRIVATE, but you can use it with MAP_SHARED to create a shared memory area for forked applications. You can use MAP_ANON as the basis for a page-level memory allocator.

MAP_FIXED

Map the object to the address specified by where_i_want_it. If a shared-memory region contains pointers within it, then you may need to force the region at the same address in all processes that map it. This can be avoided by using offsets within the region in place of direct pointers.

MAP_PHYS

This flag indicates that you want to deal with physical memory. The fd parameter should be set to NOFD. When used without MAP_ANON, the offset_within_shared_memory specifies the exact physical address to map (for example, for video frame buffers). If used with MAP_ANON, then physically contiguous memory is allocated (for example, for a DMA buffer).

You can use MAP_NOX64K and MAP_BELOW16M to further define the MAP_ANON allocated memory and address limitations present in some forms of DMA.

Note: You should use mmap_device_memory() instead of MAP_PHYS, unless you're allocating physically contiguous memory.

MAP_NOX64K

Used with MAP_PHYS | MAP_ANON. The allocated memory area does not cross a 64-KB boundary. This is required for the old 16-bit PC DMA.

MAP_BELOW16M

Used with MAP_PHYS | MAP_ANON. The allocated memory area resides in physical memory below 16 MB. This is necessary when using DMA with ISA bus devices.

MAP_NOINIT

Relax the POSIX requirement to zero the allocated memory; see Initializing allocated memory.

Using the mapping flags described above, a process can easily share memory between processes:

/* Map in a shared memory region */
fd = shm_open("datapoints", O_RDWR);
addr = mmap(0, len, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);

or allocate a DMA buffer for a bus-mastering PCI network card:

/* Allocate a physically contiguous buffer */
addr = mmap(0, 262144, PROT_READ|PROT_WRITE|PROT_NOCACHE,
            MAP_PHYS|MAP_ANON, NOFD, 0);

You can unmap all or part of a shared-memory object from your address space using munmap() . This primitive isn't restricted to unmapping shared memory—it can be used to unmap any region of memory within your process. When used in conjunction with the MAP_ANON flag to mmap() , you can easily implement a private page-level allocator/deallocator.

You can change the protections on a mapped region of memory using mprotect() . Like munmap(), mprotect() isn't restricted to shared-memory regions—it can change the protection on any region of memory within your process.

POSIX requires that mmap() zero any memory that it allocates. It can take a while to initialize the memory, so BlackBerry 10 OS provides a way to relax the POSIX requirement. This allows for faster starting, but can be a security problem. Avoiding initializing the memory requires the cooperation of the process doing the unmapping and the one doing the mapping:

• The munmap_flags() function is a non-POSIX function that's similar to munmap() but lets you control what happens when the memory is next mapped:

  int munmap_flags( void *addr, size_t len,
                    unsigned flags );
    

  If you specify a flags argument of 0, munmap_flags() behaves the same as munmap() does.

  The following bits control the clearing of memory on allocation:

  UNMAP_INIT_REQUIRED

  POSIX initialization of the page to all zeroes is required the next time the underlying physical memory is allocated.

  UNMAP_INIT_OPTIONAL

  Initialization of the underlying physical memory to zeroes on its next allocation is optional.

• If you specify the MAP_NOINIT flag to mmap() , and the physical memory being mapped was previously unmapped with UNMAP_INIT_OPTIONAL, the POSIX requirement that the memory be zeroed is relaxed.

By default, the kernel initializes the memory, but you can control this by using the -m option to procnto . The argument to this option is a string that lets you enable or disable aspects of the memory manager:

i

munmap() acts as if UNMAP_INIT_REQUIRED were specified.

~i

munmap() acts as if UNMAP_INIT_OPTIONAL were specified.

By default when memory is freed for later reuse, the contents of that memory remain untouched; whatever the application that owned the memory left behind is left intact until the next time that memory is allocated by another process. With the munmap_flags() function you can control the clearing when unmapping.

You can also use the -m option to procnto to control the default behavior:

-mc

Clear memory when it's freed.

-m~c

Don't clear memory when it's freed (the default). When memory is freed for later reuse, the contents of that memory remain untouched; whatever the application that owned the memory left behind is left intact until the next time that memory is allocated by another process. At that point, before the memory is handed to the next process, it's zeroed.