Using message passing

We've been talking about clients and servers and we've also used three key phrases:

• The client sends to the server.
• The server receives from the client.
• The server replies to the client.

We specifically used those phrases because they closely reflect the actual function names used in BlackBerry 10 OS message-passing operations.

Here's the complete list of functions dealing with message passing available under BlackBerry 10 OS (in alphabetical order):

• ChannelCreate() , ChannelDestroy()
• ConnectAttach() , ConnectDetach()
• MsgDeliverEvent()
• MsgError()
• MsgRead() , MsgReadv()
• MsgReceive() , MsgReceivePulse() , MsgReceivev()
• MsgReply() , MsgReplyv()
• MsgSend() , MsgSendnc() , MsgSendsv() , MsgSendsvnc() , MsgSendv() , MsgSendvnc() , MsgSendvs() , MsgSendvsnc()
• MsgWrite() , MsgWritev()

Don't let this list overwhelm you! You can write perfectly useful client/server applications using just a small subset of the calls from the list — as you get used to the ideas, you see that some of the other functions can be very useful in certain cases.

We break our discussion up into the functions that apply on the client side, and those that apply on the server side.

The client wants to send a request to a server, block until the server has completed the request, and then when the request is completed and the client is unblocked, to get at the answer. This implies two things: the client needs to be able to establish a connection to the server and then to transfer data via messages — a message from the client to the server (the send message) and a message back from the server to the client (the reply message, the server's reply).

So, let's look at these functions in turn. The first thing we need to do is to establish a connection. We do this with the function ConnectAttach(), which looks like this:

#include <sys/neutrino.h>

int ConnectAttach (int nd,
                   pid_t pid,
                   int chid,
                   unsigned index,
                   int flags);

ConnectAttach() is given three identifiers: the nd, which is the Node Descriptor, the pid, which is the process ID, and the chid, which is the channel ID. These three IDs, commonly referred to as ND/PID/CHID, uniquely identify the server that the client wants to connect to. We'll ignore the index and flags (just set them to 0).

So, let's assume that we want to connect to process ID 77, channel ID 1 on our node. Here's the code sample to do that:

int coid;

coid = ConnectAttach (0, 77, 1, 0, 0);

As you can see, by specifying a nd of zero, we're telling the kernel that we want to make a connection on our node.

At this point, you have a connection ID — a small integer that uniquely identifies a connection from my client to a specific channel on a particular server.

You can use this connection ID when sending to the server as many times as you like. When you are done with it, you can destroy it via:

ConnectDetach (coid);

So, let's see how you actually use it.

Message passing on the client is achieved using some variant of the MsgSend*() function family. Let's look at the simplest member, MsgSend():

#include <sys/neutrino.h>

int MsgSend (int coid,
             const void *smsg,
             int sbytes,
             void *rmsg,
             int rbytes);

MsgSend()'s arguments are:

• The connection ID of the target server (coid),
• A pointer to the send message (smsg),
• The size of the send message (sbytes),
• A pointer to the reply message (rmsg), and
• The size of the reply message (rbytes).

Let's send a simple message to process ID 77, channel ID 1:

#include <sys/neutrino.h>

char *smsg = "This is the outgoing buffer";
char rmsg [200];
int  coid;

// establish a connection
coid = ConnectAttach (0, 77, 1, 0, 0);
if (coid == -1) {
    fprintf (stderr, "Couldn't ConnectAttach to 0/77/1!\n");
    perror (NULL);
    exit (EXIT_FAILURE);
}

// send the message
if (MsgSend (coid,
             smsg, 
             strlen (smsg) + 1, 
             rmsg, 
             sizeof (rmsg)) == -1) {
    fprintf (stderr, "Error during MsgSend\n");
    perror (NULL);
    exit (EXIT_FAILURE);
}

if (strlen (rmsg) > 0) {
    printf ("Process ID 77 returns \"%s\"\n", rmsg);
}

Let's assume that process ID 77 was an active server expecting that particular format of message on its channel ID 1. After the server received the message, it would process it and at some point reply with a result. At that point, the MsgSend() would return a 0 indicating that everything went well. If the server sends us any data in the reply, we'd print it with the last line of code (we're assuming we're getting NUL-terminated ASCII data back).

Now that we've seen the client, let's look at the server. The client used ConnectAttach() to create a connection to a server, and then used MsgSend() for all its message passing.

This implies that the server has to create a channel — this is the thing that the client connected to when it issued the ConnectAttach() function call. Once the channel has been created, the server usually leaves it up forever. The channel gets created via the ChannelCreate() function, and destroyed via the ChannelDestroy() function:

#include <sys/neutrino.h>

int ChannelCreate  (unsigned flags);

int ChannelDestroy (int chid);

We'll come back to the flags argument in Channel flags. For now, let's just use a 0. Therefore, to create a channel, the server issues:

int  chid;

chid = ChannelCreate (0);

So we have a channel. At this point, clients could connect (via ConnectAttach()) to this channel and start sending messages:

As far as the message-passing aspects are concerned, the server handles message passing in two stages; a receive stage and a reply stage:

Initially we look at two simple versions of these functions, MsgReceive() and MsgReply() , and then later see some of the variants.

#include <sys/neutrino.h>

int MsgReceive (int chid,
                void *rmsg,
                int rbytes,
                struct _msg_info *info);

int MsgReply (int rcvid,
              int status,
              const void *msg,
              int nbytes);

Let's look at how the parameters relate:

As you can see from the diagram, there are four things we need to talk about:

1. The client issues a MsgSend() and specifies its transmit buffer (the smsg pointer and the sbytes length). This gets transferred into the buffer provided by the server's MsgReceive() function, at rmsg for rbytes in length. The client is now blocked.
2. The server's MsgReceive() function unblocks, and returns with a rcvid, which the server uses later for the reply. At this point, the data is available for the server to use.
3. The server has completed the processing of the message, and now uses the rcvid it got from the MsgReceive() by passing it to the MsgReply(). Note that the MsgReply() function takes a buffer (smsg) with a defined size (sbytes) as the location of the data to transmit to the client. The data is now transferred by the kernel.
4. Finally, the sts parameter is transferred by the kernel, and shows up as the return value from the client's MsgSend(). The client now unblocks.

You may have noticed that there are two sizes for every buffer transfer (in the client send case, there's sbytes on the client side and rbytes on the server side; in the server reply case, there's sbytes on the server side and rbytes on the client side.) The two sets of sizes are present so that the programmers of each component can specify the sizes of their buffers. This is done for added safety.

In our example, the MsgSend() buffer's size was the same as the message string's length. Let's look at the server and see how the size is used there.

Here's the overall structure of a server:

#include <sys/neutrino.h>

...

void
server (void)
{
    int     rcvid;         // indicates who we should reply to
    int     chid;          // the channel ID
    char    message [512]; // big enough for our purposes

    // create a channel
    chid = ChannelCreate (0);

    // this is typical of a server:  it runs forever
    while (1) {

        // get the message, and print it
        rcvid = MsgReceive (chid, message, sizeof (message),
                            NULL);
        printf ("Got a message, rcvid is %X\n", rcvid);
        printf ("Message was \"%s\".\n", message);

        // now, prepare the reply.  We reuse "message"
        strcpy (message, "This is the reply");
        MsgReply (rcvid, EOK, message, sizeof (message));
    }
}

As you can see, MsgReceive() tells the kernel that it can handle messages up to sizeof (message) (or 512 bytes). Our sample client (above) sent only 28 bytes (the length of the string). The following diagram illustrates:

The kernel transfers the minimum specified by both sizes. In our case, the kernel would transfer 28 bytes. The server would be unblocked and print out the client's message. The remaining 484 bytes (of the 512 byte buffer) remain unaffected.

We run into the same situation again with MsgReply(). The MsgReply() function says that it wants to transfer 512 bytes, but our client's MsgSend() function has specified that a maximum of 200 bytes can be transferred. So the kernel once again transfers the minimum. In this case, the 200 bytes that the client can accept limits the transfer size. (One interesting aspect here is that once the server transfers the data, if the client doesn't receive all of it, as in our example, there's no way to get the data back—it's gone forever.)

One thing that's perhaps not obvious in a message-passing environment is the need to follow a strict send-hierarchy. What this means is that two threads should never send messages to each other; rather, they should be organized such that each thread occupies a level; all sends go from one level to a higher level, never to the same or lower level. The problem with having two threads send messages to each other is that eventually you run into the problem of deadlock; both threads are waiting for each other to reply to their respective messages. Since the threads are blocked, they never get a chance to run and perform the reply, so you end up with two (or more!) hung threads.

The way to assign the levels to the threads is to put the outermost clients at the highest level, and work down from there. For example, if you have a graphical user interface that relies on some database server, and the database server in turn relies on the filesystem, and the filesystem in turn relies on a block filesystem driver, then you've got a natural hierarchy of different processes. The sends flow from the outermost client (the graphical user interface) down to the lower servers; the replies flow in the opposite direction.

While this certainly works in the majority of cases, you can encounter situations where you need to break the send hierarchy. This is never done by violating the send hierarchy and sending a message against the flow, but rather by using the MsgDeliverEvent() function.