Re: Moab Channels

[Mike Hibler <mike>]

> Date: Sun, 6 May 2001 15:53:49 -0500 (CDT)
> From: Ravi Chamarty <ravi@ittc.ukans.edu>
> To: janos-discuss@fast.cs.utah.edu
> Subject: Moab Channels
> 
> Hi,
> 
>  I am trying to run a socktest type in-channel in Moab. I am encountering
> the following problem while creating the in-channel. When a new
> thread(chan_socketdata_intr) is created, the parent thread does not run
> any further. The 'chan_socketdata_intr' thread creates a socket and blocks
> on 'recvfrom'. Hence, when a packet arrives it is received correctly, but
> encounters an error when executing the function 'ani_demuxkeylist_apply'.
> This is probably because the parent thread has not yet enabled the channel
> in ani_chan_enable which would have added the demuxkey to the
> demuxkeylist.
> 
>  Is there a race condition wherein we may be required to do cond_wait
> before recvfrom ? Or could there be any other problem with the threads.
> Any help would be appreciated.
> 
> Thanks in advance.
> Ravi 
> 
>  
> 
> ----------------------------------------------------------
> Ravi S Chamarty                    E-mail: ravi@ittc.ku.edu
> Graduate Research Assistant,       Voice :785-864-7799 
> ITTC,2335 Irving Hill Road,                
> University of Kansas,
> Lawrence KS 66045-7612
> 

Here is a patch that should take care of livelock problems in the
chan_socket code.  It also fixes a stupid bug where we forgot to
initialize the socket timeout for OSKit, which could also cause problems.

Now we'll see if that is really what you are experiencing...

Index: chan_socket.c
===================================================================
RCS file: /n/fast/usr/lsrc/flux/CVS/moab/src/chan_socket.c,v
retrieving revision 1.53
diff -u -r1.53 chan_socket.c
--- chan_socket.c	2001/01/06 06:28:27	1.53
+++ chan_socket.c	2001/05/08 22:12:27
@@ -487,6 +487,8 @@
 	{
 		int ierr;
 
+		timeo.tv_sec = 0;
+		timeo.tv_usec = 500000;	/* 500ms */
 		ierr = setsockopt(s, SOL_SOCKET, SO_RCVTIMEO,
 				  &timeo, sizeof timeo);
 		assert(ierr == 0);
@@ -506,6 +508,18 @@
 	bdata = ani_pbuf_data(rbuf);
 	bsize = ani_pbuf_size(rbuf);
 
+	/*
+	 * XXX run the loop at lowest priority.  This is counter to what a
+	 * traditional interrupt handler does but its just too easy to
+	 * livelock here otherwise.  This choice can lead to starvation
+	 * in a system with other busy flows.  This is counter to our
+	 * resource isolation tenet and needs to be addressed.  The quick
+	 * and easy way would be to make sure that flows cannot share
+	 * sockets and to make the service thread be a flow thread.
+	 */
+	err = ani_thread_setprio(ani_thread_current(), 0);
+	assert(!ani_error_occured(err));
+
 	while (1) {
 		chan = 0;
 		
@@ -607,14 +621,6 @@
 		 */
 		bdata = ani_pbuf_data(rbuf);
 		bsize = ani_pbuf_size(rbuf);
-
-		/*
-		 * Yield here so that the packet threads have a chance to
-		 * do their thing and return the buffer to the inchan.
-		 * Otherwise we might just use up all the available buffers
-		 * and have to drop the rest of the packets.
-		 */
-		ani_thread_yield();
 	}
 
 	/*