[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

FFI again



I have re-visisted my FFI suggestion from some time ago.  This is a
complete re-implementation that shares very little with the last one,
and can generally be viewed as a combination of three major
components:

1. An interface to Haible's ffcall library to call C functions with a
   dynamic list of arguments (and later, to implement Scheme
   callbacks).

2. An interface to the Unix DL library that can open a shared object
   and pull out a pointer by some name.

3. An interface to low-level memory management -- mallocing,
   referencing/storing arbitrary C objects in memory etc etc.

This follows the same principle of a minimal C interface layer,
leaving the heavy stuff to Scheme.  (I tried this last time too, but
for some reason I fell into the trap of thinking about defining new
types in the C interface instead of leaving it to Scheme.)  It is at a
pretty useful stage now, the main features that I can think about are:

1. Large parts of the source are generated by Scheme instead of
   fighting with brain-dead CPP macros (using some syntax similar to
   those dynamic html things).

2. A good coverage of all primitive C types, including floats and
   doubles.  Since it is done with the ffcall library then it
   shouldn't be too hard to make it work on Windows too (substituting
   DLL functionality instead of the Unix DL stuff).

3. Since the memory management routines are so low-level, they can be
   used for other things too.  For example, an application that needs
   memory efficient matrices can malloc the necessary memory block
   size, store and retrieve C doubles directly from it, wrapping the
   whole thing by a Scheme struct that will remember the dimensions
   etc (but this will only make memory consumption since marshaling
   values back and forth will add/remove type tags anyway).  It might
   be even possible to play with something like GNU Lightning
   (http://www.gnu.org/software/lightning/) to generate run-time code
   and use it...

4. I now believe it shouldn't be too hard to implement C struct
   object.  However, passing these *objects* rather than a pointer to
   them requires more work, but I don't think that this is something
   that gets used much (I never saw any need for it).

5. There is a simple mechanism that can be used to define new type
   symbols.  This is used to implement simple additional types that
   use functions to convert to/from one of the primitive types.  A
   good example for using this is defining `enumerations' that map
   symbols to/from fixnums.

6. No C `closures' that remember their types are generated, which
   means that a C function can be called with different types and
   arities (e.g, printf).  Making these closures is the common case,
   but much easier to do on the Scheme side...

More important notes:

* I havn't officially migrated to 200 yet (I have a *lot* of stuff
  that I use every day around), so it should be used with v103.  But
  there is no specific dependency on 103, and I will move it there one
  I move other stuff...

* There is no real documentation, which I admit is pretty important
  for something like this.

* There are still no callbacks, but it shouldn't take too long to make
  once I get some time...

* The fact that this is so low-level, requires knowing a bit about the
  internals of MzScheme -- for example, the different ways to malloc a
  block of memory.

* Also, using this thing makes it pretty easy to get crashes that are
  expected of C programs, so the whole thing can be taken as a
  "softer" way to play with the internals without playing with C
  proper.

As an example of using this, I implemented some quick glues to some
libraries: an interface to IBM's ViaVoice speech synthesis thing
(eci.ss), ESD (esd.ss), readline (but I still need to think of some
way to make it easy to make it non-blocking) (readline.ss), XMMS
controlling (xmmsctrl.ss) and TCL (just for fun) (tcl.ss).

Last time there was some interest in IBM's speech synthesis thing --
the ViaVoice system can be downloaded as RPMs from IBM, but it's a
little difficult to get there, so if anyone wants I can put the RPMs
somewhere for a short time.

The whole thing is available at
  http://www.cs.cornell.edu/eli/tmp/ffi.tgz
It is a bit big due to the ffcall library which is included in
Linux-compiled form.  I didn't try it on a different Unix, but I don't
think there should be major problems with that.  Some explanation on
some of the files:
* eci.ss, esd.ss, readline.ss, tcl.ss, xmmsctrl.ss - the sample glue
  files I mentioned above.
* ffcall  - Haible's ffcall library, compiled (unmodified otherwise).
* ffi.ssc - main Scheme-processed C source (just run to compile).
* ffi.c   - generated C code from ffi.ssc.
* ffi.so  - generated .so extension.
* ffi.ss  - some Scheme interface to ffi.so.
* junk    - a directory with some testing stuff I use.

As a side note that demonstrates the whole thing nicely, I live in
Manhattan and drive to Ithaca and back every week, so I needed
desperately a sane way to play music with my laptop -- keeping it open
is not practical (battery) and difficult (typing while driving is not
a healthy habit...), and having a long sequence of music is not too
good (for example, a burst of *LOUD* music is excellent for these
times where you realize that these big things are trucks and not
lambda expressions).  My solution is a program that controls XMMS
using the mouse (heavily using the mouse wheel), and the nice thing
about it is that it uses speech to tell me things like track names
etc.  (Of course, it also tells me the time, the amount of battery
power left and an estimated arrival time...  The only thing I'm
missing is a GPS...)  If anyone is intersted in this, I can post the
code with instructions on how to make it work.

I would love to get any comments etc on this -- I wanted to have
something like this for some time now, and I think that the result is
pretty nice.  It would be even more useful on Windows -- since there
is a lot of good stuff burried in system DLLs and the usual scenario
is having no C compiler around.  Unfortunately (maybe fortunately is a
better word here), I fall under the same category as having no Windows
C compiler, and I don't feel bad about it since I rarely use Windows,
so I'll be happy if someone can make it work on Windows too...

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                  http://www.barzilay.org/                 Maze is Life!