Offtopic: How to Design Programming Tests

[Noel Welsh <noelwelsh@yahoo.com>]

Thanks all for comments, serious and not so serious. 
In no particular order:

--------------------

Brent, this one wasn't familiar:

> Describe how you might implement "Parasitic" methods
in Java using
> a mechanism which compiles to valid Java bytecode
capable of running
> on a compliant JVM.  Cite references.

Does this refer to mix-ins?  I thought another good
test would be to write Java bytecode that the verifier
cannot verify and correct or incorrect (i.e. the
Goedel statement for Java bytecode), but it then
occurred to me that verification probably relies on
Java's type system and Java's wimpy type system is
decidable.

Brent, I completely agree that programmer tests are
bogus.  The reason we now have them is that quality of
applicants is amazing low, and the quantity amazingly
high (it's a symptom of the profession, and
particularly contracting).  The test is quick way to
separate the wheat from the chaff.  Asking them to
send in their own programs is a good idea, but I can
see two problems with it:

1.  They might copy from a book/open-source code (this
has happened when we let someone do a test at home -
but they forgot to change a reference to page number
in the original text so it gave them away!)

2.  They might do all their coding at work, and hence
not have any code they can show us.

However, it is a good idea and I will suggest it.

--------------------

Michael Bogomolny suggested:

> compare the advantages and disadvantages of arrays
versus linked lists.

This is a good question.  My answer would be:

- random access time O(1) vs O(n)
- queue/stack operations O(n) vs O(1)
- memory usage: array better

Anything I missed?

> assume you have a(n unordered) set of numbers...

Another good question, but I'm not sure there will be
space is the test for another data structure question.

--------------------

RJA writes:

> What version of Java are you testing on and going to
be using?

I'm embarassed to say 1.1 at the moment.

> What is the relation between interfaces and classes

Very good question.  I'll definitely include it.

> Also, if you're using Java 2 the collection classes
there are
> probably more useful than a linked list

Definitely, but I wanted to test their ability to
write some code
using a structure that everyone should be familiar
with.

> You can also ask about another way of scaling
servers, whether they
> are singe/multi threaded.

And the answer is?  Multiple boxes with some load
balancing hardware?
(We just got some Alteon AD3s here - they're very
nice.)

> What are the diffent types of synchronization you
can do.

Like barriers and read/write locks and such?  Good
question.

> Would you sacrifice design for performance?

Always!  The compiler should fix it up! (Hah!  I
wish.)

--------------------

Matthias writes:

>  Q4.  Threads.  Code the producer/consumer. Give
>  reasons for (responsiveness, blocking IO) and
against
>   threading (synchronisation overhead)

> hmph ... 

I'm not sure why you're hmphing, but I will relate an
experience we've
recently had here.  We're trying to build a simple
HTTP server which
does socket keepalive (keep the TCP connection open
till it times out,
to avoid the handshake overhead).  In Java the only
way to do this is
to have a thread for each connection.  Under load the
machine dies
with too many threads open.  In C (at least in
AOLserver :-) all the
inactive connections are given to a single thread that
periodically
polls them for activity, and requeues any connections
that have become
active.  Now one can certainly have a much lighter
thread
implementation than Java's, but if you don't use the
native OS
threads, you won't get SMP.  There is a non-blocking
IO add-on for
Java, but most of the people here don't have the
proper Unix grounding
to use it.  Conclusion: threads aren't always good,
and neither is
blocking IO.

> good and why doesn't this suffice? why can a
consumer not kill a
> run-away applet?

Because they didn't write the JVM in Scheme, so the
JVM can have
memory leaks/resource allocation problems as well?

Noel

__________________________________________________
Get personalized email addresses from Yahoo! Mail - only $35 
a year!  http://personal.mail.yahoo.com/