Re: scsh in PLT Scheme?

[Bruce Butterfield <bab@entricom.com>]

MJ Ray wrote:
> Paul Steckler:
> 
>>I glanced at Oleg's code last night.  It appears to build a list out of
>>an entire result set.
> 
> 
> Only if you don't tell it when to stop gathering data, I believe.
> 
> 
>>SchemeQL fetches data lazily, which is what's needed when dealing with
>>large datasets.
> 
> 
> Surely what is needed with large datasets is being careful not to grab a
> large subset in a single operation by mistake.  If you're doing it
> intentionally, you're probably doing something the wrong way (eg, doing
> counting in scheme instead of in the database is bad unless you absolutely
> have to).
> 
> I suspect cases selecting a large subset for transfer to scheme are a
> minority of cases.  In those cases, the normal standard database interface
> may not be optimal.  Lazy gathering is not possible in some database
> interfaces, so its existence should not be relied upon, IMO.

I would argue that often it is necessary to walk through large sets of 
query data on a row-by-row basis to do some external processing 
(updates, inserts, reporting, whatever). This is typically done via 
cursors which maintain a live connection to the dataset rather than 
packaging all of the set up in a single container returned to the 
caller. It would be nice to have an iterator (or a lazy evaluation 
scheme) to do the same job in a robust API.