Re: scsh in PLT Scheme?

[Anton van Straaten <anton@appsolutions.com>]

> Also, you should ask yourself "what data 
> do I need and why"? Think about why your 
> query asked for a large recordset.

One of the points I was making is that sometimes, you'd have to extend this logic to the end user to make it work.  And that's not always valid - you're basically telling them "no, you can't use the system the way you'd like to".  In heavily loaded environments, you might be forced to say this because of load and performance considerations, but in many other cases, it's perfectly doable, and the only thing standing in the way of such capabilities are developer sensibilities, which may often be misapplied.  Developers often want to micromanage how end users use a system, which can get in the way of usability.

For example, database connections aren't noticeably wasteful in an environment where the database isn't overstressed.  Gearing APIs and architectures for the most large-scale environments can be actively hostile to smaller-scale development.

> What fits on the page is not necessarily correlated with what is downloaded.
> You can easily display subsets of what you have doanloaded. 

See my comment about the need for asynchronous download.  If you don't have that, then you have to wait for the entire set to download before display can begin.  This can be a problem at multiple levels.

> I would not advocate a single pattern of access. Just remember that the
> costs are mostly associated with keeping connections open, not with
> dowloading data. 

That's a bit contradictory: costs aren't all absolute, they can depend on access patterns and be relative to system resources.  Downloading data can be a major concern over a slow link, but open connections aren't necessarily a concern for a departmental application as opposed to e.g. a public web site.

In fact, with multi-CPU, multi-gigahertz servers with gigabytes of RAM being available for a few $1000, database performance is often much less of an issue than it once was, for many applications.  Big public web sites or large organizations are a highly visible exception to this, and to a large extent they're driving the conventional wisdom on these issues.  That "wisdom" may not be universally applicable, though.

Anton