[Cosmo-dev] bug 5078: read put request content into a tmp file and compare the]

[Brian Moseley]

On 10/17/06, Randy Letness <randy at osafoundation.org> wrote:

> Yes by buffering the request into a file, you introduce the disk
> subsystem into the
> equation.

tho this isn't something new, just something i didn't port from 0.4 until now.

> The problem with buffering in memory
> is what happens
> when 50 PUTs come in with a 10MB resource.  That's 500MB of memory.  Is
> this even
> going to happen?  I guess we won't know until we see some real-world
> usage.  If we
> start seeing OutOfMemory exceptions, then that is when we need to think
> about
> streaming to disk first.

disagree. reading request content into tmp files has not yet been a
bottleneck for us, and i want to see hard numbers proving that it is
before we redesign something that's been working for almost two years.

> The only problem is that after parsing the data into a Calendar object,
> you may
> lose the byte-for-byte data blob if you then try to transform the
> Calendar object
> to a stream of data.

and that is an important feature which allows us to use strong etags,
which i'll fight tooth and nail against losing.

every web app framework and toolkit i've ever seen reads put and
multipart post content to disk before doing anything else with it.
it's a safety guarantee. this is a strong precedent, and if it were
known to be a scalability issue, at least one of these tools would
have addressed it sometime in the last ten plus years. let's not
prematurely optimize until it actually becomes a problem. and even if
it does, there are solutions like using ram for the temporary
filesystem that can also be explored.