From mboxrd@z Thu Jan 1 00:00:00 1970 Delivered-To: chneukirchen@gmail.com Received: by 10.227.155.193 with SMTP id t1csp136388wbw; Mon, 6 Jan 2014 17:30:46 -0800 (PST) Return-Path: Received-SPF: pass (google.com: domain of rack-devel+bncBDE6DHFEZEEBBRFRVWLAKGQE6SSWNPQ@googlegroups.com designates 10.50.34.161 as permitted sender) client-ip=10.50.34.161 Authentication-Results: mr.google.com; spf=pass (google.com: domain of rack-devel+bncBDE6DHFEZEEBBRFRVWLAKGQE6SSWNPQ@googlegroups.com designates 10.50.34.161 as permitted sender) smtp.mail=rack-devel+bncBDE6DHFEZEEBBRFRVWLAKGQE6SSWNPQ@googlegroups.com; dkim=pass header.i=@googlegroups.com X-Received: from mr.google.com ([10.50.34.161]) by 10.50.34.161 with SMTP id a1mr2923220igj.5.1389058246012 (num_hops = 1); Mon, 06 Jan 2014 17:30:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20120806; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:sender:list-subscribe :list-unsubscribe:content-type; bh=MopIvjyc4f0lXoKscPsg8esU+sDEe8MMIRYtANJEIck=; b=A4WFiiQVqMPGuEn/yjEWboz+wka0T6nj71ErX0911V7s3JnL8LIr5Om+OBZ5xtNVwZ NT79Qqv65zDHSdcmH8a25wvdZmsUzdf/grwxXh3LtSA5SLq/kzh7d9t7axCu9LlneXVZ iLxii9AJBtbt7DJEuf2BwIsIpniqJTkWKxQPMnKnFT0/ZmghkFh87nJLwcAfpShlMPTP ZgB3o6rxVWSIcqtgJOOpv0oU0CHJceHxW6/lkGlgAC/jGcEOsmRjTU86mtysATrNO7XR KQE1cNoNqJpSNvE9N28NPh0J4ewmW56C3oQWQEYJA6gZEMKmTqb6cl+9gNY1QM3J/1/n 2YmQ== X-Received: by 10.50.34.161 with SMTP id a1mr333789igj.5.1389058245439; Mon, 06 Jan 2014 17:30:45 -0800 (PST) X-BeenThere: rack-devel@googlegroups.com Received: by 10.50.106.105 with SMTP id gt9ls1645946igb.7.canary; Mon, 06 Jan 2014 17:30:44 -0800 (PST) X-Received: by 10.66.246.194 with SMTP id xy2mr716213pac.36.1389058244331; Mon, 06 Jan 2014 17:30:44 -0800 (PST) Received: by 10.224.127.199 with SMTP id h7msqas; Mon, 6 Jan 2014 14:45:41 -0800 (PST) X-Received: by 10.49.12.109 with SMTP id x13mr269100qeb.12.1389048341623; Mon, 06 Jan 2014 14:45:41 -0800 (PST) Date: Mon, 6 Jan 2014 14:45:41 -0800 (PST) From: Wojtek Kruszewski To: rack-devel@googlegroups.com Message-Id: <73e2d44a-2842-4348-9133-9454be615645@googlegroups.com> In-Reply-To: <20100318095409.GB15049@dcvr.yhbt.net> References: <20100306075548.GB6474@dcvr.yhbt.net> <44f3f951-889e-45ec-ae46-40a371329a9e@e1g2000yqh.googlegroups.com> <20100308002217.GB18365@dcvr.yhbt.net> <20100308011204.GA19119@dcvr.yhbt.net> <20100318095409.GB15049@dcvr.yhbt.net> Subject: Re: Not cleaning up tempfiles for multipart? MIME-Version: 1.0 X-Original-Sender: wojtek@oxos.pl Reply-To: rack-devel@googlegroups.com Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: X-Google-Group-Id: 486215384060 List-Post: , List-Help: , List-Archive: Sender: rack-devel@googlegroups.com List-Subscribe: , List-Unsubscribe: , Content-Type: multipart/alternative; boundary="----=_Part_13384_22395083.1389048341152" ------=_Part_13384_22395083.1389048341152 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I see this is still an issue. Is there a recommended way to have multipart= =20 temp files cleaned after request? On Thursday, March 18, 2010 10:54:09 AM UTC+1, Eric Wong wrote: > > Charles Oliver Nutter > wrote: > > On Sun, Mar 7, 2010 at 7:12 PM, Eric Wong >=20 > wrote: > > > I guess this deals with wonky multipart uploads that browsers still > > > generate these days[1]. =EF=BF=BDUgh, yeah, it's nasty... > > > The comment below in _call is very important. > > > > > > > > > 1. =EF=BF=BDEnsure all tempfiles created by Rack go into an array in = env, > > > =EF=BF=BD =EF=BF=BDprobably env["rack.tempfiles"]: > > > > > > =EF=BF=BD =EF=BF=BDtempfile =3D Tempfile.new("foo") > > > =EF=BF=BD =EF=BF=BD(env["rack.tempfiles"] ||=3D []) << tempfile > >=20 > > I'm with you so far... > >=20 > > > 2. =EF=BF=BDHave a middleware wrap everything, including the response= body: > >=20 > > I'm a bit new to the middleware thing. Does this mean everyone would > > have to configure middleware on their setups to get this > > tempfile-closing behavior? And your comment below...does that mean > > there are situations where this wouldn't work? > >=20 > > Closing and removing tempfiles when they're no longer needed should be > > the default behavior, not something you have to configure. It's a bug > > to not close and remove them (or at least, a bug to keep creating new > > ones and expecting GC to clean everything up eventually). > > I would expect major frameworks to stick this into the default > middleware stack as a convenience, but some users want a more bare-bones > Rack stack can still opt out. I see it as needless overhead as the vast > majority of HTTP requests do not create tempfiles. > > I don't make decisions for Rack, though. > > > > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD# the Rack server should call this (when= we're the body) > > > =EF=BF=BD =EF=BF=BD =EF=BF=BDdef close > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDtempfiles =3D env["rack.tempfi= les"] > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDif tempfiles > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDtempfiles.each { |tm= p| tmp.close! rescue nil } > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDend > > > =EF=BF=BD =EF=BF=BD =EF=BF=BDend > >=20 > > By "the Rack server should call this" do you mean that if the server > > doesn't call this, tempfiles Rack creates will not be cleaned up until > > GC runs? > > Yes, a Rack-compliant server should call body.close if it responds to > body.close at the end of the response cycle for every individual > request. > > I put this in the wrapped body because body.each {} could be relying on > the tempfiles to generate the response. body.close is the absolute last > action for any Rack application. > > > Shouldn't Rack itself be guaranteeing it doesn't leave garbage files=20 > around? > >=20 > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD# wrap the normal application call, savi= ng env > > > =EF=BF=BD =EF=BF=BD =EF=BF=BDdef _call(env) > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDself.env =3D env > > > > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD# XXX VERY IMPORTANT: > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD# you need to ensure env stays= the same throughout the=20 > request, > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD# some middlewares overwrite/r= eplace it instead of=20 > merge!-ing into it > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDstatus, headers, body =3D app.= call(env) > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDself.body =3D body > > > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD[ status, headers, self ] > > > =EF=BF=BD =EF=BF=BD =EF=BF=BDend > > > =EF=BF=BD =EF=BF=BDend > >=20 > > Same question as above...can badly-written middleware now cause > > tempfiles to linger? > > Yes, it might be safer to do this before app.call above: > > =EF=BF=BD env["rack.tempfiles"] ||=3D [] > > That way if env gets replaced down the stack, e.g. via: > > app.call(env.merge("foo.hello" =3D> "world")) > # env.merge! would be correct above > > any use of env["rack.tempfiles"] will still point to the same array. > Well, almost... > > Then again some bad middleware could do: > =EF=BF=BD env["rack.tempfiles"] +=3D [ tmp_a, tmp_b ] > > Instead of what they _should_ do: > =EF=BF=BD env["rack.tempfiles"].concat([ tmp_a, tmp_b ]) > > So yes, discourage future middleware authors from replacing > the rack.tempfiles array. > > --=20 > Eric Wong > > --=20 ---=20 You received this message because you are subscribed to the Google Groups "= Rack Development" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to rack-devel+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. ------=_Part_13384_22395083.1389048341152 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I see this is still an issue. Is there a recommended way t= o have multipart temp files cleaned after request?

On Thursday, Marc= h 18, 2010 10:54:09 AM UTC+1, Eric Wong wrote:
Charles Oliver Nutter <hea...@headius.com> wrote:
> On Sun, Mar 7, 2010 at 7:1= 2 PM, Eric Wong <normal...@yhbt.net= > wrote:
> > I guess this deals with wonky multipart upload= s that browsers still
> > generate these days[1]. =EF=BF=BDUgh, ye= ah, it's nasty...
> > The comment below in _call is very important= .
> >
> >
> > 1. =EF=BF=BDEnsure all tempfiles c= reated by Rack go into an array in env,
> > =EF=BF=BD =EF=BF=BDpro= bably env["rack.tempfiles"]:
> >
> > =EF=BF=BD =EF=BF=BDt= empfile =3D Tempfile.new("foo")
> > =EF=BF=BD =EF=BF=BD(env["rack.= tempfiles"] ||=3D []) << tempfile
>
> I'm with you so fa= r...
>
> > 2. =EF=BF=BDHave a middleware wrap everything, i= ncluding the response body:
>
> I'm a bit new to the middlewar= e thing. Does this mean everyone would
> have to configure middleware= on their setups to get this
> tempfile-closing behavior? And your co= mment below...does that mean
> there are situations where this wouldn= 't work?
>
> Closing and removing tempfiles when they're no lo= nger needed should be
> the default behavior, not something you have = to configure. It's a bug
> to not close and remove them (or at least,= a bug to keep creating new
> ones and expecting GC to clean everythi= ng up eventually).

I would expect major frameworks to stick this into the= default
middleware stack as a convenience, but some users want a more b= are-bones
Rack stack can still opt out.  I see it as needless overh= ead as the vast
majority of HTTP requests do not create tempfiles.

I don't make decisions for Rack, though.

<snip>

> >= ; =EF=BF=BD =EF=BF=BD =EF=BF=BD# the Rack server should call this (when we'= re the body)
> > =EF=BF=BD =EF=BF=BD =EF=BF=BDdef close
> &g= t; =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDtempfiles =3D env["rack.tempfiles= "]
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDif tempfiles
>= > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDtempfiles.each { |tm= p| tmp.close! rescue nil }
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF= =BF=BDend
> > =EF=BF=BD =EF=BF=BD =EF=BF=BDend
>
> By= "the Rack server should call this" do you mean that if the server
> = doesn't call this, tempfiles Rack creates will not be cleaned up until
&= gt; GC runs?

Yes, a Rack-compliant server should call body.close if i= t responds to
body.close at the end of the response cycle for every indi= vidual
request.

I put this in the wrapped body because body.each {= } could be relying on
the tempfiles to generate the response.  body= .close is the absolute last
action for any Rack application.

> = Shouldn't Rack itself be guaranteeing it doesn't leave garbage files around= ?
>
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD# wrap the normal appl= ication call, saving env
> > =EF=BF=BD =EF=BF=BD =EF=BF=BDdef _cal= l(env)
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BDself.env =3D env=
> >
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD# XXX VER= Y IMPORTANT:
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF=BF=BD# you need= to ensure env stays the same throughout the request,
> > =EF=BF= =BD =EF=BF=BD =EF=BF=BD =EF=BF=BD# some middlewares overwrite/replace it in= stead of merge!-ing into it
> > =EF=BF=BD =EF=BF=BD =EF=BF=BD =EF= =BF=BDstatus, headers, body =3D app.call(env)
> > =EF=BF=BD =EF=BF= =BD =EF=BF=BD =EF=BF=BDself.body =3D body
> > =EF=BF=BD =EF=BF=BD = =EF=BF=BD =EF=BF=BD[ status, headers, self ]
> > =EF=BF=BD =EF=BF= =BD =EF=BF=BDend
> > =EF=BF=BD =EF=BF=BDend
>
> Same = question as above...can badly-written middleware now cause
> tempfile= s to linger?

Yes, it might be safer to do this before app.call above:=

=EF=BF=BD  env["rack.tempfiles"] ||=3D []

That way if env= gets replaced down the stack, e.g. via:

   app.call(env.me= rge("foo.hello" =3D> "world"))
   # env.merge! would b= e correct above

any use of env["rack.tempfiles"] will still point to = the same array.
Well, almost...

Then again some bad middleware cou= ld do:
=EF=BF=BD  env["rack.tempfiles"] +=3D [ tmp_a, tmp_b ]

Instead of what they _should_ do:
=EF=BF=BD  env["rack.tempfiles"]= .concat([ tmp_a, tmp_b ])

So yes, discourage future middleware a= uthors from replacing
the rack.tempfiles array.

--
Eric Wong

--
 
---
You received this message because you are subscribed to the Google Groups &= quot;Rack Development" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to rack-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
------=_Part_13384_22395083.1389048341152--