From mboxrd@z Thu Jan 1 00:00:00 1970 Delivered-To: chneukirchen@gmail.com Received: by 10.140.128.1 with SMTP id a1cs349396rvd; Wed, 17 Mar 2010 08:41:35 -0700 (PDT) Received-SPF: pass (google.com: domain of 3LPigSwcLChEyvruzB9yvruzB9.t538rt1-uvCv2x55x2vx85B69.t53@groups.bounces.google.com designates 10.224.116.210 as permitted sender) client-ip=10.224.116.210; Authentication-Results: mr.google.com; spf=pass (google.com: domain of 3LPigSwcLChEyvruzB9yvruzB9.t538rt1-uvCv2x55x2vx85B69.t53@groups.bounces.google.com designates 10.224.116.210 as permitted sender) smtp.mail=3LPigSwcLChEyvruzB9yvruzB9.t538rt1-uvCv2x55x2vx85B69.t53@groups.bounces.google.com; dkim=pass header.i=3LPigSwcLChEyvruzB9yvruzB9.t538rt1-uvCv2x55x2vx85B69.t53@groups.bounces.google.com Received: from mr.google.com ([10.224.116.210]) by 10.224.116.210 with SMTP id n18mr4142058qaq.13.1268840494076 (num_hops = 1); Wed, 17 Mar 2010 08:41:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=beta; h=domainkey-signature:received:x-beenthere:received:received:received :received:received-spf:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to :x-original-authentication-results:x-original-sender:reply-to :precedence:mailing-list:list-id:list-post:list-help:list-archive :x-thread-url:x-message-url:sender:list-subscribe:list-unsubscribe :content-type:content-transfer-encoding; bh=I65g7MYtbB1LIhrrW1Mbxr2VOJnLKdZ9YN0OZJS8iAw=; b=rxxDKgJK2nJFB8ZDiyvMoy0+ESqLq30/4BKXt94rCssla9GU3c0ijw6cSA1dcwXJGI 0PzomPlK4NRxk3w9Pg31O67gal6+TQCZJUMWG1CzolET9Ll58UdL+QtfbWgBmScgmmmb 0OdfTV5JFrAIX1u7HBW2t4uCIhaGfRXYyqMik= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlegroups.com; s=beta; h=x-beenthere:received-spf:mime-version:in-reply-to:references:from :date:message-id:subject:to:x-original-authentication-results :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:x-thread-url:x-message-url:sender :list-subscribe:list-unsubscribe:content-type :content-transfer-encoding; b=AIB8xPkxfr2pARD6gzs2bdPjA39EsVf+wnV2aKosYgWRFXmiquqOBrahs8NEKuUY45 KMfFtg0daqufOyGfanb57gfipVAZIQjC8nwHded85n7RaJt6WBQw/aKWn/pl86elXGFr ZLCOHHz8E7eHP92GE8UVpHRZSXgCVrhHsIP0A= Received: by 10.224.116.210 with SMTP id n18mr526076qaq.13.1268840492759; Wed, 17 Mar 2010 08:41:32 -0700 (PDT) X-BeenThere: rack-devel@googlegroups.com Received: by 10.224.38.138 with SMTP id b10ls549772qae.4.p; Wed, 17 Mar 2010 08:41:31 -0700 (PDT) Received: by 10.224.96.215 with SMTP id i23mr365986qan.0.1268840491500; Wed, 17 Mar 2010 08:41:31 -0700 (PDT) Received: by 10.224.96.215 with SMTP id i23mr365985qan.0.1268840491461; Wed, 17 Mar 2010 08:41:31 -0700 (PDT) Return-Path: Received: from mail-qy0-f190.google.com (mail-qy0-f190.google.com [209.85.221.190]) by gmr-mx.google.com with ESMTP id 18si1552020qyk.0.2010.03.17.08.41.31; Wed, 17 Mar 2010 08:41:31 -0700 (PDT) Received-SPF: pass (google.com: domain of headius@headius.com designates 209.85.221.190 as permitted sender) client-ip=209.85.221.190; Received: by qyk28 with SMTP id 28so606156qyk.14 for ; Wed, 17 Mar 2010 08:41:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.224.105.73 with SMTP id s9mr319493qao.57.1268840491107; Wed, 17 Mar 2010 08:41:31 -0700 (PDT) In-Reply-To: <20100308011204.GA19119@dcvr.yhbt.net> References: <20100306075548.GB6474@dcvr.yhbt.net> <44f3f951-889e-45ec-ae46-40a371329a9e@e1g2000yqh.googlegroups.com> <20100308002217.GB18365@dcvr.yhbt.net> <20100308011204.GA19119@dcvr.yhbt.net> From: Charles Oliver Nutter Date: Wed, 17 Mar 2010 09:41:11 -0600 Message-ID: Subject: Re: Not cleaning up tempfiles for multipart? To: rack-devel@googlegroups.com X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of headius@headius.com designates 209.85.221.190 as permitted sender) smtp.mail=headius@headius.com X-Original-Sender: headius@headius.com Reply-To: rack-devel@googlegroups.com Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: List-Post: , List-Help: , List-Archive: X-Thread-Url: http://groups.google.com/group/rack-devel/t/6eb2bc7a1f8c072c X-Message-Url: http://groups.google.com/group/rack-devel/msg/26c50b31118a816a Sender: rack-devel@googlegroups.com List-Subscribe: , List-Unsubscribe: , Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun, Mar 7, 2010 at 7:12 PM, Eric Wong wrote: > I guess this deals with wonky multipart uploads that browsers still > generate these days[1]. =C2=A0Ugh, yeah, it's nasty... > The comment below in _call is very important. > > > 1. =C2=A0Ensure all tempfiles created by Rack go into an array in env, > =C2=A0 =C2=A0probably env["rack.tempfiles"]: > > =C2=A0 =C2=A0tempfile =3D Tempfile.new("foo") > =C2=A0 =C2=A0(env["rack.tempfiles"] ||=3D []) << tempfile I'm with you so far... > 2. =C2=A0Have a middleware wrap everything, including the response body: I'm a bit new to the middleware thing. Does this mean everyone would have to configure middleware on their setups to get this tempfile-closing behavior? And your comment below...does that mean there are situations where this wouldn't work? Closing and removing tempfiles when they're no longer needed should be the default behavior, not something you have to configure. It's a bug to not close and remove them (or at least, a bug to keep creating new ones and expecting GC to clean everything up eventually). > =C2=A0 =C2=A0class TempfileReaper < Struct.new(:app, :env, :body) > > =C2=A0 =C2=A0 =C2=A0# --------------- in config.ru ----------- > =C2=A0 =C2=A0 =C2=A0# use TempfileReaper > =C2=A0 =C2=A0 =C2=A0# run MyApp.new > =C2=A0 =C2=A0 =C2=A0def initialize(app) > =C2=A0 =C2=A0 =C2=A0 =C2=A0super(app) > =C2=A0 =C2=A0 =C2=A0end > > =C2=A0 =C2=A0 =C2=A0# we wrap the entire response body later on, so dup i= t for > =C2=A0 =C2=A0 =C2=A0# reentrancy. =C2=A0You can actually avoid the enforc= ed dup as > =C2=A0 =C2=A0 =C2=A0# an optimization, probably... > =C2=A0 =C2=A0 =C2=A0def call(env) > =C2=A0 =C2=A0 =C2=A0 =C2=A0dup._call(env) > =C2=A0 =C2=A0 =C2=A0end > > =C2=A0 =C2=A0 =C2=A0# used when wrapping the response body > =C2=A0 =C2=A0 =C2=A0def each(&block) > =C2=A0 =C2=A0 =C2=A0 =C2=A0body.each { |chunk| yield chunk } > =C2=A0 =C2=A0 =C2=A0end > > =C2=A0 =C2=A0 =C2=A0# the Rack server should call this (when we're the bo= dy) > =C2=A0 =C2=A0 =C2=A0def close > =C2=A0 =C2=A0 =C2=A0 =C2=A0tempfiles =3D env["rack.tempfiles"] > =C2=A0 =C2=A0 =C2=A0 =C2=A0if tempfiles > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tempfiles.each { |tmp| tmp.close! rescu= e nil } > =C2=A0 =C2=A0 =C2=A0 =C2=A0end > =C2=A0 =C2=A0 =C2=A0end By "the Rack server should call this" do you mean that if the server doesn't call this, tempfiles Rack creates will not be cleaned up until GC runs? Shouldn't Rack itself be guaranteeing it doesn't leave garbage files around= ? > =C2=A0 =C2=A0 =C2=A0# wrap the normal application call, saving env > =C2=A0 =C2=A0 =C2=A0def _call(env) > =C2=A0 =C2=A0 =C2=A0 =C2=A0self.env =3D env > > =C2=A0 =C2=A0 =C2=A0 =C2=A0# XXX VERY IMPORTANT: > =C2=A0 =C2=A0 =C2=A0 =C2=A0# you need to ensure env stays the same throug= hout the request, > =C2=A0 =C2=A0 =C2=A0 =C2=A0# some middlewares overwrite/replace it instea= d of merge!-ing into it > =C2=A0 =C2=A0 =C2=A0 =C2=A0status, headers, body =3D app.call(env) > =C2=A0 =C2=A0 =C2=A0 =C2=A0self.body =3D body > =C2=A0 =C2=A0 =C2=A0 =C2=A0[ status, headers, self ] > =C2=A0 =C2=A0 =C2=A0end > =C2=A0 =C2=A0end Same question as above...can badly-written middleware now cause tempfiles to linger? - Charlie