From mboxrd@z Thu Jan 1 00:00:00 1970 Delivered-To: chneukirchen@gmail.com Received: by 10.140.141.15 with SMTP id o15cs124598rvd; Thu, 4 Feb 2010 02:32:28 -0800 (PST) Received-SPF: pass (google.com: domain of 3OqJqSwgJCpI51FGy60D28w47.yA8Dwy6-z0H072AA2702DAGBE.yA8@groups.bounces.google.com designates 10.100.238.2 as permitted sender) client-ip=10.100.238.2; Authentication-Results: mr.google.com; spf=pass (google.com: domain of 3OqJqSwgJCpI51FGy60D28w47.yA8Dwy6-z0H072AA2702DAGBE.yA8@groups.bounces.google.com designates 10.100.238.2 as permitted sender) smtp.mail=3OqJqSwgJCpI51FGy60D28w47.yA8Dwy6-z0H072AA2702DAGBE.yA8@groups.bounces.google.com; dkim=pass header.i=3OqJqSwgJCpI51FGy60D28w47.yA8Dwy6-z0H072AA2702DAGBE.yA8@groups.bounces.google.com Received: from mr.google.com ([10.100.238.2]) by 10.100.238.2 with SMTP id l2mr1339620anh.31.1265279548123 (num_hops = 1); Thu, 04 Feb 2010 02:32:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=beta; h=domainkey-signature:received:x-beenthere:received:received:received :received:received-spf:received:received:received:from:mime-version :content-type:subject:date:in-reply-to:to:references:message-id :x-mailer:x-original-authentication-results:x-original-sender :reply-to:precedence:mailing-list:list-id:list-post:list-help :list-archive:x-thread-url:x-message-url:sender:list-subscribe :list-unsubscribe; bh=POIvhMEEBkHgcZm15Iv7652HoZzoFQQEi3J2/tGvPT8=; b=GoWbR6hybCiLHYLxF5AI4DUBP3K8FB22eCV6VB4L+KfqfK/dgTXDmwNhi150DUliXH am6+P8QCMH1PIAOvhfyzcWVK25KmvO5Auv6MXDrFbhQtGF68Wrmx3GKrpARf/BGwDFQV YpLYsycqC/sCbgZ6/auf/OTNiGSL4p6DXoaeQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlegroups.com; s=beta; h=x-beenthere:received-spf:from:mime-version:content-type:subject :date:in-reply-to:to:references:message-id:x-mailer :x-original-authentication-results:x-original-sender:reply-to :precedence:mailing-list:list-id:list-post:list-help:list-archive :x-thread-url:x-message-url:sender:list-subscribe:list-unsubscribe; b=AAzA7BjVQZ9rNuaer/xBCZR9oTeJcTtCCrIGFH0R623HMhbYK2VVQSJx6tWM4FGFO8 kAi5R7r5fhQcSYMRt7tqaZySaYOqRHlpVCH2mkiSEUZEZP729HETu2jxAUSZcnS5Z4xs 0bkwOKQaNBSiE/UcnUc3gu1UkjuXwLxZXTPyo= Received: by 10.100.238.2 with SMTP id l2mr62025anh.31.1265279546629; Thu, 04 Feb 2010 02:32:26 -0800 (PST) X-BeenThere: rack-devel@googlegroups.com Received: by 10.213.50.10 with SMTP id x10ls374340ebf.3.p; Thu, 04 Feb 2010 02:32:22 -0800 (PST) Received: by 10.213.97.24 with SMTP id j24mr129386ebn.2.1265279542635; Thu, 04 Feb 2010 02:32:22 -0800 (PST) Received: by 10.213.97.24 with SMTP id j24mr129385ebn.2.1265279542582; Thu, 04 Feb 2010 02:32:22 -0800 (PST) Return-Path: Received: from mail-ew0-f212.google.com (mail-ew0-f212.google.com [209.85.219.212]) by gmr-mx.google.com with ESMTP id 11si1068879ewy.9.2010.02.04.02.32.21; Thu, 04 Feb 2010 02:32:21 -0800 (PST) Received-SPF: pass (google.com: domain of jftucker@gmail.com designates 209.85.219.212 as permitted sender) client-ip=209.85.219.212; Received: by ewy4 with SMTP id 4so897722ewy.7 for ; Thu, 04 Feb 2010 02:32:21 -0800 (PST) Received: by 10.216.90.78 with SMTP id d56mr496244wef.126.1265279541408; Thu, 04 Feb 2010 02:32:21 -0800 (PST) Return-Path: Received: from ?192.168.1.101? (bb-87-81-237-21.ukonline.co.uk [87.81.237.21]) by mx.google.com with ESMTPS id t2sm19728065gve.24.2010.02.04.02.32.17 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 04 Feb 2010 02:32:19 -0800 (PST) From: James Tucker Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: multipart/signed; boundary=Apple-Mail-2--790472240; protocol="application/pkcs7-signature"; micalg=sha1 Subject: Re: streaming api Date: Thu, 4 Feb 2010 10:32:16 +0000 In-Reply-To: To: rack-devel@googlegroups.com References: <20100203105938.GA20037@dcvr.yhbt.net> Message-Id: <2C4DB504-572F-4EF5-896B-93ECF582E869@gmail.com> X-Mailer: Apple Mail (2.1077) X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jftucker@gmail.com designates 209.85.219.212 as permitted sender) smtp.mail=jftucker@gmail.com; dkim=pass (test mode) header.i=@gmail.com X-Original-Sender: jftucker@gmail.com Reply-To: rack-devel@googlegroups.com Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: List-Post: , List-Help: , List-Archive: X-Thread-Url: http://groups.google.com/group/rack-devel/t/5146e2033cfabda2 X-Message-Url: http://groups.google.com/group/rack-devel/msg/861d405d41cfec2b Sender: rack-devel@googlegroups.com List-Subscribe: , List-Unsubscribe: , --Apple-Mail-2--790472240 Content-Type: multipart/alternative; boundary=Apple-Mail-1--790472348 --Apple-Mail-1--790472348 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 3 Feb 2010, at 18:14, Randy Fischer wrote: >=20 > Rack already lets you stream the body. The body response only has to > respond to #each and the #each call can delay as much as it wants. = Ruby > 1.8 has (somewhat) cheap threads, and 1.9 has even cheaper Fibers so > you can sleep synchronously inside them. >=20 >=20 > I'm more interested in getting streaming content on the request=20 > side of things (go ahead and parse the headers, then just give=20 > me an IO object on the content). =20 >=20 > Reason: I get largish (> 4G) bodies occasionally in a particular=20 > archival service; I need to do md5 and sha1 and perhaps other > checksums soon. I don't want the web server doing a copy > to some temp space, thanks, I'll write it and checksum it and > so on on the fly. >=20 > Anyone doing work on this at the rack level? Not really no, as it's down to the webserver at this time, not down to = rack, as rack expects a full formed request, most servers deliver just = that. >=20 > -Randy Fischer --Apple-Mail-1--790472348 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii
On 3 Feb 2010, at 18:14, Randy Fischer wrote:


Rack already lets you stream the body.  The body response only has to
respond to #each and the #each call can delay as much as it wants.  Ruby
1.8 has (somewhat) cheap threads, and 1.9 has even cheaper Fibers so
you can sleep synchronously inside them.


I'm more interested in getting streaming content on the request
side of things (go ahead and parse the headers,  then just give
me an IO object on the content).  

Reason:  I get largish (> 4G) bodies occasionally in a particular
archival service;  I need to do md5 and sha1 and perhaps other
checksums soon.   I don't want the web server doing a copy
to some temp space, thanks, I'll write it and checksum it and
so on on the fly.

Anyone doing work on this at the rack level?

Not really no, as it's down to the webserver at this time, not down to rack, as rack expects a full formed request, most servers deliver just that.


-Randy Fischer

--Apple-Mail-1--790472348-- --Apple-Mail-2--790472240 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJljCCBEYw ggOvoAMCAQICEGb9R+PCGeToms2Z3fU6yyQwDQYJKoZIhvcNAQEFBQAwXzELMAkGA1UEBhMCVVMx FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAxIFB1YmxpYyBQcmltYXJ5 IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTA1MTAyODAwMDAwMFoXDTE1MTAyNzIzNTk1OVow gd0xCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNp Z24gVHJ1c3QgTmV0d29yazE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZl cmlzaWduLmNvbS9ycGEgKGMpMDUxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUG A1UEAxMuVmVyaVNpZ24gQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMjCCASIw DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMnfrOfq+PgDFMQAktXBfjbCPO98chXLwKuMPRyV zm8eECw/AO2XJua2x+atQx0/pIdHR0w+VPhs+Mf8sZ69MHC8l7EDBeqV8a1AxUR6SwWi8mD81zpl Yu//EHuiVrvFTnAt1qIfPO2wQuhejVchrKaZ2RHp0hoHwHRHQgv8xTTq/ea6JNEdCBU3otdzzwFB L2OyOj++pRpu9MlKWz2VphW7NQIZ+dTvvI8OcXZZu0u2Ptb8Whb01g6J8kn+bAztFenZiHWcec5g J925rXXOL3OVekA6hXVJsLjfaLyrzROChRFQo+A8C67AClPN1zBvhTJGG+RJEMJs4q8fef/btLUC AwEAAaOB/zCB/DASBgNVHRMBAf8ECDAGAQH/AgEAMEQGA1UdIAQ9MDswOQYLYIZIAYb4RQEHFwEw KjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYTALBgNVHQ8EBAMCAQYw EQYJYIZIAYb4QgEBBAQDAgEGMC4GA1UdEQQnMCWkIzAhMR8wHQYDVQQDExZQcml2YXRlTGFiZWwz LTIwNDgtMTU1MB0GA1UdDgQWBBQRfV4ZfTwE32ps1qKKGj8x2DuUUjAxBgNVHR8EKjAoMCagJKAi hiBodHRwOi8vY3JsLnZlcmlzaWduLmNvbS9wY2ExLmNybDANBgkqhkiG9w0BAQUFAAOBgQA8o9oC YzrEk6qrctPcrVA4HgyeFkqIt+7r2f8PjZWg1rv6aguuYYTYaEeJ70+ssh9JQZtJM3aTi55uuUMc YL3C3Ioth8FFwBFyBBprJCpsb+f8BxMp0Hc6I+f1wYVoGb/GAVQgGa41gsxiPGEJxvTV67APpp8z hZrTcY5Qj5ndYjCCBUgwggQwoAMCAQICECeMlak0fpR8Io4+aS7PnaswDQYJKoZIhvcNAQEFBQAw gd0xCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNp Z24gVHJ1c3QgTmV0d29yazE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZl cmlzaWduLmNvbS9ycGEgKGMpMDUxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUG A1UEAxMuVmVyaVNpZ24gQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMjAeFw0w OTExMjAwMDAwMDBaFw0xMDExMjAyMzU5NTlaMIIBETEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4x HzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxRjBEBgNVBAsTPXd3dy52ZXJpc2lnbi5j b20vcmVwb3NpdG9yeS9SUEEgSW5jb3JwLiBieSBSZWYuLExJQUIuTFREKGMpOTgxHjAcBgNVBAsT FVBlcnNvbmEgTm90IFZhbGlkYXRlZDEzMDEGA1UECxMqRGlnaXRhbCBJRCBDbGFzcyAxIC0gTmV0 c2NhcGUgRnVsbCBTZXJ2aWNlMRUwEwYDVQQDFAxKYW1lcyBUdWNrZXIxITAfBgkqhkiG9w0BCQEW EmpmdHVja2VyQGdtYWlsLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOAB6yX/ 8ziknn0eW8pEvxNyz16y+pkpvHXtm09QNWS9UpWuyq2j1HDDW91sLqcla79IxYDGjRuuerfVLuFw 16lvZyENeb+NoajnA1Paow+taYqKuSQMNVjVFiy2ZPcZREKFOUUB+GkYLz6ErZ/2CB8esdB11Xya r/S2/8Qm3VM4xwaAf0Thq5zKimnkM+yXZEicYV8Ny+IxnxDMEvzolqJVdfMGnlbhcv1LFj96Rt9v kuV/J4lpmDTP1SXlIqprPFD4rC8Q54ktnhBjZt4jQkQ/RRE3f4GhnACpomSzjmw99Com6gT1/YaL tT00yCMwCxrGxgYT8em2XZ79HaIpW20CAwEAAaOBzDCByTAJBgNVHRMEAjAAMEQGA1UdIAQ9MDsw OQYLYIZIAYb4RQEHFwEwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3Jw YTALBgNVHQ8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMCMEoGA1UdHwRDMEEw P6A9oDuGOWh0dHA6Ly9JbmRDMURpZ2l0YWxJRC1jcmwudmVyaXNpZ24uY29tL0luZEMxRGlnaXRh bElELmNybDANBgkqhkiG9w0BAQUFAAOCAQEALH+Csg+lKzykCtckS9T/1M/b2LYbn1egArQeg1WN cj2xWet7/4eM+nnswKcQNYgDOI8hb6dwkB/aPPR/D9buUyQZzSeCoXFEpnAMFGcp5q2mxELBjHiu toKCBAKYjV9X8kcJYnW6ypJG+9UOL8JXEYU42i16UAQCizljT+lubYefVudNKNzpcTPbGJ0lF0zh q/HN5lEPBAhJsj8P79zMy96YQdLNli0ulVDwdv761+kyCtZlV3bBtQM9YHXBArUDC0Dr3ByzkrBG rKvVf+VmRXf82ytatHKNGantVZ51jhKTBylmm0OqVI/ZIS/IzGtKcTlakp91R5EuLI4NY6RKgjGC BIswggSHAgEBMIHyMIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAd BgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBo dHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBW YWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVy IENBIC0gRzICECeMlak0fpR8Io4+aS7PnaswCQYFKw4DAhoFAKCCAm0wGAYJKoZIhvcNAQkDMQsG CSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTAwMjA0MTAzMjE2WjAjBgkqhkiG9w0BCQQxFgQU Aih0d7go5/m8rYA79Q/C+zsgFNswggEDBgkrBgEEAYI3EAQxgfUwgfIwgd0xCzAJBgNVBAYTAlVT MRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y azE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEg KGMpMDUxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuVmVyaVNpZ24g Q2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMgIQJ4yVqTR+lHwijj5pLs+dqzCC AQUGCyqGSIb3DQEJEAILMYH1oIHyMIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24s IEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9m IHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQLExVQZXJz b25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBT dWJzY3JpYmVyIENBIC0gRzICECeMlak0fpR8Io4+aS7PnaswDQYJKoZIhvcNAQEBBQAEggEAYatI Oh5RId2JuHf0Rj9jD67kBogz374Kly/Xk5FbhDCAN7qOLXIwC+9TPZaN6WqbRbOato0VcybBe3mu QfRyWG7V17OEohXv7dkL8sUDy5/TRp8fjp+YVjUtJuuleX4kI4oh6mL2F45j5//62tdzaREgQB0h Cl7nPvmXUSaH2fUn+ggyRPiQYmf1ebd/0gLZOOAyw2tc20yAsxjhe24VKGymQnG1+LmvLendpHwa CWer+aY2u+AMdSPJjh21D8hLJk/xF1+AIZHohrZhDbFDb1Hg6Xa1wbB/T8fEJcb3WdpDfb7NSGts qiE/k6uu8nOhp1LjrTkjg/XLLztB7Ui+VQAAAAAAAA== --Apple-Mail-2--790472240--