From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-3.5 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_LOW,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from mail-qk0-x23c.google.com (mail-qk0-x23c.google.com [IPv6:2607:f8b0:400d:c09::23c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 871DF20285 for ; Thu, 29 Jun 2017 02:19:23 +0000 (UTC) Received: by mail-qk0-x23c.google.com with SMTP id 16sf6060847qkg.0 for ; Wed, 28 Jun 2017 19:19:23 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1498702762; cv=pass; d=google.com; s=arc-20160816; b=A+kkS8Jc6RERiJPuCyaLpSRhgyoA6JXCq8PkNXpiQmFA+dYRn5KzwgGLpCaXaZ5KgD FNkUEQrxj1OjlSyWRroNJGnkYuX3zUu51vmRSfc24Z0i5axtPGdy8Bq7h4Nvr9H6F97D G6WiLhjTLq6s6QeQrCQ1SCWeWwQSJjJFGAlmec42UHdlJyvnB5z/PFQ6jkYvkOJJWj+7 stXI1rvKO3JvzglKv1M6mRlYunOzCB+Si4Qo8MMem6Xbin4eIUKm+eI5urjkmmPEXI0T QwJIA8sMqqRyLZ7A+Q0SnpTCQWEf4Panj5WQwQ7g7Mf/d25BWKm92iEHcqtes5X0smim hG4g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:references:in-reply-to :message-id:date:subject:to:from:arc-authentication-results :arc-message-signature:mime-version:sender:dkim-signature :arc-authentication-results; bh=OYSeks89INNwr7wq40V/eJIeIPY78wxpbbWI37Ov5QI=; b=pB3nUx5Xlj+fGhJX34eTKGFJPkWrpnLxMRwhxlAxLBeAgZMSL9B2VsI5AiRiHfsZFX ZwXpfWz/kgSYZSdpHDmX5m2sviLvSuC8w5L8bjfD52oTcr7oQGIYxRAhVcGuVUFjneB5 s23saj/PbDFBoTJCat7sJ1PXK+/+P/Gb+tmGtsSklD4yGH+KCmWqsnM1tltMxeJqLu18 fLALwLX9ToHy2mN4hS6TQmultuhxnIb4362HUyHbiWkKjjT7x9h+gPfm0rwbMt/JxccM Ppp6dLPLkmRTWOm+y88/enF2sRJVx9riXqTHYeaZHzag1UNaFEDzVX4GfsvtlHdPY9eB wnlQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of e@80x24.org designates 2600:3c01::f03c:91ff:fe96:f5d6 as permitted sender) smtp.mailfrom=e@80x24.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:mime-version:from:to:subject:date:message-id:in-reply-to :references:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=OYSeks89INNwr7wq40V/eJIeIPY78wxpbbWI37Ov5QI=; b=URSgmCYJJRNGHylMm0jg6tbMW+Ij7TAGNSIHeUkb3I371rQAHViLXpk5ugI4cF0lKK hz8lGFnlY8tsow8WneTqGmGEysQhTRZ3i+VGvN1wgqLneuj91p+MqK3OQwNGrjDrbcaA ZcB+pSR3JwMxGUA6GSMOF/Mq7RSyvcmw+qfvVeAwedOPIqeCLYEWYntv0fDmofTxniir hP1tFnUDrk/82PMH+wbCsC+KS5mvS133+WpB+a274JP4hmOMSBtAh7JK9oTU0hgrdllB ztomLU6toIIxPxfSv/eXEUQwVjEqZgzGN9eJQj6xqqS7xxrJQztvPq5dVbZB+0YUXoUo viKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:mime-version:from:to:subject:date :message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=OYSeks89INNwr7wq40V/eJIeIPY78wxpbbWI37Ov5QI=; b=gLWi4Zpahnfrxq99zD1ZXBhxDSYQMNFaIGuxjRane+HHURdFjLHd3rCPRusrqgIW/E ZIEj0+gKL+60Qr7WB+ET+r9cOMEKeoqkoz3IjZeIiursjzdn4B3JrAjW+COVJvGjh8iK bQ85wHpx28L1AAuuhvqid2KJmaQ4lBINucp7nbJKLkIdTAgLBtERbtN5BHfW6/dw247P hk6xWjCFoh/H4gfbFqKMjzB0TmAc5bDy3FD6xGSjzzITNbnDGR3+6CeH5FRpoKsZ5LvT asizyJFSmLIIIy1HDC2bFg1pEYtOTPsmxYDMZmYa7k+5zmtRmsD5PfHctKkoHyvTmmwk ogNg== Sender: rack-devel@googlegroups.com X-Gm-Message-State: AKS2vOy1v/IRGelQky+UN1XybpiXgtXeYK8lmc6v+zESGUP4OtWmp6IW lvhLyC9/XDYyCg== X-Received: by 10.36.26.134 with SMTP id 128mr132050iti.3.1498702762638; Wed, 28 Jun 2017 19:19:22 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: rack-devel@googlegroups.com Received: by 10.107.7.92 with SMTP id 89ls4660720ioh.13.gmail; Wed, 28 Jun 2017 19:19:22 -0700 (PDT) X-Received: by 10.176.81.25 with SMTP id e25mr7759082uaa.7.1498702762074; Wed, 28 Jun 2017 19:19:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498702761; cv=none; d=google.com; s=arc-20160816; b=DgsvDhqkU5hhU5eRd4SXFBXzjqaEeSUH7t0wGld++30zE9PRXc/4alQEUDn5atZg3A na3Vq6f8yO/Jegm0U0WdjuirZA759AW1H7ujNM+ICt5VUDhc8afSGshqCtQWEScZtas4 3nvz6nfNka22CbEOZQ6ONAaE+BciMCzGHuhciEHZUMubP9cswpl+2BEwa9uOLQEUSBO+ VG+MwnOkJPZs6lX+PWKrrbvR/DEWmPGQwvdAEPpXJfDbvPG5HGC1bQkUUu+J/r0ksUIV mQtDj8HuwXktj+HjJQbGYDhL4X2U8wMYaf3AAay7fO7I68K9rSs4RixPeWy9uZidAcag 9f3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:to:from :arc-authentication-results; bh=Jw04hANrT618Mu2iExZJ3Qqa4BJ+jhXy9NQ43Ny35Co=; b=MNRCYfy8f/oWo7NDblfIYOP3nr1TWAHk/OqAYEc11MGcuPt/B2qhi2QWZuYLjw/bbr lYB6Iyd0drR7IFoVgvkjVCLXzCRzuKzDHLjKj+jGuBoi+YqwkTkO3aKNCe7Qwn2OjScw y4t2YzTv0sUP1XwN+O/9uJSmAidqE+Hg2Ecw/lOONDybL809S00csCPYEmXnia56WtZo ykeJKPlkeFqeRXCBNhr7jWdTX0LHGk3aB5+cBNFqyQv5QXzkXocNN/sX1KevERFKYB9V pXaiuEHiHXX3QEG6RwCgpvMwJI0lfvveMIYLzvsozRYzrkR0xacdS5hAV4/9yNxPGEqb LgjQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of e@80x24.org designates 2600:3c01::f03c:91ff:fe96:f5d6 as permitted sender) smtp.mailfrom=e@80x24.org Received: from dcvr.yhbt.net (dcvr.yhbt.net. [2600:3c01::f03c:91ff:fe96:f5d6]) by gmr-mx.google.com with ESMTPS id 206si808634pfb.3.2017.06.28.19.19.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Jun 2017 19:19:21 -0700 (PDT) Received-SPF: pass (google.com: domain of e@80x24.org designates 2600:3c01::f03c:91ff:fe96:f5d6 as permitted sender) client-ip=2600:3c01::f03c:91ff:fe96:f5d6; Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id CFAC220287 for ; Thu, 29 Jun 2017 02:19:15 +0000 (UTC) From: Eric Wong To: rack-devel@googlegroups.com Subject: [PATCH 3/5] deflater: support "sync: false" option Date: Thu, 29 Jun 2017 02:19:13 +0000 Message-Id: <20170629021915.22517-4-e@80x24.org> In-Reply-To: <20170629021915.22517-1-e@80x24.org> References: <20170629021915.22517-1-e@80x24.org> X-Original-Sender: e@80x24.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of e@80x24.org designates 2600:3c01::f03c:91ff:fe96:f5d6 as permitted sender) smtp.mailfrom=e@80x24.org Reply-To: rack-devel@googlegroups.com Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: X-Google-Group-Id: 486215384060 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Flushing after after very flush is great for real-time apps. However, flushing is inefficient when apps use Rack::Response to generate many small writes (e.g. Rack::Lobster). Allow users to disable the default "sync: true" behavior to reduce bandwidth usage, otherwise using Rack::Deflater can lead to using more bandwidth than without it. --- lib/rack/deflater.rb | 11 ++++++++--- test/spec_deflater.rb | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 3 deletions(-) diff --git a/lib/rack/deflater.rb b/lib/rack/deflater.rb index d575adf..821f708 100644 --- a/lib/rack/deflater.rb +++ b/lib/rack/deflater.rb @@ -24,11 +24,15 @@ module Rack # 'if' - a lambda enabling / disabling deflation based on returned boolean value # e.g use Rack::Deflater, :if => lambda { |env, status, headers, body| body.map(&:bytesize).reduce(0, :+) > 512 } # 'include' - a list of content types that should be compressed + # 'sync' - Flushing after every chunk reduces latency for + # time-sensitive streaming applications, but hurts + # compression and throughput. Defaults to `true'. def initialize(app, options = {}) @app = app @condition = options[:if] @compressible_types = options[:include] + @sync = options[:sync] == false ? false : true end def call(env) @@ -56,7 +60,7 @@ module Rack headers.delete('Content-Length') mtime = headers.key?("Last-Modified") ? Time.httpdate(headers["Last-Modified"]) : Time.now - [status, headers, GzipStream.new(body, mtime)] + [status, headers, GzipStream.new(body, mtime, @sync)] when "identity" [status, headers, body] when nil @@ -67,7 +71,8 @@ module Rack end class GzipStream - def initialize(body, mtime) + def initialize(body, mtime, sync) + @sync = sync @body = body @mtime = mtime end @@ -78,7 +83,7 @@ module Rack gzip.mtime = @mtime @body.each { |part| gzip.write(part) - gzip.flush + gzip.flush if @sync } ensure gzip.close diff --git a/test/spec_deflater.rb b/test/spec_deflater.rb index 0f27c85..410a143 100644 --- a/test/spec_deflater.rb +++ b/test/spec_deflater.rb @@ -372,4 +372,38 @@ describe Rack::Deflater do verify(200, response, 'gzip', options) end + + it 'will honor sync: false to avoid unnecessary flushing' do + app_body = Object.new + class << app_body + def each + (0..20).each { |i| yield "hello\n".freeze } + end + end + + options = { + 'deflater_options' => { :sync => false }, + 'app_body' => app_body, + 'skip_body_verify' => true, + } + verify(200, app_body, deflate_or_gzip, options) do |status, headers, body| + headers.must_equal({ + 'Content-Encoding' => 'gzip', + 'Vary' => 'Accept-Encoding', + 'Content-Type' => 'text/plain' + }) + + buf = '' + raw_bytes = 0 + inflater = auto_inflater + body.each do |part| + raw_bytes += part.bytesize + buf << inflater.inflate(part) + end + buf << inflater.finish + expect = "hello\n" * 21 + buf.must_equal expect + raw_bytes.must_be(:<, expect.bytesize) + end + end end -- EW -- --- You received this message because you are subscribed to the Google Groups "Rack Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to rack-devel+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.