On Sat, Aug 22, 2015 at 2:54 PM David Unric wrote: > Simple script takes about 6.2 MiB of memory with Ruby 2.2.3 on x86_64 > machine > > ruby --disable=gems,rubyopt -e 'gets' > > so 31 - 5 - 6.2 ≈ 20 MiB is for Rack > No. These are not comparable numbers. You've included random gems in there. Here: ~rack/rack % ruby -v ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-linux] ~rack/rack % ruby --disable=gems -e 'system "ps -orss= #$$"' 6028 ~rack/rack % ruby --disable=gems -Ilib -rrack -e 'system "ps -orss= #$$"' 6244 That's 216KB. But I'm bending the truth too now, so in the interest of being honest, lets load ALL of rack: ~rack/rack % ruby --disable=gems -e 'system "ps -orss= #$$"' 6060 ~rack/rack % ruby --disable=gems -Ilib -rrack -e 'Dir["lib/**/*.rb"].each { |f| begin; require f[%r%lib/(.*)$%, 1]; rescue LoadError; puts "skipped #{f}"; end}; system "ps -orss= #$$"' skipped lib/rack/session/memcache.rb skipped lib/rack/handler/thin.rb skipped lib/rack/handler/lsws.rb skipped lib/rack/handler/scgi.rb skipped lib/rack/handler/fastcgi.rb 20244 So that's ~14MB. HOWEVER, this is counting GC_MALLOC_LIMIT space, and the waste I produced loading things: :heap_live_num=>73962, :heap_free_num=>50543 Ruby 1.8.7, which the most recent releases dropped support for (you can fix this by undoing silly stylistic patches that I rejected for years), would only add about 10MB of heap fully loaded too. But lets look now at something a bit more real than just smash loading support for 9 servers, several authentication systems, session stuff, and maybe 10 (?) middleware that you're not using, and instead just load an actual minimal app: ~rack/rack % ruby --disable=gems -Ilib -rrack -e 'require "rack/server"; Rack::Server.new.start; system "ps -orss= #$$"' example/lobster.ru [2015-08-24 15:36:35] INFO WEBrick 1.3.1 [2015-08-24 15:36:35] INFO ruby 2.0.0 (2013-06-27) [x86_64-linux] [2015-08-24 15:36:35] INFO WEBrick::HTTPServer#start: pid=16353 port=9292 ^C[2015-08-24 15:36:37] INFO going to shutdown ... [2015-08-24 15:36:37] INFO WEBrick::HTTPServer#start done. 16444 So now we're talking <10MB for a fully running application using webrick (which is not the smallest), on ruby 2.0 still, which is pretty fat. To just prove that, lets demonstrate some cgi.rb simplicity: ~rack/rack % env SCRIPT_NAME=/ HTTP_VERSION=1.0 SERVER_PORT=101010 SERVER_NAME=cgi REQUEST_METHOD=GET ruby --disable=gems -Ilib -rrack -e 'require "rack/server"; Rack::Server.new.start; system "ps -orss= #$$"; p GC.stat' example/lobster.ru Status: 200 Content-Length: 592 Lobstericious!
                             ,.---._
                   ,,,,     /       `,
                    \\\\   /    '\_  ;
                     |||| /\/``-.__\;'
                     ::::/\/_
     {{`-.__.-'(`(^^(^^^(^ 9 `.========='
    {{{{{{ { ( ( (  (   (-----:=
     {{.-'~~'-.(,(,,(,,,(__6_.'=========.
                     ::::\/\
                     |||| \/\  ,-'/,
                    ////   \ `` _/ ;
                   ''''     \  `  .'
                             `---'

flip!

crash!

13232 {:count=>4, :heap_used=>136, :heap_length=>138, :heap_increment=>2, :heap_live_num=>55223, :heap_free_num=>20068, :heap_final_num=>0, :total_allocated_object=>84612, :total_freed_object=>29389} So that's 7172KB, or about 7MB, and again, look at GC.stat there, a large part of the heap is free. Not saying this is bad but if there is a way how to push it lower. AFAIK > all ruby web servers are built upon Rack, not aware of any going its own > independent way. > Nope. Most webservers just implement the Rack SPEC. They shouldn't need any hard dependency on Rack itself. If memory serves, and I haven't been doing Rack/Ruby for a while, but Puma has no hard dependency on Rack. There are novelty servers such as the evented servers and so on that also implement other specs/protocols (normally their own). Thanks for link to derailed_benchmarks gem. Looks promising, definitely > would take a closer look at it. > > Btw. CRuby does not seem any worse then his "main" rival. Equivalent > script takes about 6.4 MiB with Python 2.7.10 and 7.7 MiB on Python 3.4.3. > Well since YARV and the various GC tunings it's got a lot more heavy actually, and a lot more overhead and so on have landed in the stdlib (e.g. "packaged gems" and so on). This has a lot to do with the fact that the bulk of users for a long time have been running fat applications (e.g. rails). I still have some suuuuper old campfire daemons around that after many years of uptime are still sitting on a less than 10MB rss. *shrug* Some final notes: Rack::Server isn't all that light, it uses option parsers and maps env vars and so on, and creates some garbage. It may also be loading more than it needs to. You could setup a much lighter runtime by using the Handler API directly and configuring things in more memory efficient way (i.e. hardcoded). That's up to you. I also made a mistake in the above runs, but now can't be bothered to fix it, which is that they're all using the Rack development environment, which is loading middleware you probably don't need in production (e.g. rack/show_exceptions). You'll only really notice the difference if you trim the GC though, as there's more other garbage going on to expand the heap than is evident just avoiding a few more middleware - test it out yourself adding and removing the `-E none` parameter. I just re-ran with 2.2.3 and it seems to do better in not making excessive allocations: ~rack/rack 2.2.3 % ruby --disable=gems -e 'system "ps -orss= #$$"' 6192 ~rack/rack 2.2.3 % ruby -Ilib -rrack -e 'Dir["lib/**/*.rb"].each { |f| begin; require f[%r%lib/(.*)$%, 1]; rescue LoadError; puts "skipped #{f}"; end}; system "ps -orss= #$$"' skipped lib/rack/session/memcache.rb skipped lib/rack/handler/thin.rb skipped lib/rack/handler/lsws.rb skipped lib/rack/handler/scgi.rb skipped lib/rack/handler/fastcgi.rb 20400 ~rack/rack 2.2.3 % env SCRIPT_NAME=/ HTTP_VERSION=1.0 SERVER_PORT=101010 SERVER_NAME=cgi REQUEST_METHOD=GET ruby --disable=gems -Ilib -rrack -e 'require "rack/server"; Rack::Server.new.start; system "ps -orss= #$$"; p GC.stat' example/lobster.ru -E none Status: 200 Content-Length: 592 Lobstericious!
                             ,.---._
                   ,,,,     /       `,
                    \\\\   /    '\_  ;
                     |||| /\/``-.__\;'
                     ::::/\/_
     {{`-.__.-'(`(^^(^^^(^ 9 `.========='
    {{{{{{ { ( ( (  (   (-----:=
     {{.-'~~'-.(,(,,(,,,(__6_.'=========.
                     ::::\/\
                     |||| \/\  ,-'/,
                    ////   \ `` _/ ;
                   ''''     \  `  .'
                             `---'

flip!

crash!

11268 {:count=>7, :heap_allocated_pages=>74, :heap_sorted_length=>75, :heap_allocatable_pages=>0, :heap_available_slots=>30163, :heap_live_slots=>29829, :heap_free_slots=>334, :heap_final_slots=>0, :heap_marked_slots=>13743, :heap_swept_slots=>8794, :heap_eden_pages=>74, :heap_tomb_pages=>0, :total_allocated_pages=>74, :total_freed_pages=>0, :total_allocated_objects=>89907, :total_freed_objects=>60078, :malloc_increase_bytes=>712520, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>4, :major_gc_count=>3, :remembered_wb_unprotected_objects=>346, :remembered_wb_unprotected_objects_limit=>692, :old_objects=>10816, :old_objects_limit=>21634, :oldmalloc_increase_bytes=>1082840, :oldmalloc_increase_bytes_limit=>16777216} That's only 5MB fully loaded, after serving a request... You could also consider compiling with / LD_PRELOADING in jemalloc or tcmalloc or whatever, although that doesn't tend to make too much difference in heap size, it's more of a performance concern. Depends a bit on your platform and allocator(s), but there's a possibility these could save you relevant heap space at this level. ... blah ... I'll leave it there for now. You said you "measured" this and found that Rack was taking 26MB of 31. That doesn't really make sense, can you please provide a reproduction case? > > > On Saturday, August 22, 2015 at 10:54:44 PM UTC+2, richard schneeman wrote: > >> Just running `irb` with no libraries is 9.9 mb for me with Ruby 2.2.3. In >> Ruby 2.0 it's 17.9mb for me. Ruby is not known to be the most memory >> efficient language. >> >> I wrote a tool to help debug memor use in apps, check it out >> https://github.com/schneems/derailed_benchmarks >> >> You can run >> >> $ derailed bundle:mem >> >> To see the memory impact of all the gems you're using at require time. >> >> Here's more information: >> http://www.schneems.com/2015/05/11/how-ruby-uses-memory.html >> >> >> >> >> --- >> Richard Schneeman >> http://www.schneems.com >> >> >> >> On Sat, Aug 22, 2015 at 3:39 PM, David Unric wrote: >> > Hi, >>> >>> I'm asking if am I doing something wrong or is normal for a minimal rack >>> application have memory footprint over 25 MiB ? >>> >>> I'd like to reduce the memory usage as possible for deployment of a >>> Sinatra app on small x86 Linux NAS device. I did measured the memory usage >>> and Rack occupies about excessive 26 from total of 31 MiB (the rest is Thin >>> server + the app). >>> >>> Is there some Rack-lite or some trick how to shake-off unnecessary code >>> to get it on diet ? >>> >>> Thanks. >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Rack Development" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> email to rack-devel+...@googlegroups.com. >>> >> >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "Rack Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rack-devel+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "Rack Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to rack-devel+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.