Bootstrapping rackup with multiple rack gems installed

Bootstrapping rackup with multiple rack gems installed

[Joshua Peek]

This came up as an issue in Rails 2.3 along with the release of Rack
1.2. Rails 2.3 is locked to Rack 1.0 so if Rack 1.2 is activated
before hand, Rails will fail to boot. Most people now have Rack 1.0
and 1.2 both installed. In a prefect rubygems world everything would
work fine. (This is why I don't use RG, but a ton of people are still
using Rails 2.3 and RG)

Starting Rails with `script/server` works fine because Rails boots
first. You can start Rails with `rackup` but you have to make sure you
are using the matching rack version with `rackup _1.0.1_` which is
kind of lame though. The major issue is with servers like unicorn and
passenger which have their own bootstrapping scripts.

If you boot a Rails 2.3 app with `unicorn` and Rack 1.2 is installed,
Unicorn attempts to require rack pulling in the latest version since
it doesn't know any better then boots Rails causing the gem conflict.
Similar deal with passenger.

Unicorn has a valid reason for requiring rack since it needs to load
Rack::Builder before it evals the config.ru file. The use of the
builder dsl in .ru files causes a serious fundamental problem with
loading rack. The application can never specify which version of rack
to use since it has to be chosen before its loaded.

Possible workarounds:

1) Always bootstrap with the correct version of rackup with `rackup
_1.0.0_`, `/path/to/rack/1.0.0/bin/rackup`, etc.
2) Unicorn and Passenger should allow you to specify which rack
version to use.
3) Specify a load path or environment file on boot that setups the
correct paths or activates the right version of rack. This could be
done by requiring an environment that loads Bundler or similar tools.
4) Hacky version independent version of rackup. I hacked this up to
demonstration how http://gist.github.com/560926 you can lazily
evaluate the builder dsl after the application loads.

Are there anymore workarounds? Whats the suggested fix? I'd like to
hear some opinions on the whole bootstapping matter.

Re: Bootstrapping rackup with multiple rack gems installed

[Steve Klabnik]

Forgive my overly simplistic answer, but isn't this what Bundler is for?

Re: Bootstrapping rackup with multiple rack gems installed

[Konstantin Haase]

On Sep 1, 2010, at 21:16 , Steve Klabnik wrote:

> Forgive my overly simplistic answer, but isn't this what Bundler is for?

Rails 2.3 does not include Bundler.

Konstantin

Re: Bootstrapping rackup with multiple rack gems installed

[Steve Klabnik]

Whoops! I knew there was some reason that seemed stupid. Thanks.

On Wed, Sep 1, 2010 at 3:20 PM, Konstantin Haase <k.haase@finn.de> wrote:
>
> On Sep 1, 2010, at 21:16 , Steve Klabnik wrote:
>
>> Forgive my overly simplistic answer, but isn't this what Bundler is for?
>
> Rails 2.3 does not include Bundler.
>
> Konstantin
>

Re: Bootstrapping rackup with multiple rack gems installed

[Joshua Peek]

On Sep 1, 2:16 pm, Steve Klabnik <st...@steveklabnik.com> wrote:
> Forgive my overly simplistic answer, but isn't this what Bundler is for?

This affects anyone using Rubygems. The same situation is possible on
Sinatra or any Rack app.

Re: Bootstrapping rackup with multiple rack gems installed

[Steve Klabnik]

I use bundler with Sinatra all the time, it's not just a Rails thing.

On Wed, Sep 1, 2010 at 3:23 PM, Joshua Peek <josh@joshpeek.com> wrote:
> On Sep 1, 2:16 pm, Steve Klabnik <st...@steveklabnik.com> wrote:
>> Forgive my overly simplistic answer, but isn't this what Bundler is for?
>
> This affects anyone using Rubygems. The same situation is possible on
> Sinatra or any Rack app.
>

Re: Bootstrapping rackup with multiple rack gems installed

[Yehuda Katz]

You can use bundler with Rails 2.3 and quite a few people do:
http://gembundler.com/rails23.html

Bundler's Rails 3 integration is very light; it's just a few lines in
the default generated app.

Sent from my iPhone

On Sep 1, 2010, at 12:21 PM, Steve Klabnik <steve@steveklabnik.com> wrote:

> Whoops! I knew there was some reason that seemed stupid. Thanks.
>
> On Wed, Sep 1, 2010 at 3:20 PM, Konstantin Haase <k.haase@finn.de> wrote:
>>
>> On Sep 1, 2010, at 21:16 , Steve Klabnik wrote:
>>
>>> Forgive my overly simplistic answer, but isn't this what Bundler is for?
>>
>> Rails 2.3 does not include Bundler.
>>
>> Konstantin
>>

Re: Bootstrapping rackup with multiple rack gems installed

[Konstantin Haase]

My point was just that it does not use it out of the box, so just saying "that's what bundler is for" does not solve the issue in general. Although it is debatable if that issue has to be solved at all.

Konstantin

On Sep 1, 2010, at 21:25 , Yehuda Katz wrote:

> You can use bundler with Rails 2.3 and quite a few people do:
> http://gembundler.com/rails23.html
> 
> Bundler's Rails 3 integration is very light; it's just a few lines in
> the default generated app.
> 
> Sent from my iPhone
> 
> On Sep 1, 2010, at 12:21 PM, Steve Klabnik <steve@steveklabnik.com> wrote:
> 
>> Whoops! I knew there was some reason that seemed stupid. Thanks.
>> 
>> On Wed, Sep 1, 2010 at 3:20 PM, Konstantin Haase <k.haase@finn.de> wrote:
>>> 
>>> On Sep 1, 2010, at 21:16 , Steve Klabnik wrote:
>>> 
>>>> Forgive my overly simplistic answer, but isn't this what Bundler is for?
>>> 
>>> Rails 2.3 does not include Bundler.
>>> 
>>> Konstantin
>>> 

Re: Bootstrapping rackup with multiple rack gems installed

[Lawrence Pit]

Loads of things are not included out of the box, including unicorn and 
passenger. So if you're going to use that it's not a big deal to ask to 
use bundler as well I'd think; besides it makes your app so much better 
if you can rely on bundler. I've been using bundler with 2.3.5 / 
passenger and 2.3.8 / unicorn apps since bundler 0.8x, and have no 
bootstrap issues, certainly not in the realm of rack.


Cheers
Lawrence

> My point was just that it does not use it out of the box, so just saying "that's what bundler is for" does not solve the issue in general. Although it is debatable if that issue has to be solved at all.
>
> Konstantin
>
> On Sep 1, 2010, at 21:25 , Yehuda Katz wrote:
>
>> You can use bundler with Rails 2.3 and quite a few people do:
>> http://gembundler.com/rails23.html
>>
>> Bundler's Rails 3 integration is very light; it's just a few lines in
>> the default generated app.
>>
>> Sent from my iPhone
>>
>> On Sep 1, 2010, at 12:21 PM, Steve Klabnik<steve@steveklabnik.com>  wrote:
>>
>>> Whoops! I knew there was some reason that seemed stupid. Thanks.
>>>
>>> On Wed, Sep 1, 2010 at 3:20 PM, Konstantin Haase<k.haase@finn.de>  wrote:
>>>> On Sep 1, 2010, at 21:16 , Steve Klabnik wrote:
>>>>
>>>>> Forgive my overly simplistic answer, but isn't this what Bundler is for?
>>>> Rails 2.3 does not include Bundler.
>>>>
>>>> Konstantin
>>>>
>

Re: Bootstrapping rackup with multiple rack gems installed

[Randy Fischer]

Funny, I just hit this problem myself today, doing some test deploys
using the newest version of bundler.

Seems like the minimal pre-install requirements for deploying
a sinatra 1.x app with bundler on a server running passenger
phusion is a recentish version of gem (I guess), bundler itself
of course and rack 1.2.  Not sure why bundler is pulling in 1.2
though.

My problem is configuring capistrano to get bundler to do
the right things on the deployment side - pull my gems
out of the repository, no problem there, tell bundler to
use them with the --local switch, fine, but not rebuild
everything.  Currently, I have this pulled out of the
repository

   .../vendor/bundle/ruby/1.8/cache

then capistrano builds linktrees into a shared directory
for bin, doc gems and specifications.  The shared directory
is not torn down over deploys.

Gotta be a better way.  Either there's too much magic, or not
enough, heh?

Thanks
Randy Fischer

Re: Bootstrapping rackup with multiple rack gems installed

[Eric Wong]

Joshua Peek <josh@joshpeek.com> wrote:
> 2) Unicorn and Passenger should allow you to specify which rack
> version to use.

I've looked into this before.  Once the gemspec for Unicorn is loaded,
the latest rack is already activated, so it's too late.  And the gemspec
that lands during the installation is always regenerated by RubyGems
so I can't put conditional logic in the gemspec for runtime use.

This would still work for people who install Unicorn using setup.rb,
but that's a minority.

> 3) Specify a load path or environment file on boot that setups the
> correct paths or activates the right version of rack. This could be
> done by requiring an environment that loads Bundler or similar tools.

I do something like this with Isolate for the few apps I've deployed.
I used to use RUBYLIB for most "hello world" demo apps.

> 4) Hacky version independent version of rackup. I hacked this up to
> demonstration how http://gist.github.com/560926 you can lazily
> evaluate the builder dsl after the application loads.

Yikes...

> Are there anymore workarounds? Whats the suggested fix? I'd like to
> hear some opinions on the whole bootstapping matter.

Multiple Ruby installations?

Or somebody (not me :P) could maintain gems like: "unicorn-old-rack-XYZ"
that have hard dependencies on older racks and only have a bin/
wrapper that loads Unicorn after it activates the older Rack.

Honestly, all options suck and people have been trying to solve these
types of packaging/dependency for years now (not just in Ruby).  I don't
know...

-- 
Eric Wong

Re: Bootstrapping rackup with multiple rack gems installed

[James Tucker]

On 1 Sep 2010, at 16:14, Joshua Peek wrote:

> This came up as an issue in Rails 2.3 along with the release of Rack
> 1.2. Rails 2.3 is locked to Rack 1.0 so if Rack 1.2 is activated
> before hand, Rails will fail to boot. Most people now have Rack 1.0
> and 1.2 both installed. In a prefect rubygems world everything would
> work fine. (This is why I don't use RG, but a ton of people are still
> using Rails 2.3 and RG)

It's not RubyGems fault if people don't use dependency manifests and/or call Kernel#gem. RubyGems activates dependencies in order to avoid cross-dependency failures at runtime - this "error" is protecting against potentially far far more horrible bugs. Dependency management is not a "perfect world". If you have a serious suggestion about how this could possibly be solved in RubyGems without removing features or failing critical use cases, please pass your thoughts on as if you can genuinely solve it, I'll get it committed to RubyGems immediately.

> Starting Rails with `script/server` works fine because Rails boots
> first. You can start Rails with `rackup` but you have to make sure you
> are using the matching rack version with `rackup _1.0.1_` which is
> kind of lame though. The major issue is with servers like unicorn and
> passenger which have their own bootstrapping scripts.

If they support multiple versions of rack, then they need to load the version they want as configured by the user.

> If you boot a Rails 2.3 app with `unicorn` and Rack 1.2 is installed,
> Unicorn attempts to require rack pulling in the latest version since
> it doesn't know any better then boots Rails causing the gem conflict.
> Similar deal with passenger.
> 
> Unicorn has a valid reason for requiring rack since it needs to load
> Rack::Builder before it evals the config.ru file. The use of the
> builder dsl in .ru files causes a serious fundamental problem with
> loading rack. The application can never specify which version of rack
> to use since it has to be chosen before its loaded.
> 
> Possible workarounds:
> 
> 1) Always bootstrap with the correct version of rackup with `rackup
> _1.0.0_`, `/path/to/rack/1.0.0/bin/rackup`, etc.
> 2) Unicorn and Passenger should allow you to specify which rack
> version to use.
> 3) Specify a load path or environment file on boot that setups the
> correct paths or activates the right version of rack. This could be
> done by requiring an environment that loads Bundler or similar tools.
> 4) Hacky version independent version of rackup. I hacked this up to
> demonstration how http://gist.github.com/560926 you can lazily
> evaluate the builder dsl after the application loads.
> 
> Are there anymore workarounds? Whats the suggested fix? I'd like to
> hear some opinions on the whole bootstapping matter.

I generally write independent boostrap code for applications whenever necessary, to me this is as important a practice as writing proper build scripts and ensuring that 'make' or 'rake' from the top level boostraps, builds, and runs tests or the application as much without fail as is possible. I know many developers find this diligent practice "boring", but it is very important. Bundler made this all a hell of a lot easier.

The idea from 4 could maybe be extended to pushing the whole Rackup and Rack::Builder DSL (which seldom changes, and is not that tied to Rack spec or the Rack application API) to another gem. That gem needn't depend on rack (in theory it could load other things), and could provide a DSL method to load a specific version of rack dynamically. This would mean that users could once again take the easy way out, avoid writing manifests or builds, and simply spec a rack version in the rackup DSL.

Something like this could work:

    rack '1.0.1' # calls Kernel#gem with 'rack', '1.0.1'
    map '/' do ...

In scenarios where RubyGems is not in use, the users would either have to load a plugin that alters the Kernel#gem call to something else (although I don't know of another package manager in ruby that actually manages dependencies properly at runtime), otherwise if it's using the filesystem namespace for versioning (as most alternatives do), one could just omit the call, and rely on a later call to "require 'rack'".