[ruby-core:39873] Windows IO and multi-platform MRI (was: 2.0 feature questionnaire)

[ruby-core:39873] Windows IO and multi-platform MRI (was: 2.0 feature questionnaire)

[Jon]

Hi Matz,

Forking this platform specific subtopic so as to not clutter responses to Koichi-san's original request.


> |One issue that continually fails to gain sustainable traction is identifying
> |the root causes and fixing the 1.9.x IO performance regressions on Windows.
>
> Besides I've heard that the IO performance has been improved on 1.9.3;

This is true and the performance improvement is much appreciated progress.


> Complaining motivates us opposite way.

You're mistaken; it was no complaint. Like feedback on any other important issue, I simply requested that this issue be tracked and prioritized against the others in order that it be resolved in a timely manner. Mislabeling 'feedback' as 'complaint' demotivates legitimate feedback.


> We need more contributors with deep knowledge of C programming on Windows.

Generally speaking, I agree. But I did not give generalities for this issue or the time before

  http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38342 

The specific issue I've raised is: resolving long-standing 1.9 IO performance problems on Windows given the current resourcing realities.

Ruby-core already has deeply experienced contributors very capable of solving hard multi-platform issues. Usa, Nobu, Yusuke, Aaron, and others do so on a regular basis.

The contributors I mention are more than capable of identifying and fixing the Windows IO issue which will likely turn out to be caused by incorrect API choice (msvcrt rather than Win32) and unoptimized algorithms. These contributors may need to spend a little time experimenting with the Windows APIs, but the Windows API is no match for their deep C, Ruby, and system expertise.

In fact, any tenacious developer with strong C skills, time, and interest in seeing a strong multi-platform MRI can contribute to solving this issue. For example, there's enough profiling experiments and API tracing to keep us busy for awhile. And the more eyes looking over the current implementation the better. That said, spelunking through io.c (~276K) and file.c (~125K) requires determination.

If part of the issue is setting up a Windows environment for interested ruby-core devs, let's discuss how to make that happen. In today's world of VMs like VirtualBox and our DevKit toolchain, the main issue is the Windows license. If the licenses are an issue, it's time to decide how to approach MSFT on how they can help an influencial language run better on Windows.


> Currently we have only one volunteer (with no payment).  I sympathize his burden and stress.

Yes and no.  While Luis is quite visible in the MRI on Windows space and has contributed greatly (some claim he's superhuman) with the RubyInstaller, rake-compiler and other projects, there are a number of others who have helped keep the MRI on Windows torch burning by contributing their free time in key areas.

What's needed is to take a step back and investigate ways to re-energize their efforts and better enable their contributions.


In closing, I ask that you consider the following two suggestions. One longer term, and one shorter term:

1. As the father of Ruby, consider more consistently and vocally promoting Ruby as a multi-platform language. People need to repeatedly hear from you that Ruby is a great language for developing real-world solutions regardless of whether they're using *nix, OSX, or Windows. This message also has very real business implications.

Hearing this message directly from you reiterates your commitment to Ruby the language, provides critically important tone for MRI (and non-MRI) development, and helps attract knowledgable platform contributors.

IMO, it's mistake if Ruby becomes (mis)perceived as a "*nix/OSX-only" language, or MRI on Windows becomes (mis)perceived as "a science fair project" (overemphasis for effect all mine).


2. Commit to prioritizing resources to solve the Windows IO performance issues. This challenge needs additional experienced developer horsepower and I don't see any shortcuts or magic bullets.

Specifically, consider:

a) discussing the following with the most experienced cross-platform ruby-core devs:

   * Should MRI be a best-of-breed, multi-platform Ruby implementation or not?
   * Is the Windows IO performance issue a real problem, or a nice-to-have "Dream" for MRI?
   * If it is a problem, do we care?
   * If we care, are we willing to divert limited resources to determine and fix the root causes?
   * For how long are we willing to divert resources?
   * If we're not willing to divert resources, what other creative options do we see?
   * If we don't diminish this technical debt now, when?
   * How should this issue be prioritized against the other 2.0 "Requirements"?
   * Once root cause is found, are we OK with a short-term patch if longer term IO refactoring is needed?

b) depending upon the outcome from (a) ask whether any ruby-core contributor is interested enough in a multi-platform MRI to "adopt" the issue and add horsepower to Luis' efforts (and hopefully others) to resolve. 


If you've slogged through this far and still are not persuaded, let's see if I can make it more personal for you ;)

Here's my challenge in hopes that you more clearly see why I as an Arch/Ubuntu/Windows user am fighting so verbosely for this issue:

  * download either our installer or 7z archive from http://rubyinstaller.org/downloads/
  * glance over http://rubyonwindowsguides.github.com/book/ch02-01.html if needed
  * install our MSYS/MinGW DevKit toolchain https://github.com/oneclick/rubyinstaller/wiki/Development-Kit
  * install some gems (binary and source) and start playing with MRI on Windows

While not perfect, I think you'll find that the MRI on Windows experience is quite nice due to the hard work of all the RubyInstaller contributors and ruby-core devs who have committed Windows specific patches over the years.  MRI on Windows deserves ruby-core resourcing to quickly help resolve this key issue.

If you have any problems, don't waste a single second fighting them. Post a question to our ML http://groups.google.com/group/rubyinstaller or send a private email to both Luis and me. I'm not going to make commitments for Luis, but I think all of us RubyInstaller committers are interested in making enhancements that make easier for others to join in and quickly resolve this problem.


Obviously there are other options and perspectives for discussion, but for now let's see where this leads.

Thank you for your time, insight, and consideration.

Jon

---
blog: http://jonforums.github.com/
twitter: @jonforums

Most people die of a sort of creeping common sense, and discover when it
is too late that the only things one never regrets are one's mistakes.
                                                           - Oscar Wilde

[ruby-core:40085] Re: Windows IO and multi-platform MRI (was: 2.0 feature questionnaire)

[Roger Pack]

>> |One issue that continually fails to gain sustainable traction is identifying
>> |the root causes and fixing the 1.9.x IO performance regressions on Windows.
>>
>> Besides I've heard that the IO performance has been improved on 1.9.3;
>
> This is true and the performance improvement is much appreciated progress.

I think rubygems' load time has been improved, but I was surprised to
run some benchmarks a few days ago with 1.9.3RC1 and find it 20-100%
slower than 1.9.2
http://groups.google.com/group/rubyinstaller/browse_thread/thread/326973e5ed3feb22

FWIW.
-roger-