Steps to get multiple interpreters per process...

Steps to get multiple interpreters per process...

[Sean Chittenden]

Can someone chart out what would need to happen to get multiple ruby
interpreters per process that way myself and others can do the leg
work?  -sc

-- 
Sean Chittenden

Re: Steps to get multiple interpreters per process...

[Yukihiro Matsumoto]

Hi,

In message "Steps to get multiple interpreters per process..."
    on 02/06/25, Sean Chittenden <sean@ruby-lang.org> writes:

|Can someone chart out what would need to happen to get multiple ruby
|interpreters per process that way myself and others can do the leg
|work?  -sc

We have to list all global variables and pack everything into a
interpreter struct (like Perl guys did once).  YACC parser still uses
globals, but it must be another story.

							matz.

Re: Steps to get multiple interpreters per process...

[Sean Chittenden]

> |Can someone chart out what would need to happen to get multiple
> |ruby interpreters per process that way myself and others can do the
> |leg work?  -sc
> 
> We have to list all global variables and pack everything into a
> interpreter struct (like Perl guys did once).  YACC parser still
> uses globals, but it must be another story.

Makes sense.  What I don't fully understand (I've been staring at this
screen for too long and am about to go for a run, so bare with me) is:

*) Once the globals are contained in a struct, where will they live?
 In a global list that gets locked every time a new interpreter
 instance gets added to the process?

*) How will functions that need to make use of this structure identify
 which stuct in the list is theirs to play with?

As an after thought, once the globals are all contained, someone with
Win32 knowledge could easily implement a win32 fork call that uses
CreateProcess() and then pass only the data structure to the new
process.  I'm going off of what I've skimmed from the PostgreSQL guys,
but that'd put another nail in the cygwin coffin.

-sc

-- 
Sean Chittenden

Re: Steps to get multiple interpreters per process...

[Matt Armstrong]

matz@ruby-lang.org (Yukihiro Matsumoto) writes:

> In message "Steps to get multiple interpreters per process..."
>     on 02/06/25, Sean Chittenden <sean@ruby-lang.org> writes:
>
> |Can someone chart out what would need to happen to get multiple ruby
> |interpreters per process that way myself and others can do the leg
> |work?  -sc
>
> We have to list all global variables and pack everything into a
> interpreter struct (like Perl guys did once).  YACC parser still
> uses globals, but it must be another story.

I imagine there might also be issues with Ruby stuff that touches the
process' global state: signal handlers, cwd, etc.  Multiple
interpreters might have to communicate with each other somehow for
this.  Or maybe not.

Re: Steps to get multiple interpreters per process...

[Yukihiro Matsumoto]

Hi,

In message "Re: Steps to get multiple interpreters per process..."
    on 02/06/25, Matt Armstrong <matt@lickey.com> writes:

|I imagine there might also be issues with Ruby stuff that touches the
|process' global state: signal handlers, cwd, etc.  Multiple
|interpreters might have to communicate with each other somehow for
|this.  Or maybe not.

process-wise states are often ignored by multiple interpreter
implementations (they are just shared), for example by tcl and perl.

							matz.

Re: Steps to get multiple interpreters per process...

[Chris Ross]

On Tue, 25 Jun 2002 02:50:45 Sean Chittenden wrote:
: > |Can someone chart out what would need to happen to get multiple
: > |ruby interpreters per process that way myself and others can do the
: > |leg work?  -sc
: > 
: > We have to list all global variables and pack everything into a
: > interpreter struct (like Perl guys did once).  YACC parser still
: > uses globals, but it must be another story.

The YACC parser can be made re-entrant by using Bison and the directive
%pure_parser. It is also possible to pass a pointer to the yyparse()
call by doing:

	#define YYPARSE_PARAM parm

Where you then can access the data passed to yyparse() with the name
parm in the rules of the grammar.

My main issue is lex, it is completely un-thread safe and is therefore
damn near impossible to use to allow for concurrent compilation. One
of these days I am going to rework these tools to remove global variables
and states to make them threadsafe. The only part of ferite I can't get
completely concurrent is the compilation - I have to do higher level
locking.

: Makes sense.  What I don't fully understand (I've been staring at this
: screen for too long and am about to go for a run, so bare with me) is:
: 
: *) Once the globals are contained in a struct, where will they live?
:  In a global list that gets locked every time a new interpreter
:  instance gets added to the process?

You would either have to:

1) Pass the structure a round the program such that it is accessable,
   both php and ferite do it this way [not sure about others as I have
   not looked in a while].

2) Thread specific data - the global data has a key, whenever you
   want the global data you get the pointer to by calling a method. The
   bad thing here is the obvious speed hit. Go and see 
   "man pthread_key_create" for the concept.

Either way is a nasty conversion [I had to do (1) in ferite]. (1) is a
much better long term goal.

: *) How will functions that need to make use of this structure identify
:  which stuct in the list is theirs to play with?

If you just pass the 'context' to each function and onwards [at first it
feels very odd] you actually remove any headaches, they are all just 
accessed like local variables.
 
: As an after thought, once the globals are all contained, someone with
: Win32 knowledge could easily implement a win32 fork call that uses
: CreateProcess() and then pass only the data structure to the new
: process.  I'm going off of what I've skimmed from the PostgreSQL guys,
: but that'd put another nail in the cygwin coffin.

What you are talking about there [I believe] is (2) with regard to 
thread specific data.

Regards,

Chris
--
+------------------------------------------------------------------+
| Chris Ross   |      chris@darkrock.co.uk | ctr@ferite.org        |
|              | http://www.darkrock.co.uk | http://www.ferite.org |
+------------------------------------------------------------------+
"Life is an onion and one peels away its layers crying."

Re: Steps to get multiple interpreters per process...

[ts]

>>>>> "C" == Chris Ross <chris@darkrock.co.uk> writes:

C> 1) Pass the structure a round the program such that it is accessable,
C>    both php and ferite do it this way [not sure about others as I have
C>    not looked in a while].

 How do you do it ?

 Via #define like perl ?


Guy Decoux

Re: Steps to get multiple interpreters per process...

[Chris Ross]

On Tue, 25 Jun 2002 09:24:56 ts wrote:
: >>>>> "C" == Chris Ross <chris@darkrock.co.uk> writes:
: 
: C> 1) Pass the structure a round the program such that it is accessable,
: C>    both php and ferite do it this way [not sure about others as I have
: C>    not looked in a while].
: 
:  How do you do it ?
: 
:  Via #define like perl ?

I store all things that are related to a execution instance in a
FeriteScript* and then pass that to all functions. They simply all 
take a FeriteScript* as the first argument. 

Regards,

Chris
--
+------------------------------------------------------------------+
| Chris Ross   |      chris@darkrock.co.uk | ctr@ferite.org        |
|              | http://www.darkrock.co.uk | http://www.ferite.org |
+------------------------------------------------------------------+
"Life is an onion and one peels away its layers crying."

Re: Steps to get multiple interpreters per process...

[ts]

>>>>> "C" == Chris Ross <chris@darkrock.co.uk> writes:

C> I store all things that are related to a execution instance in a
C> FeriteScript* and then pass that to all functions. They simply all 
C> take a FeriteScript* as the first argument. 

 OK, this mean modify all extensions written in C, right ?

 Now if I don't want a version of ruby with multiple interpreter, why give
 an extra argument for each functions.

 perl solve this with #define at compile time, and the same extension can
 be compiled with or without multi interpreter without modifications.


Guy Decoux

Re: Steps to get multiple interpreters per process...

[Chris Ross]

On Tue, 25 Jun 2002 10:05:50 ts wrote:
: >>>>> "C" == Chris Ross <chris@darkrock.co.uk> writes:
: 
: C> I store all things that are related to a execution instance in a
: C> FeriteScript* and then pass that to all functions. They simply all 
: C> take a FeriteScript* as the first argument. 
: 
:  OK, this mean modify all extensions written in C, right ?
:
:  Now if I don't want a version of ruby with multiple interpreter, why give
:  an extra argument for each functions.
: 
:  perl solve this with #define at compile time, and the same extension can
:  be compiled with or without multi interpreter without modifications.

Sure, it can be done with a define. You just stumble upon the issue 
that you can't mix modules for a multiple interpreter compiled ruby
with standard modules and visa versa. Instantly confusion is added to 
offering multiple downloads of binary forms. This is definatly an issue
as debian and other package managers have this problem.You also face 
the issue of people having to make their library compatible with both 
sides.

In ferite I made the decission the script would be there at all times
to keep things consistant and simple. This means that half the hard 
work of making modules thread safe is done, the rest is up to the 
programmer.

Unfortunatly because ruby has been around longer you face the issue 
of backwards compatibility. The only way to do this is to change CVS 
and get people to update their code against it. Yes its a lot of work 
- but the residual benifits are [imho] worth it.

Regards,

Chris
--
+------------------------------------------------------------------+
| Chris Ross   |      chris@darkrock.co.uk | ctr@ferite.org        |
|              | http://www.darkrock.co.uk | http://www.ferite.org |
+------------------------------------------------------------------+
"Life is an onion and one peels away its layers crying."