From mboxrd@z Thu Jan 1 00:00:00 1970 Delivered-To: chneukirchen@gmail.com Received: by 10.229.70.138 with SMTP id d10cs19514qcj; Mon, 25 Jul 2011 13:23:32 -0700 (PDT) Return-Path: Received-SPF: pass (google.com: domain of rack-devel+bncCOCkpKnzHBDBobfxBBoEcBJ9bw@googlegroups.com designates 10.220.198.203 as permitted sender) client-ip=10.220.198.203; Authentication-Results: mr.google.com; spf=pass (google.com: domain of rack-devel+bncCOCkpKnzHBDBobfxBBoEcBJ9bw@googlegroups.com designates 10.220.198.203 as permitted sender) smtp.mail=rack-devel+bncCOCkpKnzHBDBobfxBBoEcBJ9bw@googlegroups.com; dkim=pass header.i=rack-devel+bncCOCkpKnzHBDBobfxBBoEcBJ9bw@googlegroups.com Received: from mr.google.com ([10.220.198.203]) by 10.220.198.203 with SMTP id ep11mr2425274vcb.41.1311625411912 (num_hops = 1); Mon, 25 Jul 2011 13:23:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=beta; h=x-beenthere:received-spf:date:from:to:message-id:in-reply-to :references:subject:x-mailer:mime-version:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-google-group-id:list-post:list-help:list-archive:sender :list-subscribe:list-unsubscribe:content-type; bh=+ccbOOEYjD809ns+599y23VHg8snAaIv5hg+0X9dfjA=; b=5zA1ktAGDepF4WzAxoxTgPeOzj+1EwbRuwMgwnQy3+WLNnc1V9f+WX6W9LAxpCNUcS MDslROjAGHVyORIhBNpUsxnwMP7AZJDzeYlwRbRv+c0ugD8RP63AJylfb8J4ivg48Pf6 3yXep9U2L3r/AF6fHZEkqHWRUA9x80pNSYXvg= Received: by 10.220.198.203 with SMTP id ep11mr753781vcb.41.1311625409749; Mon, 25 Jul 2011 13:23:29 -0700 (PDT) X-BeenThere: rack-devel@googlegroups.com Received: by 10.52.161.199 with SMTP id xu7ls1738467vdb.0.gmail; Mon, 25 Jul 2011 13:23:28 -0700 (PDT) Received: by 10.52.91.74 with SMTP id cc10mr1450564vdb.37.1311625408677; Mon, 25 Jul 2011 13:23:28 -0700 (PDT) Received: by 10.52.91.74 with SMTP id cc10mr1450563vdb.37.1311625408654; Mon, 25 Jul 2011 13:23:28 -0700 (PDT) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by gmr-mx.google.com with ESMTPS id h2si2683205vdv.0.2011.07.25.13.23.27 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 25 Jul 2011 13:23:27 -0700 (PDT) Received-SPF: pass (google.com: domain of jballanc@gmail.com designates 209.85.216.175 as permitted sender) client-ip=209.85.216.175; Received: by qyk30 with SMTP id 30so1211866qyk.13 for ; Mon, 25 Jul 2011 13:23:27 -0700 (PDT) Received: by 10.224.33.73 with SMTP id g9mr1945108qad.179.1311625407298; Mon, 25 Jul 2011 13:23:27 -0700 (PDT) Received: from DeepThought.local (h-64-236-128-13.nat.aol.com [64.236.128.13]) by mx.google.com with ESMTPS id s14sm3747768qct.30.2011.07.25.13.23.26 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 25 Jul 2011 13:23:26 -0700 (PDT) Date: Mon, 25 Jul 2011 16:23:25 -0400 From: Joshua Ballanco To: rack-devel@googlegroups.com Message-ID: <8F93F290D08845C89C2A7C1519A9138B@gmail.com> In-Reply-To: References: Subject: Re: Session collisions on rails 3.1rc4 (authlogic, omniauth, memcache store, passenger) X-Mailer: sparrow 1.3.1 (build 814.15) MIME-Version: 1.0 X-Original-Sender: jballanc@gmail.com X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jballanc@gmail.com designates 209.85.216.175 as permitted sender) smtp.mail=jballanc@gmail.com; dkim=pass (test mode) header.i=@gmail.com Reply-To: rack-devel@googlegroups.com Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: X-Google-Group-Id: 486215384060 List-Post: , List-Help: , List-Archive: Sender: rack-devel@googlegroups.com List-Subscribe: , List-Unsubscribe: , Content-Type: multipart/alternative; boundary="4e2dd0bd_567bd50a_149ab" --4e2dd0bd_567bd50a_149ab Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Have you considered the possibility that memcache might be recycling keys on you? We had an issue a while back where we were using memcache for both fragment caching and session storage. Occasionally, we would get exceptions in retrieving a session, and looking at the error message it was clear that what was retrieved from memcache was a fragment and not a session. Unfortunately, we moved back to a cookie-based session store before we had a chance to look deeper into the issue. What I can say is that based on the key generation scheme we were using for sessions and cache keys, there was effectively 0 chance that we were duplicating keys. Instead, it seemed like memcache was re-using slots for different keys when we started exhausting free slots. Hope that helps! - Joshua Ballanco On Monday, July 25, 2011 at 2:22 PM, Neil wrote: > While it's entirely possible that this issue is caused by some other > factor, but we are getting session collisions as well as an issue > where one user is getting another user's session. This is clearly > bad, but I cannot for the life of me figure out how this could even > happen in the first place. The code looks thread safe to me, and a > quick discussion on #ruby-lang seems to support that. > > Thoughts: > 1. Session IDs are being generated in the same sequence (uses > securerandom -> openssl which does not have a static seed) > 2. Threads. Looks good to me. > 3. Maybe memcached is returning something other than "STORED/ > NOT_STORED" for @pool.add(sid, session), but the operation still > succeeded? > 4. Gnomes. > > Any input is GREATLY appreciated. Please don't say "it's an RC, what > do you expect?" :) > > > From https://github.com/rack/rack/blob/master/lib/rack/session/memcache.rb > def generate_sid > loop do > sid = super > break sid unless @pool.get(sid, true) > end > end > > def get_session(env, sid) > with_lock(env, [nil, {}]) do > unless sid and session = @pool.get(sid) > sid, session = generate_sid, {} > unless /^STORED/ =~ @pool.add(sid, session) > raise "Session collision on '#{sid.inspect}'" > end > end > [sid, session] > end > end > > def set_session(env, session_id, new_session, options) > expiry = options[:expire_after] > expiry = expiry.nil? ? 0 : expiry + 1 > > with_lock(env, false) do > @pool.set session_id, new_session, expiry > session_id > end > end --4e2dd0bd_567bd50a_149ab Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Have you considered the possibility that memcache mi= ght be recycling keys on you=3F

We had an issue a whil= e back where we were using memcache for both fragment caching and session= storage. Occasionally, we would get exceptions in retrieving a session, = and looking at the error message it was clear that what was retrieved fro= m memcache was a fragment and not a session. Unfortunately, we moved back= to a cookie-based session store before we had a chance to look deeper in= to the issue. What I can say is that based on the key generation scheme w= e were using for sessions and cache keys, there was effectively 0 chance = that we were duplicating keys. Instead, it seemed like memcache was re-us= ing slots for different keys when we started exhausting free slots.
=

Hope that helps=21

- Joshua = Ballanco
=20

On Monday, July 25, 20= 11 at 2:22 PM, Neil wrote:

While it's entirely possible that thi= s issue is caused by some other
factor, but we are getting session col= lisions as well as an issue
where one user is getting another user's s= ession. This is clearly
bad, but I cannot for the life of me figure o= ut how this could even
happen in the first place. The code looks thre= ad safe to me, and a
quick discussion on =23ruby-lang seems to support= that.

Thoughts:
1. Session IDs are being generated in the sam= e sequence (uses
securerandom -> openssl which does not have a stat= ic seed)
2. Threads. Looks good to me.
3. Maybe memcached is ret= urning something other than =22STORED/
NOT=5FSTORED=22 for =40pool.add= (sid, session), but the operation still
succeeded=3F
4. Gnomes.
Any input is GREATLY appreciated. Please don't say =22it's an RC, w= hat
do you expect=3F=22 :)


=46rom https://gith= ub.com/rack/rack/blob/master/lib/rack/session/memcache.rb
de= f generate=5Fsid
loop do
sid =3D super
= break sid unless =40pool.get(sid, true)
end
end
def get=5Fsession(env, sid)
with=5Flock(env, =5Bnil= , =7B=7D=5D) do
unless sid and session =3D =40pool.get(sid)<= br> sid, session =3D generate=5Fsid, =7B=7D
unl= ess /=5ESTORED/ =3D=7E =40pool.add(sid, session)
raise =22= Session collision on '=23=7Bsid.inspect=7D'=22
end
= end
=5Bsid, session=5D
end
end
=
def set=5Fsession(env, session=5Fid, new=5Fsession, options) expiry =3D options=5B:expire=5Fafter=5D
expiry =3D ex= piry.nil=3F =3F 0 : expiry + 1

with=5Flock(env, false) do<= br> =40pool.set session=5Fid, new=5Fsession, expiry
= session=5Fid
end
end
 =20 =20 =20 =20
=20

--4e2dd0bd_567bd50a_149ab--