* [ruby-core:90749] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0
[not found] <redmine.issue-15472.20181227185820@ruby-lang.org>
@ 2018-12-27 18:58 ` mseneadza
2018-12-27 21:36 ` [ruby-core:90751] " Eric Wong
2019-01-12 21:03 ` [ruby-core:91048] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP insome " naruse
1 sibling, 1 reply; 3+ messages in thread
From: mseneadza @ 2018-12-27 18:58 UTC (permalink / raw)
To: ruby-core
Issue #15472 has been reported by mseneadza (Michael Seneadza).
----------------------------------------
Bug #15472: Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0
https://bugs.ruby-lang.org/issues/15472
* Author: mseneadza (Michael Seneadza)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: 2.6.0p0
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
It seems that invalid (JSON) data is being sent once the request's body passes a size threshold. I'm sending emails via the sparkpost.com API using the sparkpost_rails gem ( https://github.com/the-refinery/sparkpost_rails ). That gem uses Net::HTTP to make its API calls.
I upgraded my app from Ruby 2.5.3 to Ruby 2.6.0 on December 25th. Yesterday, on the 26th, my app attempted to send its nightly emails to users. I was flooded with these errors from the API:
~~~
{ "errors": [ { "message": "invalid data format\/type", "description": "Problems parsing request as json", "code": "1300" } ] }
~~~
After some debugging / trial & error I discovered that the length of the email messages seemed to be the problem. I haven't nailed down the exact maximum length that will work but I know that if the request body is 16,511 characters long the message will be sent. If it's 17,396 characters it will fail.
I've also tried replacing Net::HTTP with the rest_client gem. The full-sized messages get sent with no problems using RestClient.
So I can replace this (https://github.com/the-refinery/sparkpost_rails/blob/master/lib/sparkpost_rails/delivery_method.rb#L379):
~~~
def post_to_api
url = "https://api.sparkpost.com/api/v1/transmissions"
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Post.new(uri.path, @headers)
request.body = JSON.generate(@data)
http.request(request)
end
~~~
with
~~~
def post_to_api
url = "https://api.sparkpost.com/api/v1/transmissions"
RestClient.post(url, JSON.generate(@data), @headers)
end
~~~
and my (full-length, much larger than 17,396 characters) messages get sent with no problem.
I also tried logging the actual HTTP requests using the httplog gem (https://github.com/trusche/httplog) in Ruby 2.5.3 and 2.6.0. Unfortunately that showed the request payloads were identical. But I suspect that gem's logging does some conversion that isn't actually happening in the real data being sent to the SparkPost API.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 3+ messages in thread
* [ruby-core:90751] Re: [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0
2018-12-27 18:58 ` [ruby-core:90749] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0 mseneadza
@ 2018-12-27 21:36 ` Eric Wong
0 siblings, 0 replies; 3+ messages in thread
From: Eric Wong @ 2018-12-27 21:36 UTC (permalink / raw)
To: ruby-core
> https://bugs.ruby-lang.org/issues/15472
Looks like [Bug #15468] and r66582 fixes it
^ permalink raw reply [flat|nested] 3+ messages in thread
* [ruby-core:91048] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP insome cases with Ruby 2.6.0
[not found] <redmine.issue-15472.20181227185820@ruby-lang.org>
2018-12-27 18:58 ` [ruby-core:90749] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0 mseneadza
@ 2019-01-12 21:03 ` naruse
1 sibling, 0 replies; 3+ messages in thread
From: naruse @ 2019-01-12 21:03 UTC (permalink / raw)
To: ruby-core
Issue #15472 has been updated by naruse (Yui NARUSE).
Backport changed from 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: DONE
ruby_2_6 r66799 merged revision(s) 66582.
----------------------------------------
Bug #15472: Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0
https://bugs.ruby-lang.org/issues/15472#change-76268
* Author: mseneadza (Michael Seneadza)
* Status: Closed
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: 2.6.0p0
* Backport: 2.4: DONTNEED, 2.5: DONTNEED, 2.6: DONE
----------------------------------------
It seems that invalid (JSON) data is being sent once the request's body passes a size threshold. I'm sending emails via the sparkpost.com API using the sparkpost_rails gem ( https://github.com/the-refinery/sparkpost_rails ). That gem uses Net::HTTP to make its API calls.
I upgraded my app from Ruby 2.5.3 to Ruby 2.6.0 on December 25th. Yesterday, on the 26th, my app attempted to send its nightly emails to users. I was flooded with these errors from the API:
~~~
{ "errors": [ { "message": "invalid data format\/type", "description": "Problems parsing request as json", "code": "1300" } ] }
~~~
After some debugging / trial & error I discovered that the length of the email messages seemed to be the problem. I haven't nailed down the exact maximum length that will work but I know that if the request body is 16,511 characters long the message will be sent. If it's 17,396 characters it will fail.
I've also tried replacing Net::HTTP with the rest_client gem. The full-sized messages get sent with no problems using RestClient.
So I can replace this (https://github.com/the-refinery/sparkpost_rails/blob/master/lib/sparkpost_rails/delivery_method.rb#L379):
~~~
def post_to_api
url = "https://api.sparkpost.com/api/v1/transmissions"
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Post.new(uri.path, @headers)
request.body = JSON.generate(@data)
http.request(request)
end
~~~
with
~~~
def post_to_api
url = "https://api.sparkpost.com/api/v1/transmissions"
RestClient.post(url, JSON.generate(@data), @headers)
end
~~~
and my (full-length, much larger than 17,396 characters) messages get sent with no problem.
I also tried logging the actual HTTP requests using the httplog gem (https://github.com/trusche/httplog) in Ruby 2.5.3 and 2.6.0. Unfortunately that showed the request payloads were identical. But I suspect that gem's logging does some conversion that isn't actually happening in the real data being sent to the SparkPost API.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-01-12 21:03 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <redmine.issue-15472.20181227185820@ruby-lang.org>
2018-12-27 18:58 ` [ruby-core:90749] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0 mseneadza
2018-12-27 21:36 ` [ruby-core:90751] " Eric Wong
2019-01-12 21:03 ` [ruby-core:91048] [Ruby trunk Bug#15472] Invalid JSON data being sent from Net::HTTP insome " naruse
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).