The Tech Shovel

1. 

   Ruby 1.9.1 modifies request headers

   I just ran across something in Ruby 1.9.1 that definitely violates the least-surprise principle (at least to me). It is in Net::HTTP#get.

   def get(path, initheader = {}, dest = nil, &block) # :yield: +body_segment+
         res = nil
         if HAVE_ZLIB
           unless  initheader.keys.any?{|k| k.downcase == "accept-encoding"}
             initheader["accept-encoding"] = "gzip;q=1.0,deflate;q=0.6,identity;q=0.3"
             @compression = true
           end
         end
         request(Get.new(path, initheader)) {|r|
           if r.key?("content-encoding") and @compression
             @compression = nil # Clear it till next set.
            the_body = r.read_body dest, &block
             case r["content-encoding"]
             when "gzip"
               r.body= Zlib::GzipReader.new(StringIO.new(the_body)).read
               r.delete("content-encoding")
             when "deflate"
               r.body= Zlib::Inflate.inflate(the_body);
               r.delete("content-encoding")
             when "identity"
               ; # nothing needed
             else
               ; # Don't do anything dramatic, unless we need to later
             end
           else
             r.read_body dest, &block
           end
           res = r
         }
         unless @newimpl
           res.value
           return res, res.body
         end
   
         res
       end
   

   At the top, you can see it sets “accept-encoding” in the headers if it isn’t set. When I didn’t expect this, my data source packed the response and the feed broke. I was trying to parse a DOM from it and it failed. Hard setting the accept-encoding header to UTF-8 fixed this issue.

   That assignment also modifies the header parameter passed in from the outside scope, so any redirects will have this problem too. I don’t think it should do this, but I haven’t found any open tickets. Has anyone else run into this?

1 year ago 1 note