You'll have to bear with me here because this is a long one. A few months ago we ran into a problem after upgrading some of our servers to ColdFusion 8.0.1 CHF3 from CHF1. We had an application that integrates with a web service running on a remote .Net/IIS server that started throwing a "Connection Failure" message for code that worked fine pre-upgrade. It turns out that in CF 8.0.1, CHF 2/3 there was a "bug" that was fixed for cfhttp:

72744 Fix for CFHTTP making disable deflate as true by default in the header when CF sends an HTTP request, since compression is not handled by CFHTTP client.

Problem for us is that this fix actually broke code we had previously running. Here are the test cases:

Here we have a CF 8.0.1 server running CHF 3:

Execute this code, where we tell CF to pass in two headers:


<cfhttp url= "http://xxxxx.amkor.com/AeXHD/WEbService.asmx/WorkItemRetrieve" result= "res" username= "xxxx" password= "xxxx" >
<cfhttpparam type= "header" name= "Accept-Encoding" value= "*" />
<cfhttpparam type= "Header" name= "TE" value= "deflate;q=0" >
<cfhttpparam type= "formfield" name= "id" value= "596" />
</cfhttp>

<cfdump var= "#res#" >

Wireshark shows CF appending deflate;q=0 to the * which we told CF to pass. This is what was added in CHF 2/3.

IIS 6 gets this and decides to return the data compressed as you can see in the Content-Encoding:

This time, take out the accept-encoding header:


<cfhttp url= "http://xxxxx.amkor.com/AeXHD/WEbService.asmx/WorkItemRetrieve" result= "res" username= "xxxx" password= "xxxx" >
<cfhttpparam type= "Header" name= "TE" value= "deflate;q=0" >
<cfhttpparam type= "formfield" name= "id" value= "596" />
</cfhttp>

<cfdump var= "#res#" >

When you run this, notice that ColdFusion appends all sorts of stuff for the Accept-Encoding header:

The result coming back to CF is again compressed by IIS 6:

Let's try this now with CF 8.0.1 with no CHF (works the same with CHF 1):

Run the code passing in both headers:


<cfhttp url= "http://xxxxx.amkor.com/AeXHD/WEbService.asmx/WorkItemRetrieve" result= "res" username= "xxxx" password= "xxxx" >
<cfhttpparam type= "header" name= "Accept-Encoding" value= "*" />
<cfhttpparam type= "Header" name= "TE" value= "deflate;q=0" >
<cfhttpparam type= "formfield" name= "id" value= "596" />
</cfhttp>

<cfdump var= "#res#" >

Wireshark shows ColdFusion passing in only the accept-encoding value we specified. For the TE trailer, we pass in deflate;q=0 which is supposed to tell IIS not to return deflated (compressed data):

What you see next is what we expect to see. IIS is returning the correct data uncompressed:

Now go ahead and take out the accept-encoding like we did last time:


<cfhttp url= "http://xxxxx.amkor.com/AeXHD/WEbService.asmx/WorkItemRetrieve" result= "res" username= "xxxx" password= "xxxx" >
<cfhttpparam type= "Header" name= "TE" value= "deflate;q=0" >
<cfhttpparam type= "formfield" name= "id" value= "596" />
</cfhttp>

<cfdump var= "#res#" >

Notice how CF just passes along deflate, gzip, x-gzip, etc?

IIS gets this and returns the content deflated, which CFHTTP can't handle and we get our error:

One last time. Now for CF 9:

Run the code passing in both headers:


<cfhttp url= "http://xxxxx.amkor.com/AeXHD/WEbService.asmx/WorkItemRetrieve" result= "res" username= "xxxx" password= "xxxx" >
<cfhttpparam type= "header" name= "Accept-Encoding" value= "*" />
<cfhttpparam type= "Header" name= "TE" value= "deflate;q=0" >
<cfhttpparam type= "formfield" name= "id" value= "596" />
</cfhttp>

<cfdump var= "#res#" >

What we see in wireshark is the same thing we just saw for CF 8.0.1 up to CHF1:

Data comes back uncompressed as we expect:

Remove the accept-encoding header:


<cfhttp url= "http://xxxxx.amkor.com/AeXHD/WEbService.asmx/WorkItemRetrieve" result= "res" username= "xxxx" password= "xxxx" >
<cfhttpparam type= "Header" name= "TE" value= "deflate;q=0" >
<cfhttpparam type= "formfield" name= "id" value= "596" />
</cfhttp>

<cfdump var= "#res#" >

Same values passed as for CF 8.0.1:

And we get compressed data back (and the error again), as expected:

So, what this long exercise shows me is two things:

  1. ColdFusion 8.0.1 up to CHF 1 and ColdFusion 9.0 have identical functionality. A hotfix that was applied in CF 8.0.1 CHF2/3 is not present in CF 9.0
  2. The change that was made in CF 8.0.1 CHF 2/3 (72744) was not properly implemented. We've fixed this in our CF 8.0.1 CHF3 installs by stripping the class files from the CHF JAR that updated CFHTTP and stuff's working again. I can understand Adobe wanting to tell web servers not to compress by default, but setting deflate:q=0 needs to be done in the TE, not in the accept-encoding header. Putting it there causes the data to be returned compressed.

I filed a bug report on this and verified the problem with Adobe (there's a bug number but for some reason it isn't showing up in the bug base). They've stated they'll be releasing a fix for it in CHF4 for CF 8.0.1. Just a reminder that this issue does not affect ColdFusion 9.

Thanks go to my coworker Adam Crump for helping to troubleshoot and diagnose the issue.

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Sami Hoda's Gravatar This has been an ongoing issue. There was a time when Scheduled Tasks would fail if the server (IIS) was compressing back content. I had to fight long and hard for that one, and one other gzip fix.
# Posted By Sami Hoda | 1/15/10 12:51 AM
Nam P's Gravatar We are experiencing the same issue with CF HotFix 4. You mentioned to remove the cfhttp class file from the CHF JAR to fix this problem. Can you be specific what class
file or files we need to remove for the CHF8010004.jar. Thank you.
# Posted By Nam P | 2/16/10 1:05 PM
Brad Wood's Gravatar Why would you use an Accept-Encoding: *? That tells the server that all forms of encoding are valid. Wouldn't you want to use Accept-Encoding: *;q=0 to indicate that "identity" is the only encoding you can accept?

Maybe I'm missing something obvious, but if you override CF's default headers with an accept encoding of * then I would expect the remote server to assume you can handle a deflated response.

Also, you may not realize this but you are broadcasting your username and password in those screenshots. Basic Auth is simply a base64 encoded string comprised of your realm/user:pass.
# Posted By Brad Wood | 4/20/10 11:19 AM
Rob Brooks-Bilson's Gravatar Brad,

The reason for the accept-encoding=* is due to an issue with some versions of IIS. See this blog entry here:

http://www.talkingtree.com/blog/index.cfm/2004/7/2...

What I've observed in the wild is that some servers will respect deflate;q=0 in the http-accept while others do not. Using * worked in my particular use-case in the past while deflate;q=0 didn't.

In any case, it doesn't matter if you pass * or deflate;q=0 in the accept. The issue is that CHF 2/3 appends values that cause IIS to ignore the directive not to return compressed data. This problem doesn't exist in other versions of CF, including 9.0.

Thx for the heads up on the authorization encoding. I hadn't noticed that, and you are indeed correct.
# Posted By Rob Brooks-Bilson | 4/20/10 12:23 PM



Copyright 1995-2010 Rob Brooks-Bilson. All rights reserved.
Aura skin for Raymond Camden's BlogCFC inspired by Joe Rinehart & Steven Erat. This blog is running version 5.9.004.