Share this page 

HttpUrlConnection with GZIP encoding Tag(s): Networking


By default, when you make a request to URL, the response is not compressed. 
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;

public class HttpConnect {
    public static void main(String[] args) throws Exception {
      URL url = new URL("https://www.rgagnon.com/howto.html");
      HttpURLConnection con = (HttpURLConnection) url.openConnection();
      // con.setRequestProperty("Accept-Encoding", "gzip");
      System.out.println("Length : " + con.getContentLength());
      Reader reader = new InputStreamReader(con.getInputStream());
      while (true) {
        int ch = reader.read();
        if (ch==-1) {
          break;
        }
        System.out.print((char)ch);
      }
    }
}
From the response, we see that the length is 21740 bytes. 
Length : 21740
<!DOCTYPE HTML>
<HTML>
    <head>
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <LINK REL="SHORTCUT ICON" HREF="http://www.rgagnon.com/favicon.ico">
      <META NAME="description"
            Content="Real's JAVA JAVASCRIPT WSH and PowerBuilder How-to pages with useful code snippets">
      <META NAME="keywords"
      Content="java,javascript,wsh,vbscript,how-to,powerbuilder">
...
By setting the Request Header "Accept" to "gzip", we are telling to the server that we want the response to be compressed if possible with the GZIP compression scheme. If it's not supported by the server then the response will be sent as plain text. 
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;

public class HttpConnect {
    public static void main(String[] args) throws Exception {
      URL url = new URL("https://www.rgagnon.com/howto.html");
      HttpURLConnection con = (HttpURLConnection) url.openConnection();
      con.setRequestProperty("Accept-Encoding", "gzip");
      System.out.println("Length : " + con.getContentLength());
      Reader reader = new InputStreamReader(con.getInputStream());
      while (true) {
        int ch = reader.read();
        if (ch==-1) {
          break;
        }
        System.out.print((char)ch);
      }
    }
}
Now the response length is 4865 but the content is not readable because we need to uncompress it! 
Length : 4865
j¼Eðßôš[è^úðGR¾vø
...
We add a check to determine if the response is compressed or not to use the appropriate InputStream because the web server is by no means obliged to use any compression method - this depends on the internal settings of the web server. 
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.zip.GZIPInputStream;

public class HttpConnect {
   public static void main(String[] args) throws Exception {
      URL url = new URL("http://www.rgagnon.com/howto.html");
      HttpURLConnection con = (HttpURLConnection) url.openConnection();
      con.setRequestProperty("Accept-Encoding", "gzip");
      System.out.println("Length : " + con.getContentLength());

      Reader reader = null;
      if ("gzip".equals(con.getContentEncoding())) {
         reader = new InputStreamReader(new GZIPInputStream(con.getInputStream()));
      }
      else {
         reader = new InputStreamReader(con.getInputStream());
      }

      while (true) {
         int ch = reader.read();
         if (ch==-1) {
            break;
         }
         System.out.print((char)ch);
      }
   }
}
And the result is 
Length : 4865
<!DOCTYPE HTML>
<HTML>
    <head>
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <LINK REL="SHORTCUT ICON" HREF="http://www.rgagnon.com/favicon.ico">
      <META NAME="description"
            Content="Real's JAVA JAVASCRIPT WSH and PowerBuilder How-to pages with useful code snippets">
      <META NAME="keywords"
      Content="java,javascript,wsh,vbscript,how-to,powerbuilder">
...
mail_outline
Send comment, question or suggestion to howto@rgagnon.com