Sunday, August 07, 2011

Java 7: Working with Zip Files

The Zip File System Provider in JDK7 allows you to treat a zip or jar file as a file system, which means that you can perform operations, such as moving, copying, deleting, renaming etc, just as you would with ordinary files. In previous versions of Java, you would have to use ZipEntry objects and read/write using ZipInputStreams and ZipOutputStreams which was quite messy and verbose. The zip file system makes working with zip files much easier!

This post shows you how to create a zip file and extract/list its contents, all using a zip file system.

Constructing a zip file system:
In order to work with a zip file, you have to construct a "zip file system" first. The method below shows how this is done. You need to pass in a properties map with create=true if you want the file system to create the zip file if it doesn't exist.

/**
 * Returns a zip file system
 * @param zipFilename to construct the file system from
 * @param create true if the zip file should be created
 * @return a zip file system
 * @throws IOException
 */
private static FileSystem createZipFileSystem(String zipFilename,
                                              boolean create)
                                              throws IOException {
  // convert the filename to a URI
  final Path path = Paths.get(zipFilename);
  final URI uri = URI.create("jar:file:" + path.toUri().getPath());

  final Map<String, String> env = new HashMap<>();
  if (create) {
    env.put("create", "true");
  }
  return FileSystems.newFileSystem(uri, env);
}
Once you have a zip file system, you can invoke methods of the java.nio.file.FileSystem, java.nio.file.Path and java.nio.file.Files classes to manipulate the zip file.

Unzipping a Zip File:
In order to extract a zip file, you can walk the zip file tree from the root and copy files to the destination directory. Since you are dealing with a zip file system, extracting a directory is exactly the same as copying a directory recursively to another directory. The code below demonstrates this. (Note the use of the try-with-resources statement to close the zip file system automatically when done.)

/**
 * Unzips the specified zip file to the specified destination directory.
 * Replaces any files in the destination, if they already exist.
 * @param zipFilename the name of the zip file to extract
 * @param destFilename the directory to unzip to
 * @throws IOException
 */
public static void unzip(String zipFilename, String destDirname)
    throws IOException{

  final Path destDir = Paths.get(destDirname);
  //if the destination doesn't exist, create it
  if(Files.notExists(destDir)){
    System.out.println(destDir + " does not exist. Creating...");
    Files.createDirectories(destDir);
  }

  try (FileSystem zipFileSystem = createZipFileSystem(zipFilename, false)){
    final Path root = zipFileSystem.getPath("/");

    //walk the zip file tree and copy files to the destination
    Files.walkFileTree(root, new SimpleFileVisitor<Path>(){
      @Override
      public FileVisitResult visitFile(Path file,
          BasicFileAttributes attrs) throws IOException {
        final Path destFile = Paths.get(destDir.toString(),
                                        file.toString());
        System.out.printf("Extracting file %s to %s\n", file, destFile);
        Files.copy(file, destFile, StandardCopyOption.REPLACE_EXISTING);
        return FileVisitResult.CONTINUE;
      }

      @Override
      public FileVisitResult preVisitDirectory(Path dir,
          BasicFileAttributes attrs) throws IOException {
        final Path dirToCreate = Paths.get(destDir.toString(),
                                           dir.toString());
        if(Files.notExists(dirToCreate)){
          System.out.printf("Creating directory %s\n", dirToCreate);
          Files.createDirectory(dirToCreate);
        }
        return FileVisitResult.CONTINUE;
      }
    });
  }
}
Creating a Zip File:
The following method shows how to create a zip file from a list of files. If a directory is passed in, it walks the directory tree and copies files into the zip file system:
/**
 * Creates/updates a zip file.
 * @param zipFilename the name of the zip to create
 * @param filenames list of filename to add to the zip
 * @throws IOException
 */
public static void create(String zipFilename, String... filenames)
    throws IOException {

  try (FileSystem zipFileSystem = createZipFileSystem(zipFilename, true)) {
    final Path root = zipFileSystem.getPath("/");

    //iterate over the files we need to add
    for (String filename : filenames) {
      final Path src = Paths.get(filename);

      //add a file to the zip file system
      if(!Files.isDirectory(src)){
        final Path dest = zipFileSystem.getPath(root.toString(),
                                                src.toString());
        final Path parent = dest.getParent();
        if(Files.notExists(parent)){
          System.out.printf("Creating directory %s\n", parent);
          Files.createDirectories(parent);
        }
        Files.copy(src, dest, StandardCopyOption.REPLACE_EXISTING);
      }
      else{
        //for directories, walk the file tree
        Files.walkFileTree(src, new SimpleFileVisitor<Path>(){
          @Override
          public FileVisitResult visitFile(Path file,
              BasicFileAttributes attrs) throws IOException {
            final Path dest = zipFileSystem.getPath(root.toString(),
                                                    file.toString());
            Files.copy(file, dest, StandardCopyOption.REPLACE_EXISTING);
            return FileVisitResult.CONTINUE;
          }

          @Override
          public FileVisitResult preVisitDirectory(Path dir,
              BasicFileAttributes attrs) throws IOException {
            final Path dirToCreate = zipFileSystem.getPath(root.toString(),
                                                           dir.toString());
            if(Files.notExists(dirToCreate)){
              System.out.printf("Creating directory %s\n", dirToCreate);
              Files.createDirectories(dirToCreate);
            }
            return FileVisitResult.CONTINUE;
          }
        });
      }
    }
  }
}
Listing the contents of a zip file:
This is the same as extracting a zip file except that instead of copying the files visited, we simply print them out:
/**
 * List the contents of the specified zip file
 * @param filename
 * @throws IOException
 * @throws URISyntaxException
 */
public static void list(String zipFilename) throws IOException{

  System.out.printf("Listing Archive:  %s\n",zipFilename);

  //create the file system
  try (FileSystem zipFileSystem = createZipFileSystem(zipFilename, false)) {

    final Path root = zipFileSystem.getPath("/");

    //walk the file tree and print out the directory and filenames
    Files.walkFileTree(root, new SimpleFileVisitor<Path>(){
      @Override
      public FileVisitResult visitFile(Path file,
          BasicFileAttributes attrs) throws IOException {
        print(file);
        return FileVisitResult.CONTINUE;
      }

      @Override
      public FileVisitResult preVisitDirectory(Path dir,
          BasicFileAttributes attrs) throws IOException {
        print(dir);
        return FileVisitResult.CONTINUE;
      }

      /**
       * prints out details about the specified path
       * such as size and modification time
       * @param file
       * @throws IOException
       */
      private void print(Path file) throws IOException{
        final DateFormat df = new SimpleDateFormat("MM/dd/yyyy-HH:mm:ss");
        final String modTime= df.format(new Date(
                             Files.getLastModifiedTime(file).toMillis()));
        System.out.printf("%d  %s  %s\n",
                          Files.size(file),
                          modTime,
                          file);
      }
    });
  }
}
Further Reading:
Zip File System Provider

9 comments:

  1. Warning !!

    This method describe above consumes much more memory thant using classical java.util.zip (ZipEntry and ZipOutputStream).

    When building 10 zip(s) of 12Mo each, the method describe above generate 135Mo of JVM Heap where the "old ways" only generate 18Mo.

    API is nice too bad memory "skyrockets"...

    ReplyDelete
  2. Hi i have used your code to append files in to an zip file. But it is not adding the files to zip.Could you please throw some light on it.

    ReplyDelete
  3. Have you tried your create-Example? It does not work and it cannot work. Why? Because - according to article by Oracle's Gutupalli on "Zip File System Provider Implementation details" - ZipFileSystem is a READ ONLY Filesystem.
    I was heavily searching for bugs in the code I copied from you because I could not populate the created ZIP - it always throws a ReadOnlyFilesystemException. Finally I've found the above mentioned article.
    So, I must find another way for creating and populating zip files - maybe I must fall back to older ZIP implementation?

    ReplyDelete
  4. I have run my code again and it creates a zip file without any problems. I think your issue might be related to your environment. Are you trying to create a zip on a different filesystem? Is it on a mapped drive? If so, have you tried creating it on your local disk e.g in /tmp (or C:\temp)? Have you tried the example from the javadocs: http://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html?

    Gutupalli's article was written in 2008 and is out-of-date. Java 7 was released in 2011. The ZipFileSystem is certainly not read-only; there wouldn't be any point in making it so.

    ReplyDelete
  5. Hi Fahd,
    it's getting curious, at least for me. When I tried your code with a Junit test class, I run it on my local disk (unter Windows 7), no special environment. What happened was: the ZIP file was created as an empty file. But the first try to write to it (a directory entry) ran into a ReadOnlyFilesystemException (in your code at Files.createDirectories(parent).

    Well the I began to google around, and also found the example from javadocs. Next bad surprise: the example calls copyTo on Path-Interface - a method which does not exist in my JDK version (1.7 update 5)!

    Another page on oracle mentioned the NIO.2 demo with an example written for ZipFileSystem. I downloaded, tried - and got the some error populating the zip.

    So, to get results, I fell back to the old java.util.zip-classes for creating and populating zips.

    ReplyDelete
  6. Thank you very much for your manual. I would like to complete the code with one small thing. There is a problem, when you have a space in your file path. Creating URI fails and it's a big problem to force the Java to accept the path. Solution is to replace
    URI.create("jar:file:" + path.toUri().getPath());
    by
    URI.create("jar:file:" + path.toUri().getPath().replaceAll(" ", "%2520"));
    It's workaround for double-escaping the space, so URI.create method accept it and then FileSystems.newFileSystem accept the URI.

    ReplyDelete
  7. Great article thanks. In case anyone runs into same problem, if the file name has a space in it path.toUri().getPath().replace("\\s", "%20") doesnt work because of the bug in URI class. (Bug ID 7156873).
    Either put two space symbols in it "%25%20" or use another constructor that doesnt take uri as argument FileSystem fs = FileSystems.newFileSystem(zipfile, null).
    Both work.

    ReplyDelete
  8. nice :)
    --thank you
    replace(" ", "%20") works now

    ReplyDelete
  9. A better way is to create the URI using URI.create("jar:" + path.toUri().toString());
    That properly adds the platform dependent "file:///" prefix, and also escapes spaces, percentage signs, etc. correctly.

    ReplyDelete