File upload with progress bar, and paged file downloads.
Adapting the event-based code from Chapter 9 of the excellent book Google Web Toolkit Applications, I coded up a file upload application with a progress bar, which is updated using server pseudo-push events.
The application also provides a download servlet, and layout using GWT UiBinder.
Full source for the webapp is available at the bottom of the page. Not all of the code is listed below, but some of the most important classes are explained here.
The UploadProgressService interface defines the server side functions available, the most important being getEvents(). This function will be called recursively on the client side to simulate a main loop.
UploadProgressService.java
package org.adrianwalker.gwt.uploadprogress.client;
import java.util.List;
import com.google.gwt.user.client.rpc.RemoteService;
import com.google.gwt.user.client.rpc.RemoteServiceRelativePath;
import org.adrianwalker.gwt.uploadprogress.common.dto.FileDto;
import org.adrianwalker.gwt.uploadprogress.common.event.Event;
@RemoteServiceRelativePath("uploadprogress")
public interface UploadProgressService extends RemoteService {
void initialise();
int countFiles();
List<FileDto> readFiles(int page, int pageSize);
List<Event> getEvents();
}
The only event in the application is UploadProgressChangeEvent. It is concerned with reporting changes in file upload percentage.
UploadProgressChangeEvent.java
package org.adrianwalker.gwt.uploadprogress.common.event;
import java.io.Serializable;
public final class UploadProgressChangeEvent implements Event, Serializable {
private String filename;
private Integer percentage;
public UploadProgressChangeEvent() {
}
public String getFilename() {
return filename;
}
public void setFilename(final String filename) {
this.filename = filename;
}
public Integer getPercentage() {
return percentage;
}
public void setPercentage(final Integer percentage) {
this.percentage = percentage;
}
@Override
public String toString() {
return filename + " - " + percentage;
}
}
The UploadProgressServlet implements the methods defined by UploadProgressService. The getEvents() method returns the events stored in the servlets session scope. If the event list is empty, the thread waits for 30 seconds. If the list contained events, they are retuned to the client, and then the event list is cleared.
UploadProgressServlet.java
package org.adrianwalker.gwt.uploadprogress.server;
import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpSession;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;
import java.io.File;
import java.io.FileFilter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.Date;
import java.util.List;
import java.util.Properties;
import org.adrianwalker.gwt.uploadprogress.client.UploadProgressService;
import org.adrianwalker.gwt.uploadprogress.common.dto.FileDto;
import org.adrianwalker.gwt.uploadprogress.common.event.Event;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class UploadProgressServlet extends RemoteServiceServlet implements UploadProgressService {
private static final int EVENT_WAIT = 30 * 1000;
private static final String PROPERTIES_FILE = "WEB-INF/classes/uploadprogress.properties";
private static final Logger LOGGER = LoggerFactory.getLogger(UploadProgressServlet.class);
private String uploadDirectory;
@Override
public void init() throws ServletException {
Properties properties = new Properties();
try {
properties.load(getServletContext().getResourceAsStream(PROPERTIES_FILE));
} catch (IOException ioe) {
throw new ServletException(ioe);
}
uploadDirectory = properties.getProperty("upload.directory", "target");
}
@Override
public void initialise() {
getThreadLocalRequest().getSession(true);
}
@Override
public List<FileDto> readFiles(final int page, final int pageSize) {
File[] listFiles = readFiles(this.uploadDirectory);
sortFiles(listFiles);
int firstFile = pageSize * (page - 1);
int lastFile = firstFile + pageSize;
int fileCount = listFiles.length;
if (fileCount < lastFile) {
lastFile = fileCount;
}
if (firstFile < fileCount) {
List<FileDto> files = new ArrayList<FileDto>();
for (int i = firstFile; i < lastFile; i++) {
File file = listFiles[i];
FileDto fileDto = new FileDto();
fileDto.setFilename(file.getName());
fileDto.setDateUploaded(new Date(file.lastModified()));
files.add(fileDto);
}
return files;
} else {
return Collections.EMPTY_LIST;
}
}
@Override
public List<Event> getEvents() {
HttpSession session = getThreadLocalRequest().getSession();
UploadProgress uploadProgress = UploadProgress.getUploadProgress(session);
List<Event> events = null;
if (null != uploadProgress) {
if (uploadProgress.isEmpty()) {
try {
synchronized (uploadProgress) {
LOGGER.debug("waiting...");
uploadProgress.wait(EVENT_WAIT);
}
} catch (final InterruptedException ie) {
LOGGER.debug("interrupted...");
}
}
synchronized (uploadProgress) {
events = uploadProgress.getEvents();
uploadProgress.clear();
}
}
return events;
}
@Override
public int countFiles() {
return readFiles(this.uploadDirectory).length;
}
private File[] readFiles(final String directory) {
File uploadDirectory = new File(directory);
return uploadDirectory.listFiles(new FileFilter() {
@Override
public boolean accept(final File file) {
return null == file ? false : file.isFile();
}
});
}
private void sortFiles(final File[] listFiles) {
Arrays.sort(listFiles, new Comparator<File>() {
@Override
public int compare(final File f1, final File f2) {
return Long.valueOf(f2.lastModified()).compareTo(f1.lastModified());
}
});
}
}
UploadProgress stores the list of events, and has a factory method for creating/retrieving an instance of its self in the session.
UploadProgress.java
package org.adrianwalker.gwt.uploadprogress.server;
import java.util.ArrayList;
import java.util.List;
import javax.servlet.http.HttpSession;
import org.adrianwalker.gwt.uploadprogress.common.event.Event;
public final class UploadProgress {
private static final String SESSION_KEY = "uploadProgress";
private List<Event> events = new ArrayList<Event>();
private UploadProgress() {
}
public List<Event> getEvents() {
return events;
}
public void add(final Event event) {
events.add(event);
}
public void clear() {
events = new ArrayList<Event>();
}
public boolean isEmpty() {
return events.isEmpty();
}
public static UploadProgress getUploadProgress(final HttpSession session) {
Object attribute = session.getAttribute(SESSION_KEY);
if (null == attribute) {
attribute = new UploadProgress();
session.setAttribute(SESSION_KEY, attribute);
}
return null == attribute ? null : (UploadProgress) attribute;
}
}
The UploadServlet uses Apache Commons FileUpload library to parse the multipart post request and stream the file to disk. An UploadProgressListener is added to a custom input stream, UploadProgressInputStream, which together report the status of the upload back to the client.
UploadServlet.java
package org.adrianwalker.gwt.uploadprogress.server;
import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;
import java.io.File;
import java.io.FileOutputStream;
import java.util.Properties;
import javax.servlet.http.HttpServlet;
import org.apache.commons.fileupload.FileItemFactory;
import org.apache.commons.fileupload.FileItemIterator;
import org.apache.commons.fileupload.FileItemStream;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.fileupload.util.Streams;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class UploadServlet extends HttpServlet {
private static final String PROPERTIES_FILE = "WEB-INF/classes/uploadprogress.properties";
private static final Logger LOGGER = LoggerFactory.getLogger(UploadServlet.class);
private static final String FILE_SEPERATOR = System.getProperty("file.separator");
private String uploadDirectory;
@Override
public void init() throws ServletException {
Properties properties = new Properties();
try {
properties.load(getServletContext().getResourceAsStream(PROPERTIES_FILE));
} catch (IOException ioe) {
throw new ServletException(ioe);
}
uploadDirectory = properties.getProperty("upload.directory", "target");
}
@Override
protected void doPost(final HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException {
try {
uploadFile(request);
} catch (FileUploadException fue) {
throw new ServletException(fue);
}
}
private void uploadFile(final HttpServletRequest request) throws FileUploadException, IOException {
if (!ServletFileUpload.isMultipartContent(request)) {
throw new FileUploadException("error multipart request not found");
}
FileItemFactory fileItemFactory = new DiskFileItemFactory();
ServletFileUpload servletFileUpload = new ServletFileUpload(fileItemFactory);
FileItemIterator fileItemIterator = servletFileUpload.getItemIterator(request);
HttpSession session = request.getSession();
UploadProgress uploadProgress = UploadProgress.getUploadProgress(session);
while (fileItemIterator.hasNext()) {
FileItemStream fileItemStream = fileItemIterator.next();
String filePath = fileItemStream.getName();
String fileName = filePath.substring(filePath.lastIndexOf(FILE_SEPERATOR) + 1);
UploadProgressListener uploadProgressListener = new UploadProgressListener(fileName, uploadProgress);
UploadProgressInputStream inputStream = new UploadProgressInputStream(fileItemStream.openStream(), request.getContentLength());
inputStream.addListener(uploadProgressListener);
File file = new File(uploadDirectory, fileName);
Streams.copy(inputStream, new FileOutputStream(file), true);
LOGGER.info(String.format("uploaded file %s", file.getAbsolutePath()));
}
}
}
UploadProgressInputStream is a FilterInputStream which reports the number of bytes read to attached listeners.
UploadProgressInputStream.java
package org.adrianwalker.gwt.uploadprogress.server;
import java.io.FilterInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.fileupload.ProgressListener;
public final class UploadProgressInputStream extends FilterInputStream {
private List<ProgressListener> listeners;
private long bytesRead = 0;
private long totalBytes = 0;
public UploadProgressInputStream(final InputStream in, final long totalBytes) {
super(in);
this.totalBytes = totalBytes;
listeners = new ArrayList<ProgressListener>();
}
public void addListener(final ProgressListener listener) {
listeners.add(listener);
}
@Override
public int read() throws IOException {
int b = super.read();
this.bytesRead++;
updateListeners(bytesRead, totalBytes);
return b;
}
@Override
public int read(final byte b[]) throws IOException {
return read(b, 0, b.length);
}
@Override
public int read(final byte b[], final int off, final int len) throws IOException {
int bytesRead = in.read(b, off, len);
this.bytesRead = this.bytesRead + bytesRead;
updateListeners(this.bytesRead, totalBytes);
return bytesRead;
}
@Override
public void close() throws IOException {
super.close();
updateListeners(totalBytes, totalBytes);
}
private void updateListeners(final long bytesRead, final long totalBytes) {
for (ProgressListener listener : listeners) {
listener.update(bytesRead, totalBytes, listeners.size());
}
}
}
An UploadProgressListener attached to the input stream, adds UploadProgressChangeEvent events to the event list and calls notifyAll() to reply to the waiting client.
UploadProgressListener.java
package org.adrianwalker.gwt.uploadprogress.server;
import org.adrianwalker.gwt.uploadprogress.common.event.UploadProgressChangeEvent;
import org.apache.commons.fileupload.ProgressListener;
public final class UploadProgressListener implements ProgressListener {
private static final double COMPLETE_PERECENTAGE = 100d;
private int percentage = -1;
private String fileName;
private UploadProgress uploadProgress;
public UploadProgressListener(final String fileName, final UploadProgress uploadProgress) {
this.fileName = fileName;
this.uploadProgress = uploadProgress;
}
@Override
public void update(final long bytesRead, final long totalBytes, final int items) {
int percentage = (int) Math.floor(((double) bytesRead / (double) totalBytes) * COMPLETE_PERECENTAGE);
if (this.percentage == percentage) {
return;
}
this.percentage = percentage;
UploadProgressChangeEvent event = new UploadProgressChangeEvent();
event.setFilename(this.fileName);
event.setPercentage(percentage);
synchronized (this.uploadProgress) {
this.uploadProgress.add(event);
this.uploadProgress.notifyAll();
}
}
}
The client side ProgressController is responsible for calling the getEvents() method on the service servlet, and calling it again after a successful response; creating an event loop.
ProgressController.java
package org.adrianwalker.gwt.uploadprogress.client.controller;
import com.google.gwt.core.client.GWT;
import java.util.List;
import com.google.gwt.user.client.rpc.AsyncCallback;
import org.adrianwalker.gwt.uploadprogress.client.state.UploadProgressState;
import org.adrianwalker.gwt.uploadprogress.common.dto.FileDto;
import org.adrianwalker.gwt.uploadprogress.common.event.Event;
import org.adrianwalker.gwt.uploadprogress.common.event.UploadProgressChangeEvent;
public final class ProgressController extends AbstractController {
public static final ProgressController INSTANCE = new ProgressController();
private ProgressController() {
}
public void findFiles(final int page, final int pageSize) {
SERVICE.readFiles(page, pageSize, new AsyncCallback<List<FileDto>>() {
@Override
public void onFailure(final Throwable t) {
GWT.log("error find files", t);
}
@Override
public void onSuccess(final List<FileDto> files) {
UploadProgressState.INSTANCE.setFiles(files);
}
});
}
private void getEvents() {
SERVICE.getEvents(new AsyncCallback<List<Event>>() {
@Override
public void onFailure(final Throwable t) {
GWT.log("error get events", t);
}
@Override
public void onSuccess(final List<Event> events) {
for (Event event : events) {
handleEvent(event);
}
SERVICE.getEvents(this);
}
private void handleEvent(final Event event) {
if (event instanceof UploadProgressChangeEvent) {
UploadProgressChangeEvent uploadPercentChangeEvent = (UploadProgressChangeEvent) event;
String filename = uploadPercentChangeEvent.getFilename();
Integer percentage = uploadPercentChangeEvent.getPercentage();
UploadProgressState.INSTANCE.setUploadProgress(filename, percentage);
}
}
});
}
public void initialise() {
SERVICE.initialise(new AsyncCallback<Void>() {
@Override
public void onFailure(final Throwable t) {
GWT.log("error initialise", t);
}
@Override
public void onSuccess(final Void result) {
getEvents();
}
});
}
public void countFiles() {
SERVICE.countFiles(new AsyncCallback<Integer>() {
@Override
public void onFailure(final Throwable t) {
GWT.log("error count files", t);
}
@Override
public void onSuccess(final Integer result) {
int pageSize = UploadProgressState.INSTANCE.getPageSize();
int pages = (int) Math.ceil((double) result / (double) pageSize);
UploadProgressState.INSTANCE.setPages(pages);
}
});
}
}
Changes to the files upload percentage are stored in UploadProgressState which can have listerners attached to be notified of updates to its fields.
UploadProgressState.java
package org.adrianwalker.gwt.uploadprogress.client.state;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.adrianwalker.gwt.uploadprogress.common.dto.FileDto;
public final class UploadProgressState extends PageableState {
public static final UploadProgressState INSTANCE = new UploadProgressState();
private Map<String, Integer> uploadProgress;
private List<FileDto> files;
private UploadProgressState() {
uploadProgress = new HashMap<String, Integer>();
}
public List<FileDto> getFiles() {
return files;
}
public void setFiles(final List<FileDto> files) {
List<FileDto> old = this.files;
this.files = files;
firePropertyChange("files", old, files);
}
public Integer getUploadProgress(final String filename) {
return uploadProgress.get(filename);
}
public void setUploadProgress(final String filename, final Integer percentage) {
Integer old = this.uploadProgress.get(filename);
uploadProgress.put(filename, percentage);
firePropertyChange("uploadProgress", old, uploadProgress);
}
}
Finally, UploadProgress listens for changes to UploadProgressState and updates the UploadPanel with percentage changes for the file upload, which in turn updates the panels ProgressBar.
UploadProgress.java
package org.adrianwalker.gwt.uploadprogress.client.view;
import com.google.gwt.user.client.Timer;
import java.beans.PropertyChangeEvent;
import java.beans.PropertyChangeListener;
import com.google.gwt.user.client.ui.Composite;
import com.google.gwt.user.client.ui.HorizontalPanel;
import com.google.gwt.user.client.ui.Label;
import com.google.gwt.user.client.ui.Panel;
import com.google.gwt.user.client.ui.VerticalPanel;
import java.util.HashMap;
import java.util.Map;
import org.adrianwalker.gwt.uploadprogress.client.state.UploadProgressState;
public final class UploadProgress extends Composite {
private Panel panel;
private Map<String, UploadPanel> uploads;
public UploadProgress() {
panel = new VerticalPanel();
panel.setStyleName("UploadProgress");
uploads = new HashMap<String, UploadPanel>();
this.initWidget(panel);
UploadProgressState.INSTANCE.addPropertyChangeListener("uploadProgress", new UploadProgressListener());
}
private final class UploadProgressListener implements PropertyChangeListener {
private static final int COMPLETE_PERECENTAGE = 100;
private static final int REMOVE_DELAY = 3000;
@Override
public void propertyChange(final PropertyChangeEvent event) {
Map<String, Integer> uploadPercentage = (Map<String, Integer>) event.getNewValue();
for (Map.Entry<String, Integer> entry : uploadPercentage.entrySet()) {
String file = entry.getKey();
Integer percentage = entry.getValue();
final UploadPanel uploadPanel;
if (!uploads.containsKey(file)) {
uploadPanel = new UploadPanel(file);
uploads.put(file, uploadPanel);
panel.add(uploadPanel);
} else {
uploadPanel = uploads.get(file);
}
uploadPanel.update(percentage);
if (percentage == COMPLETE_PERECENTAGE) {
Timer timer = new Timer() {
@Override
public void run() {
panel.remove(uploadPanel);
}
};
timer.schedule(REMOVE_DELAY);
}
}
}
}
private static final class UploadPanel extends HorizontalPanel {
private ProgressBar bar;
private Label label;
public UploadPanel(final String file) {
setStyleName("UploadPanel");
bar = new ProgressBar();
label = new Label(file);
add(bar);
add(label);
}
public void update(final int percentage) {
bar.update(percentage);
}
}
}
Source Code
Usage
Run the project with 'mvn clean install tomcat:run-war' and point your brower at http://localhost:8080/uploadprogress.
Update - 29/7/2012
If you're seeing the following exception during compilation try this updated
pom.xml
:
[gwt:compile]
auto discovered modules [org.adrianwalker.gwt.uploadprogress.UploadProgress]
You're project declares dependency on gwt-user 2.1.0. This plugin is designed for at least gwt version 2.5.0-rc1
You're project declares dependency on gwt-user 2.1.0. This plugin is designed for at least gwt version 2.5.0-rc1
Compiling module org.adrianwalker.gwt.uploadprogress.UploadProgress
Validating units:
Ignored 50 units with compilation errors in first pass.
Compile with -strict or with -logLevel set to TRACE or DEBUG to see all errors.
[ERROR] Unexpected internal compiler error
java.lang.NullPointerException
at com.google.gwt.dev.javac.CompilationProblemReporter.reportErrors(CompilationProblemReporter.java:200)
at com.google.gwt.dev.jjs.impl.UnifyAst.assimilateUnit(UnifyAst.java:666)
at com.google.gwt.dev.jjs.impl.UnifyAst.searchForTypeBySource(UnifyAst.java:985)
at com.google.gwt.dev.jjs.impl.UnifyAst.addRootTypes(UnifyAst.java:530)
at com.google.gwt.dev.jjs.JavaToJavaScriptCompiler.precompile(JavaToJavaScriptCompiler.java:601)
at com.google.gwt.dev.jjs.JavaScriptCompiler.precompile(JavaScriptCompiler.java:33)
at com.google.gwt.dev.Precompile.precompile(Precompile.java:278)
at com.google.gwt.dev.Precompile.precompile(Precompile.java:229)
at com.google.gwt.dev.Precompile.precompile(Precompile.java:141)
at com.google.gwt.dev.Compiler.run(Compiler.java:232)
at com.google.gwt.dev.Compiler.run(Compiler.java:198)
at com.google.gwt.dev.Compiler$1.run(Compiler.java:170)
at com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:88)
at com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:82)
at com.google.gwt.dev.Compiler.main(Compiler.java:177)
