Skip to main content

Posts

Showing posts from June, 2015

Writing a download server. Part III: headers: Content-length and Range

We will explore more HTTP request and response headers this time to improve download server implementation: Content-length and Range. The former signals how big the download is, the latter allows downloading files partially or continue after failure from where we started.

Content-length response headerContent-length response header is tremendously helpful for clients that track download progress. If you send expected resource size in advance before even starting to stream bytes, client like web browser can show very accurate progress bar and even estimate total download time by measuring average download speed. Without Content-length client will just keep downloading as long as possible, hoping the stream will end one day. There are however some circumstances when obtaining precise content length is hard. For example maybe you stream resources from some other download server or your resource is compressed on the fly and sent directly to servlet response. In both of these cases the best…

Writing a download server. Part II: headers: Last-Modified, ETag and If-None-Match

Caching on the client side is one of the foundations of World Wide Web. Server should inform client about validity of resources and client should cache them as eagerly as possible. Without caching the web as we see it would be insanely slow. Just hit Ctrl + F5 on any website and compare it with ordinary F5 - the latter is much faster as it uses already cached resources. Caching is also important for downloading. If we already fetched several megabytes of data and they haven't changed, pushing them through network is quite wasteful.

Use ETag and If-None-Match headers HTTP ETag header can be used to avoid repeatable downloads of resources client already has. Along with first response server returns an ETag header, which is typically a hash value of the contents of a file. Client can keep ETag and send it (in If-None-Match request header) when requesting the same resource later. If it wasn't changed in the meantime, server can simply return 304 Not Modified response. Let's st…

Writing a download server. Part I: Always stream, never keep fully in memory

Downloading various files (either text or binary) is a bread and butter of every enterprise application. PDF documents, attachments, media, executables, CSV, very large files, etc. Almost every application, sooner or later, will have to provide some form of download. Downloading is implemented in terms of HTTP, so it's important to fully embrace this protocol and take full advantage of it. Especially in Internet facing applications features like caching or user experience are worth considering. This series of articles provides a list of aspects that you might want to consider when implementing all sorts of download servers. Note that I avoid "best practices" term, these are just guidelines that I find useful but are not necessarily always applicable.

One of the biggest scalability issues is loading whole file into memory before streaming it. Loading full file into byte[] to later return it e.g. from Spring MVC controller is unpredictable and doesn't scale. The amount…

How LongAccumulator and DoubleAccumulator classes work?

Two classes new in Java 8 deserve some attention: LongAccumulator and DoubleAccumulator. They are designed to accumulate (more on what does that mean later) values across threads safely while being extremely fast. A test is worth a thousand words, so here is how it works:

class AccumulatorSpec extends Specification { public static final long A = 1 public static final long B = 2 public static final long C = 3 public static final long D = -4 public static final long INITIAL = 0L def 'should add few numbers'() { given: LongAccumulator accumulator = new LongAccumulator({ long x, long y -> x + y }, INITIAL) when: accumulator.accumulate(A) accumulator.accumulate(B) accumulator.accumulate(C) accumulator.accumulate(D) then: accumulator.get() == INITIAL + A + B + C + D } So the accumulator takes a binary operator and combines initial value with every accumula…