Sunday, April 17, 2011

Concurrent file write in Java on Windows

What happens when you concurrently open two (or more) FileOutputStreams on the same file?

The Java API says this:

Some platforms, in particular, allow a file to be opened for writing by only one FileOutputStream (or other file-writing object) at a time.

I'm guessing Windows isn't such a platform, because I have two threads that read some big file (each one a different one) then write it to the same output file. No exception is thrown, the file is created and seems to contain chunks from both input files.

Side questions:

  • Is this true for Unix, too?
  • And since I want the behaviour to be the same (actually I want one thread to write correctly and the other to be warned of the conflict), how can I determine that the file is already opened for writing?
From stackoverflow
  • I would be wary of letting the OS determine file status for you (since this is OS-dependent). If you've got a shared resource I would restrict access to it using a Re-entrant lock

    Using this lock means one thread can get the resource (file) and write to it. The next thread can check for this lock being held by another thread, and/or block indefinitely until the first thread releases it.

    Windows (I think) would restrict two processes writing to the same file. I don't believe Unix would do the same.

    Andrei Vajna II : But I don't want to block whenever I write, I want to block, if someone is already writing to the file I want to write. Actually, I don't want to block at all. I want to tell the second thread that he can't write and let the client decide on further action.
    Brian Agnew : CHeck that lock API. You'll be able to inspect the lock to determine if someone else has already locked it (i.e. not bock) and then decide what to do
  • If the 2 threads you are talking about are in the same JVM, then you could have a boolean variable somewhere that is accessed by both threads.

  • There's not a reliable, cross-platform way to be passively notified when a file has another writer—i.e., raise an exception if a file is already open for writing. There are a couple of techniques that help you actively check for this, however.

    If multiple processes (which can be a mix of Java and non-Java) might be using the file, use a FileLock. A key to using file locks successfully is to remember that they are only "advisory". The lock is guaranteed to be visible if you check for it, but it won't stop you from doing things to the file if you forget. All processes that access the file should be designed to use the locking protocol.

    If a single Java process is working with the file, you can use the concurrency tools built into Java to do it safely. You need a map visible to all threads that associates each file name with its corresponding lock instance. The answers to a related question can be adapted easily to do this with File objects or canonical paths to files. The lock object could be a FileOutputStream, some wrapper around the stream, or a ReentrantReadWriteLock.

  • Unix allows concurrent writers to the same file.

    You shouldn't be attempting to write to the same file more than once. If you are you have a design flaw.

0 comments:

Post a Comment