#1448 Mmaped Buf end of stream issue

dsav Wed 16 Mar 2011

If a Buf is made by File.mmap, its methods which should return null if the end of stream has been reached (e.g. readChar, readLine), spawn java.nio.BufferUnderflowException instead on this condition. Looks like a bug.

sys::Err: java.nio.BufferUnderflowException
java.nio.Buffer.nextGetIndex (Buffer.java:474)
java.nio.DirectByteBuffer.get (DirectByteBuffer.java:208)
fan.sys.MmapBuf$MmapBufInStream.r (MmapBuf.java:272)
fan.sys.Charset$Utf8Decoder.decode (Charset.java:120)
fan.sys.InStream.rChar (InStream.java:78)
fan.sys.InStream.readLine (InStream.java:413)
fan.sys.InStream.readLine (InStream.java:399)
fan.sys.Buf.readLine (Buf.java:333)
pegsamples::MmapTest.main (MmapTest.fan:10)
java.lang.reflect.Method.invoke (Method.java:597)
fan.sys.Method.invoke (Method.java:552)
fan.sys.Method$MethodFunc.callList (Method.java:198)
fan.sys.Method.callList (Method.java:138)
fanx.tools.Fan.callMain (Fan.java:135)
fanx.tools.Fan.executeType (Fan.java:102)
fanx.tools.Fan.execute (Fan.java:38)
fanx.tools.Fan.run (Fan.java:250)
fanx.tools.Fan.main (Fan.java:288)

Code example is below:

Buf b := File.make(`filepath`).mmap    
Str? s := null
try {
  Int i := 0
  while (null != (s = b.readLine)) {
    echo("$i: $s")
    ++i
  }
} finally {
  b.close
}    

It prints the entire file and then crashes with the above exception. However, if File.open is used, it works fine.

brian Wed 16 Mar 2011

Promoted to ticket #1448 and assigned to brian

Yeah it doesn't look like nio ByteBuffer has a method that we really need - to read the bytes available and return actual number read (which is how every other Java input stream works). I will have to dig into it to see the best way to handle that case without introducing any big performance penalty.

If anyone else knows NIO well and wants to help with this bug, please shoot me an email.

brian Thu 2 Jun 2011

Ticket resolved in 1.0.59

Fixed mmap implementation to use java.nio.Buffer.remaining to avoid under flow exceptions.

Login or Signup to reply.