Package dev.vortex.io

Interface NativeReadable

All Superinterfaces:
AutoCloseable, Closeable
All Known Implementing Classes:
PooledReadable
public interface NativeReadable extends Closeable
A random-access byte source supplied by the caller and read from native code.

Implementations bridge external I/O abstractions — for example Iceberg's FileIO input streams — into the Vortex native reader. The native side invokes readFully(long, ByteBuffer) from its own worker threads, potentially many calls concurrently, so implementations must be safe for concurrent positional reads (typically by opening one underlying stream per concurrent call, or delegating to a positional-read API).

Lifecycle: the creator owns this object. Native code never calls Closeable.close(); close it only after every data source built on top of it has been closed and every derived scan, partition, and Arrow reader has finished reading.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    DEFAULT_COALESCE_DISTANCE
    Default maximum gap between coalesced reads, in bytes (1 MiB), matching the native object-storage policy.
    static final int
    DEFAULT_COALESCE_MAX_SIZE
    Default maximum span of a coalesced read, in bytes (16 MiB).

  • Method Summary

    Modifier and Type
    Method
    Description
    default int
    coalesceDistance()
    Maximum gap in bytes between reads that native code may merge.
    default int
    coalesceMaxSize()
    Maximum payload span in bytes of a merged read, excluding leading alignment padding.
    long
    length()
    Total length of the source in bytes.
    String
    name()
    Unique name of this source, typically its original location (URI or file path).
    void
    readFully(long position, ByteBuffer buffer)
    Read exactly buffer.remaining() bytes starting at absolute position into buffer.

    Methods inherited from interface java.io.Closeable

    close

  • Field Details

    • DEFAULT_COALESCE_DISTANCE

      static final int DEFAULT_COALESCE_DISTANCE
      Default maximum gap between coalesced reads, in bytes (1 MiB), matching the native object-storage policy.
      See Also:

    • DEFAULT_COALESCE_MAX_SIZE

      static final int DEFAULT_COALESCE_MAX_SIZE
      Default maximum span of a coalesced read, in bytes (16 MiB).
      See Also:

  • Method Details

    • coalesceDistance

      default int coalesceDistance()
      Maximum gap in bytes between reads that native code may merge. Must be non-negative; zero merges only adjacent or overlapping reads. Captured once when the source is registered.

      Larger gaps reduce storage requests but may re-read segments already held in the file's segment cache.

    • coalesceMaxSize

      default int coalesceMaxSize()
      Maximum payload span in bytes of a merged read, excluding leading alignment padding. Must be positive. Individual reads may exceed this size. Captured once when the source is registered. Native code caps values near Integer.MAX_VALUE to reserve alignment headroom within the Java ByteBuffer limit.

    • name

      String name()
      Unique name of this source, typically its original location (URI or file path). Used to key per-session caches and in native error messages, so it must be stable and unique across sources; it must not contain glob characters (*?[).

    • length

      long length()
      Total length of the source in bytes. Must be cheap; called once when the source is registered.

    • readFully

      void readFully(long position, ByteBuffer buffer) throws IOException
      Read exactly buffer.remaining() bytes starting at absolute position into buffer.

      Unlike InputStream.read(), short reads are not permitted: implementations must either fill the buffer completely (buffer.remaining() == 0 on return) or throw.

      buffer is a direct ByteBuffer wrapping native memory owned by the caller, so bytes written to it land in the native reader without an extra copy. It is valid only for the duration of this call: implementations must not retain a reference to it after returning. Implementations backed by byte[] APIs can bulk-put into it.

      Throws:
      EOFException - if the range extends past the end of the source
      IOException - if the underlying storage fails