Class | RangesIO |
In: |
lib/ole/ranges_io.rb
|
Parent: | Object |
RangesIO is a basic class for wrapping another IO object allowing you to arbitrarily reorder slices of the input file by providing a list of ranges. Intended as an initial measure to curb inefficiencies in the Dirent#data method just reading all of a file‘s data in one hit, with no method to stream it.
This class will encapuslate the ranges (corresponding to big or small blocks) of any ole file and thus allow reading/writing directly to the source bytes, in a streamed fashion (so just getting 16 bytes doesn‘t read the whole thing).
In the simplest case it can be used with a single range to provide a limited io to a section of a file.
On further reflection, this class is something of a joining/optimization of two separate IO classes. a SubfileIO, for providing access to a range within a File as a separate IO object, and a ConcatIO, allowing the presentation of a bunch of io objects as a single unified whole.
I will need such a ConcatIO if I‘m to provide Mime#to_io, a method that will convert a whole mime message into an IO stream, that can be read from. It will just be the concatenation of a series of IO objects, corresponding to headers and boundaries, as StringIO‘s, and SubfileIO objects, coming from the original message proper, or RangesIO as provided by the Attachment#data, that will then get wrapped by Mime in a Base64IO or similar, to get encoded on-the- fly. Thus the attachment, in its plain or encoded form, and the message as a whole never exists as a single string in memory, as it does now. This is a fair bit of work to achieve, but generally useful I believe.
This class isn‘t ole specific, maybe move it to my general ruby stream project.
pos | -> | tell |
io | [R] | |
mode | [R] | |
pos | [R] | |
ranges | [R] | |
size | [R] |
io: | the parent io object that we are wrapping. |
mode: | the mode to use |
params: | hash of params. |
NOTE: the ranges can overlap.
# File lib/ole/ranges_io.rb, line 54 54: def initialize io, mode='r', params={} 55: mode, params = 'r', mode if Hash === mode 56: ranges = params[:ranges] 57: @params = {:close_parent => false}.merge params 58: @mode = IO::Mode.new mode 59: @io = io 60: # initial position in the file 61: @pos = 0 62: self.ranges = ranges || [[0, io.size]] 63: # handle some mode flags 64: truncate 0 if @mode.truncate? 65: seek size if @mode.append? 66: end
add block form. TODO add test for this
# File lib/ole/ranges_io.rb, line 69 69: def self.open(*args, &block) 70: ranges_io = new(*args) 71: if block_given? 72: begin; yield ranges_io 73: ensure; ranges_io.close 74: end 75: else 76: ranges_io 77: end 78: end
# File lib/ole/ranges_io.rb, line 147 147: def close 148: @io.close if @params[:close_parent] 149: end
# File lib/ole/ranges_io.rb, line 243 243: def inspect 244: "#<#{self.class} io=#{io.inspect}, size=#{@size}, pos=#{@pos}>" 245: end
# File lib/ole/ranges_io.rb, line 110 110: def pos= pos, whence=IO::SEEK_SET 111: case whence 112: when IO::SEEK_SET 113: when IO::SEEK_CUR 114: pos += @pos 115: when IO::SEEK_END 116: pos = @size + pos 117: else raise Errno::EINVAL 118: end 119: raise Errno::EINVAL unless (0..@size) === pos 120: @pos = pos 121: 122: # do a binary search throuh @offsets to find the active range. 123: a, c, b = 0, 0, @offsets.length 124: while a < b 125: c = (a + b) / 2 126: pivot = @offsets[c] 127: if pos == pivot 128: @active = c 129: return 130: elsif pos < pivot 131: b = c 132: else 133: a = c + 1 134: end 135: end 136: 137: @active = a - 1 138: end
# File lib/ole/ranges_io.rb, line 80 80: def ranges= ranges 81: # convert ranges to arrays. check for negative ranges? 82: ranges = ranges.map { |r| Range === r ? [r.begin, r.end - r.begin] : r } 83: # combine ranges 84: if @params[:combine] == false 85: # might be useful for debugging... 86: @ranges = ranges 87: else 88: @ranges = [] 89: next_pos = nil 90: ranges.each do |pos, len| 91: if next_pos == pos 92: @ranges.last[1] += len 93: next_pos += len 94: else 95: @ranges << [pos, len] 96: next_pos = pos + len 97: end 98: end 99: end 100: # calculate cumulative offsets from range sizes 101: @size = 0 102: @offsets = [] 103: @ranges.each do |pos, len| 104: @offsets << @size 105: @size += len 106: end 107: self.pos = @pos 108: end
read bytes from file, to a maximum of limit, or all available if unspecified.
# File lib/ole/ranges_io.rb, line 156 156: def read limit=nil 157: data = '' 158: return data if eof? 159: limit ||= size 160: pos, len = @ranges[@active] 161: diff = @pos - @offsets[@active] 162: pos += diff 163: len -= diff 164: loop do 165: @io.seek pos 166: if limit < len 167: s = @io.read(limit).to_s 168: @pos += s.length 169: data << s 170: break 171: end 172: s = @io.read(len).to_s 173: @pos += s.length 174: data << s 175: break if s.length != len 176: limit -= len 177: break if @active == @ranges.length - 1 178: @active += 1 179: pos, len = @ranges[@active] 180: end 181: data 182: end
you may override this call to update @ranges and @size, if applicable.
# File lib/ole/ranges_io.rb, line 185 185: def truncate size 186: raise NotImplementedError, 'truncate not supported' 187: end
# File lib/ole/ranges_io.rb, line 195 195: def write data 196: return 0 if data.empty? 197: data_pos = 0 198: # if we don't have room, we can use the truncate hook to make more space. 199: if data.length > @size - @pos 200: begin 201: truncate @pos + data.length 202: rescue NotImplementedError 203: raise IOError, "unable to grow #{inspect} to write #{data.length} bytes" 204: end 205: end 206: pos, len = @ranges[@active] 207: diff = @pos - @offsets[@active] 208: pos += diff 209: len -= diff 210: loop do 211: @io.seek pos 212: if data_pos + len > data.length 213: chunk = data[data_pos..-1] 214: @io.write chunk 215: @pos += chunk.length 216: data_pos = data.length 217: break 218: end 219: @io.write data[data_pos, len] 220: @pos += len 221: data_pos += len 222: break if @active == @ranges.length - 1 223: @active += 1 224: pos, len = @ranges[@active] 225: end 226: data_pos 227: end