setMaxSizeOf32bitArray(int index) for index > Integer.MAX_VALUE?

[GSQuarch]

Hi

I was asking recently about Transforms dealing with large arrays and running of memory, to which you replied that by default JLargeArrays use the regular 1D Java arrays up to 2^30 elements, and you can change that value using LargeArray.setMaxSizeOf32bitArray(). Let's say I want to use 2^34 or approx. 17bn. That exceeds the range of type int, which LargeArray.setMaxSizeOf32bitArray() uses and so throws a compilation error or a runtime error if a long is squeezed into an int.

Thanks.

[piotrwendykier]

You cannot create 1D Java arrays of size 2^34. That is why this project exists. You don't need to call LargeArray.setMaxSizeOf32bitArray(). Just make sure you have enough memory and set -Xmx accordingly.

[GSQuarch]

I must confess I'm confused. The JavaDoc for FloatLargeArray states "An array of floats that can store up to 2^63 elements." but setMaxSizeOf32bitArray() takes an int size and then you state "You cannot create 1D Java arrays of size 2^34". I'm baffled. I thought the whole idea of using, say, FloatLargeArray was to exceed the Java limit? If this is not the case then why would I bother using FloatLargeArray?

"An array of floats that can store up to 2^63 elements." --> How is this achieved?

[wendykierp]

Just pass 2^34 to the FloatLargeArray constructor. Make sure you have enough memory and set -Xmx accordingly.
If you still have a problem, then share your code, settings, results and machine configuration.