View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0012730VTK(No Category)public2011-11-16 06:182011-11-17 08:21
ReporterBryn Lloyd 
Assigned ToDavid Gobbi 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version5.8.0 
Target VersionFixed in Version 
Summary0012730: Importing large arrays from numpy inyo VTK does not work.
DescriptionImporting large arrays from numpy into VTK does not work.
This was already mentioned on the users list, and reproduced by David Gobbi.

Example 1:
==========
from numpy import zeros
import vtk

a = zeros(2**32 - 1).astype(uint8) #to create a massive
array im = vtk.vtkImageImport() #believe this
is the standard method to convert to vtk
im.CopyImportVoidPointer(a.tostring(),a.size) #do conversion

throws the following error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  SystemError: error return without exception set


Example 2:
==========
from vtk.util import numpy_support as vn
import vtk
import numpy

a = numpy.zeros(2**32-1, numpy.uint8)
aV = vn.numpy_to_vtk(pp, deep=False)

The last line throws the error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "P:\vtk-5.6.1\x64\Wrapping\Python\vtk\util\numpy_support.py", line 165, in numpy_to_vtk
    result_array.SetVoidArray(z_flat, len(z_flat), 1)
OverflowError: size does not fit in an int

TagsNo tags attached.
ProjectTBD
Typeincorrect functionality
Attached Files

 Relationships

  Notes
(0027697)
Bryn Lloyd (reporter)
2011-11-16 06:31

 I just noticed that the second example was run using VTK 5.6.1. I checked in the docs and see that now vtkIdType is used, instead of int in the function SetVoidArray.

Sorry for the confusion. Using VTK 5.8 I get the same error also for the second example:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "P:\vtk-5.8.0-epd-712\x64\Wrapping\Python\vtk\util\numpy_support.py", lin
e 160, in numpy_to_vtk
    result_array.SetVoidArray(z_flat, len(z_flat), 1)
SystemError: error return without exception set
(0027698)
David Gobbi (developer)
2011-11-16 10:27
edited on: 2011-11-16 10:27

 The problem might not be due to large arrays, it might have something to do specifically with the value "2**32-1". For example, the following code with "2**32+1" or "2**32" works fine on my mac. Bryn, can you confirm?

====
import numpy
import vtk

a = numpy.zeros(2**32 + 1, numpy.uint8)
im = vtk.vtkImageImport()
im.CopyImportVoidPointer(a, len(a))
====
(0027699)
David Gobbi (developer)
2011-11-16 10:35

 On further inspection, it seems to be a sign issue. On my machine it fails for any array size between 2**31 and 2**32-1, which correspond to values that can be represented by unsigned int but not by signed int.
(0027704)
David Gobbi (developer)
2011-11-16 11:47

 I have merged commit dee7431302a85b94bc112bde7855e06e26f35b54 to fix this bug.

http://vtk.org/gitweb?p=VTK.git;a=commitdiff;h=dee74313 [^]

diff --git a/Wrapping/Python/vtkPythonArgs.cxx b/Wrapping/Python/vtkPythonArgs.cxx
index 8302337..29f59d9 100644
--- a/Wrapping/Python/vtkPythonArgs.cxx
+++ b/Wrapping/Python/vtkPythonArgs.cxx
@@ -228,9 +228,11 @@ bool vtkPythonGetValue(PyObject *o, const void *&a)
     if (b->bf_getsegcount(o, NULL) == 1)
       {
       void *p;
- int s = (int)b->bf_getreadbuffer(o, 0, &p);
- if (s >= 0)
+ Py_ssize_t sz = b->bf_getreadbuffer(o, 0, &p);
+ if (sz >= 0 && sz <= VTK_INT_MAX)
         {
+ // check for pointer mangled as string
+ int s = (int)sz;
         a = vtkPythonUtil::UnmanglePointer((char *)p, &s, "void_p");
         if (s >= 0)
           {
@@ -247,6 +249,12 @@ bool vtkPythonGetValue(PyObject *o, const void *&a)
           PyErr_SetString(PyExc_TypeError, "cannot get a void pointer");
           }
         }
+ else if (sz >= 0)
+ {
+ // directly use the pointer to the buffer contents
+ a = p;
+ return true;
+ }
       return false;
       }
     PyErr_SetString(PyExc_TypeError, "buffer must be single-segment");
(0027705)
Bryn Lloyd (reporter)
2011-11-17 08:10

 I tested the fix on a Linux machine and it seems to work. Thanks!
(0027706)
David Gobbi (developer)
2011-11-17 08:21

 I'll recommend this patch for VTK 5.8.1.

 Issue History
Date Modified Username Field Change
2011-11-16 06:18 Bryn Lloyd New Issue
2011-11-16 06:31 Bryn Lloyd Note Added: 0027697
2011-11-16 10:27 David Gobbi Note Added: 0027698
2011-11-16 10:27 David Gobbi Note Edited: 0027698
2011-11-16 10:35 David Gobbi Note Added: 0027699
2011-11-16 11:43 David Gobbi Assigned To => David Gobbi
2011-11-16 11:43 David Gobbi Status backlog => tabled
2011-11-16 11:47 David Gobbi Note Added: 0027704
2011-11-17 08:10 Bryn Lloyd Note Added: 0027705
2011-11-17 08:21 David Gobbi Note Added: 0027706
2011-11-17 08:21 David Gobbi Status tabled => closed
2011-11-17 08:21 David Gobbi Resolution open => fixed

Copyright © 2000 - 2018 MantisBT Team