Notes
(0029162) David Gobbi (developer) 2012-09-10 09:41	Can you replace Wrapping/vtkParsePreprocess.c with the attached file? It contains a bug fix that is being considered for inclusion in VTK 5.10.1. The bug could cause the buffer for the wrapper's parser to become corrupt in certain situations.

(0029163) Pietro Cerutti (reporter) 2012-09-10 10:28	David, thanks for your prompt reply. Unfortunately the attached files doesn't help. I have exactly the same failure. Is there any way I can help debugging it?

(0029164) David Gobbi (developer) 2012-09-10 10:41	Can you add an assert just after the label yyexhaustedlab: in vtkParse.tab.c? That is where it is printing "memory exhausted". If you can generate a stack trace from that point, it should give some hints about what is going wrong.

(0029165) Pietro Cerutti (reporter) 2012-09-10 11:28	here's the backtrace, I don't know how much it helps.. unfortunately bison-generated files are not recognized by gdb (gdb) bt #0 0x000000080096a2bc in kill () from /lib/libc.so.7 #1 0x0000000800968eeb in abort () from /lib/libc.so.7 #2 0x000000080094c8a5 in __assert () from /lib/libc.so.7 #3 0x000000000040b1c0 in yyparse () at vtkParse.tab.c:8385 #4 0x0000000000418ccd in vtkParse_ParseFile (filename=0x7fffffffd751 "/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Common/vtkMath.h", ifile=0x800bc1700, errfile=0x800ba7ee0) at vtkParse.y:4654 #5 0x0000000000404ff3 in main (argc=214, argv=0x7fffffffb888) at /home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Wrapping/vtkParseMain.c:275 (gdb) frame 4 #4 0x0000000000418ccd in vtkParse_ParseFile (filename=0x7fffffffd751 "/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Common/vtkMath.h", ifile=0x800bc1700, errfile=0x800ba7ee0) at vtkParse.y:4654 4654 vtkParse.y: No such file or directory. in vtkParse.y

(0029166) David Gobbi (developer) 2012-09-10 13:54	I guess that a stack trace doesn't help much. There isn't any stack to trace, because the whole parse takes place within yyparse(). The next step would be to play around the header files to see what text in the files is causing the problem. For instance, you could edit Common/vtkCommonHierarchy.data to remove vtkMath.h from that file, and then re-run the vtkWrapHierarchy command to see if it was specifically vtkMath that was causing the problem. If so, then you could try some simple edits to vtkMath.h such as adding extra blank lines at the top or at the bottom of the file to see if that makes any difference.

(0029168) Pietro Cerutti (reporter) 2012-09-11 04:37	I have removed vtkMath.h from Common/vtkCommonHierarchy.data but vtkWrapJava still fails with it. I don't know what I'm doing wrong... cd /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Common && ../bin/vtkWrapJava --concrete --hints /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Wrapping/hints --types /usr/ho me/cerutti/FreeBSD/ports/math/vtk5/work/.build/Common/vtkCommonHierarchy.txt -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.b uild/Common -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/VolumeRendering -I /usr/home/cerutti/FreeBSD/ports /math/vtk5/work/.build/Rendering -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Charts -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Chemistry -I /usr/home/cerutti/F reeBSD/ports/math/vtk5/work/.build/Utilities/vtkalglib -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Wrapping/Python -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Wrap ping/Python -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Infovis -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Geovis -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/V TK/Views -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Parallel -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/VolumeRendering -I /usr/home/cerutti/FreeBSD/ports/math/vtk5 /work/VTK/Hybrid -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Widgets -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Rendering -I /usr/home/cerutti/FreeBSD/ports/math/vtk 5/work/VTK/Charts -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Chemistry -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Rendering/Testing/Cxx -I /usr/home/cerutti/FreeBSD /ports/math/vtk5/work/VTK/IO -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Imaging -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Graphics -I /usr/home/cerutti/FreeBSD/por ts/math/vtk5/work/VTK/GenericFiltering -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Filtering -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Common -I /usr/home/cerutti/F reeBSD/ports/math/vtk5/work/VTK/Utilities -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Common/Testing/Cxx -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/vtkn etcdf/include -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/vtknetcdf/include -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/vtklibproj4 -I /usr/hom e/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/vtklibproj4 -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/DICOMParser -I /usr/home/cerutti/FreeBSD/ports/math/vt k5/work/VTK/Utilities/DICOMParser -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/vtkfreetype/include -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/vtkfreetype/include -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/LSDyna -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/LSDyna -I /usr/home/cerutti/F reeBSD/ports/math/vtk5/work/.build/Utilities/MaterialLibrary -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/MaterialLibrary -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/ work/.build/Utilities/vtkmetaio -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/vtkmetaio -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/verdict -I /u sr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/verdict -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/vtkhdf5 -I /usr/home/cerutti/FreeBSD/ports/math/vtk5 /work/VTK/Utilities/vtkhdf5 -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/vtkhdf5/src -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/vtkhdf5/src -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/vtkhdf5/hl/src -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/vtkhdf5/hl/src -I /usr/home/cerutti/FreeBSD /ports/math/vtk5/work/.build/Utilities/Cosmo -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/Cosmo -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/Utilities/VPIC -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/VPIC -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilities/utf8/source -I /usr/home/cerutti/FreeBSD/ports/math/ vtk5/work/VTK/GUISupport/Qt -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/.build/GUISupport/Qt -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/GUISupport/Qt/Chart -I /usr/home/ cerutti/FreeBSD/ports/math/vtk5/work/.build/GUISupport/Qt/Chart -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Infovis -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Utilit ies/vtkalglib -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Geovis -I /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Views -I /usr/local/include -I /usr/local/include -I /usr /local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/inc lude -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /u sr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/i nclude -I /usr/local/include -I /usr/local/include/tcl8.6 -I /usr/local/include/tk8.6 -I /usr/local/openjdk6/include -I /usr/local/openjdk6/include/freebsd -I /usr/local/openjdk6/include -I /usr/include -I /usr/local/include -I /usr/local/include -I /usr/local/include -I /usr/local/include /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Common/vtkMath.h /usr/home/cer utti/FreeBSD/ports/math/vtk5/work/.build/Common/vtkMathJava.cxx memory exhausted * SYNTAX ERROR found in parsing the header file /usr/home/cerutti/FreeBSD/ports/math/vtk5/work/VTK/Common/vtkMath.h before line 6338 * *** [Common/vtkMathJava.cxx] Error code 1 1 error

(0029169) David Gobbi (developer) 2012-09-11 11:01	The vtk*Hierarchy.data files only control what goes into vtkWrapHierarchy, not what goes into vtkWrapJava. So we have discovered something here: the file vtkMath.h seems to be the only file that causes the crash (otherwise we would have seen vtkWrapHierarchy crash on some other file after vtkMath.h was removed). The fact that vtkWrapJava also crashes on vtkMath.h gives further evidence that it is something specific about vtkMath.h that the wrapper-parser does not like. My suggestions: 1) Download a new copy of VTK 5.10 and do a diff of your vtkMath.h against the new copy. There is a chance (however unlikely) that your copy of vtkMath.h is corrupted somehow. 2) If (1) wasn't the problem, then you can try trimming lines from the end of vtkMath.h until vtkWrapJava doesn't crash (run vtkWrapJava on its own to test for this, rather than letting it execute as part of the VTK build). Basically this amounts to trial-and-error to figure out what it is about vtkMath.h that is causing the crash. 3) You could get the git master of VTK (which will soon become the VTK 6 release) and see if it has the same build problem.

(0029175) Pietro Cerutti (reporter) 2012-09-12 04:33	Removing the last two methods vtkMath::IsInf and vtkMath::IsNan solves the problem. It doesn't seem to be a matter of lines of code, since removing the previous method vtkMath::ClampAndNormalizeValue leaving the last two in the file doesn't solve it.

(0029176) David Gobbi (developer) 2012-09-12 07:51	That's enough info that we should be able to do a work-around. Instead of removing those lines, put them in a "#ifndef __WRAP__" block so that the wrapper tools ignore them: #ifndef __WRAP__ #if defined(VTK_HAS_ISINF) //----------------------------------------------------------------------------- inline int vtkMath::IsInf(double x) { return (isinf(x) ? 1 : 0); } #endif #if defined(VTK_HAS_ISNAN) //----------------------------------------------------------------------------- inline int vtkMath::IsNan(double x) { return (isnan(x) ? 1 : 0); } #endif #endif I guess that we still don't know why this problem only occurs on FreeBSD. The VTK 5.10 release was tested on Linux, OS X, AIX, and Windows. It is possible that on FreeBSD "isnan()" and "isinf()" are macros that expand into some complex code that the wrapper parser cannot deal with. Can you provide details about what versions of FreeBSD, gcc, and glibc you are using?

(0029177) Pietro Cerutti (reporter) 2012-09-12 08:12	Here are the definitions of isnan and isinf from our math.h header #define isinf(x) \ ((sizeof (x) == sizeof (float)) ? __isinff(x) \ : (sizeof (x) == sizeof (double)) ? isinf(x) \ : __isinfl(x)) #define isnan(x) \ ((sizeof (x) == sizeof (float)) ? __isnanf(x) \ : (sizeof (x) == sizeof (double)) ? isnan(x) \ : __isnanl(x)) and the relevant functions implementations from our libc __weak_reference(__isinf, isinf); int __isinf(double d) { union IEEEd2bits u; u.d = d; return (u.bits.exp == 2047 && u.bits.manl == 0 && u.bits.manh == 0); } int __isinff(float f) { union IEEEf2bits u; u.f = f; return (u.bits.exp == 255 && u.bits.man == 0); } int __isinfl(long double e) { union IEEEl2bits u; u.e = e; mask_nbit_l(u); #ifndef __alpha__ return (u.bits.exp == 32767 && u.bits.manl == 0 && u.bits.manh == 0); #else return (u.bits.exp == 2047 && u.bits.manl == 0 && u.bits.manh == 0); #endif } __weak_reference(__isnan, isnan); __weak_reference(__isnanf, isnanf); int __isnan(double d) { union IEEEd2bits u; u.d = d; return (u.bits.exp == 2047 && (u.bits.manl != 0 \|\| u.bits.manh != 0)); } int __isnanf(float f) { union IEEEf2bits u; u.f = f; return (u.bits.exp == 255 && u.bits.man != 0); } > g++ --version g++ (GCC) 4.2.1 20070831 patched [FreeBSD] Copyright (C) 2007 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(0029179) David Gobbi (developer) 2012-09-12 08:32	The wrapper-parser seems to be expanding this macro recursively, because of the "isnan(x)" within the macro definition. There is a check in the parser to block recursive expansion. Now that I have this code, I can test to see why the check is not working. #define isnan(x) \ ((sizeof (x) == sizeof (float)) ? __isnanf(x) \ : (sizeof (x) == sizeof (double)) ? isnan(x) \ : __isnanl(x)) Thanks for all the work that you've done on this. Now that I know the root cause of the crash, I can try to find a fix.

(0029180) Pietro Cerutti (reporter) 2012-09-12 08:35	Yes the wrapper-parse should be taught to understand __weak_reference :) Thanks for your time!

(0029181) David Gobbi (developer) 2012-09-12 08:37	Why do you think __weak_reference has anything to do with the crash?

(0029182) Pietro Cerutti (reporter) 2012-09-12 08:41	wouldn't __weak_reference(__isinf, isinf); resolve the recursive expansion?

(0029184) David Gobbi (developer) 2012-09-12 08:54	It isn't necessary, as far as I understand. One of the rules of the C/C++ preprocessor is that a macro is undefined within its own expansion. So when "isnan" is found within the "isnan" expansion, the parser looks for an "isnan" function. So I'm guessing that it is only then, _after_ the recursion has already been broken, that isnan -> __isnan because of the weakref.

(0029186) Pietro Cerutti (reporter) 2012-09-12 09:05	Yes, it's correct. What I was thinking is, maybe the parser screws up things because it doesn't find an isnan/isinf function... but I'm only guessing here, since I have no idea of the parser's internals..

(0029188) David Gobbi (developer) 2012-09-12 09:25	This parser only reads the function definitions, it doesn't evaluate or compile the functions, so a missing symbol wouldn't cause a problem. Of course the parser doesn't do anything until the preprocessor has expanded the macros, and it is the preprocessor that seems to be the problem. And the "memory exhausted" error strongly hints at infinite recursion. After I get this fixed, I'll have the definitive answer.

(0029198) David Gobbi (developer) 2012-09-13 10:56	I've looked through the parser code and have verified that the macro recursion checks were not merged until after the VTK 5.10 release. The VTK git master branch (and the upcoming VTK 6.0 release) should be able to parse vtkMath.h on FreeBSD. Pietro, is there any chance that you can checkout the VTK git master and verify this?

(0029210) Pietro Cerutti (reporter) 2012-09-14 12:41	David, git/master builds fine on FreeBSD wrt vtkMath. If you're not in a hurry to tag vtk 6, I might send in a few patches next week, especially for Utilities/KWSys/vtksys/SystemInformation.cxx. It would be great to have those applied upstream to avoid having to patch the port here.

(0029288) David Gobbi (developer) 2012-09-26 13:00	A patch has been submitted (commit f27a4ba6) for inclusion in the upcoming 5.10.1. VTK 6 is probably still a few weeks away (there has not been a release announcement). There are three ways that you can contribute your patches. 1) through gerrit, http://www.vtk.org/Wiki/VTK/Git/Develop [^] 2) by submitting the patch file to the bugtracker (obviously) 3) by submitting the patch to the vtk-developers mailing list

Issue History
Date Modified	Username	Field	Change
2012-09-10 08:34	Pietro Cerutti	New Issue
2012-09-10 09:37	David Gobbi	File Added: vtkParsePreprocess.c
2012-09-10 09:41	David Gobbi	Note Added: 0029162
2012-09-10 10:28	Pietro Cerutti	Note Added: 0029163
2012-09-10 10:41	David Gobbi	Note Added: 0029164
2012-09-10 11:28	Pietro Cerutti	Note Added: 0029165
2012-09-10 13:54	David Gobbi	Note Added: 0029166
2012-09-11 04:37	Pietro Cerutti	Note Added: 0029168
2012-09-11 11:01	David Gobbi	Note Added: 0029169
2012-09-12 04:33	Pietro Cerutti	Note Added: 0029175
2012-09-12 07:51	David Gobbi	Note Added: 0029176
2012-09-12 08:12	Pietro Cerutti	Note Added: 0029177
2012-09-12 08:32	David Gobbi	Note Added: 0029179
2012-09-12 08:35	Pietro Cerutti	Note Added: 0029180
2012-09-12 08:37	David Gobbi	Note Added: 0029181
2012-09-12 08:41	Pietro Cerutti	Note Added: 0029182
2012-09-12 08:54	David Gobbi	Note Added: 0029184
2012-09-12 09:05	Pietro Cerutti	Note Added: 0029186
2012-09-12 09:25	David Gobbi	Note Added: 0029188
2012-09-13 10:56	David Gobbi	Note Added: 0029198
2012-09-14 12:41	Pietro Cerutti	Note Added: 0029210
2012-09-26 13:00	David Gobbi	Note Added: 0029288
2012-09-28 08:13	David Gobbi	Assigned To	=> David Gobbi
2012-09-28 08:13	David Gobbi	Status	backlog => tabled
2012-09-28 08:55	David Gobbi	Status	tabled => closed
2012-09-28 08:55	David Gobbi	Resolution	open => fixed

Relationships