View Issue Details [ Jump to Notes ] | [ Print ] | ||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||
0014407 | ParaView | (No Category) | public | 2013-11-14 09:06 | 2016-08-24 08:25 | ||||
Reporter | Alejandro Ribes | ||||||||
Assigned To | Utkarsh Ayachit | ||||||||
Priority | immediate | Severity | minor | Reproducibility | have not tried | ||||
Status | closed | Resolution | moved | ||||||
Platform | OS | OS Version | |||||||
Product Version | 4.0 | ||||||||
Target Version | Fixed in Version | ||||||||
Summary | 0014407: Embedded Python interpreter and multi-threading | ||||||||
Description | Hi Utkarsh, We are having trouble with our integration of ParaView into SALOME, this concerns Python. I asked Adrien from CEA to describe the problem in English. Please find enclosed this description. This problem is quite important for us because we have an "alpha" tag of SALOME that should be done this week ... Cheers, Alejandro. ---------------------------------------------------------- Hi Alejandro, Here are the problems we face with PV 4.0.1 related to the use of the Python C API. Python's embedding/wrapping logic has significantly changed between 3.98 and 4.0.1. For us, this is mainly an issue because of the Python trace generation (Python module "smtrace") which works as follows: - Python module "smtrace" is loaded and creates a vtkSMPythonTraceObserver object - this C++ class registers and execute Python callbacks, located also in "smtrace" module This back and forth between Python and C++ means the Python C API is invoked from ParaView C++ code. On the other hand, in SALOME, we need multi-threading, and we also need mutliple embedded Python interpreters (function Py_NewInterpreter()) for YACS execution graphs mainly. There is one central entity managing the whole thing (GUI/PyInterp/PyLockWrapper class). It takes care of switching the proper interpreter and thread states, and acquiring the GIL when necessary, depending on the current OS thread, and selected sub-interpreter. This logic works fine, albeit a bit complicated. Now, when both ParaView's trace-generation callbacks and SALOME's Python code are called simultaneously, we have a conflict and the whole thing crashes (this happens when we load a script and interacts with the GUI at the same time). The two parts are unaware of each other. I thought a simple solution would be to de-activate VTK_NO_PYTHON_THREADS CMake option when building ParaView, but this is more complicated: this flag controls whether vtkPythonCommand::Execute() acquires the GIL before invoking Python C API (PyGILState_Ensure() function). Unfortunately, it is officially said in Python doc that: "Also note that combining this functionality [Py_NewInterpreter] with PyGILState_*() APIs is delicate, because these APIs assume a bijection between Python thread states and OS-level threads, an assumption broken by the presence of sub-interpreters. It is highly recommended that you don’t switch sub-interpreters between a pair of matching PyGILState_Ensure() and PyGILState_Release() calls." We avoid this clash in SALOME with our PyLockWrapper logic mentioned above. I tried several hacks in both ParaView and/or SALOME, none of them is satisfactory. It either ends with a SEGFAULT or a dead-lock. Simple solution for now is to have the trace generation done in pure C++, not in Python (by porting the Python module "smtrace" to a pure C++ impl). This avoids calling Python C API in ParaView, leaving SALOME's logic doing its work without interference. I am starting this as we speak, and KitWare's help would be more than welcome :-) It seems trace generation was initially written in Python to save some time, but I don't think this is a requirement. Please tell me if I am wrong! (class vtkSMPythonTraceObserver is only used by smtrace module). Longer term, if ParaView is to embed more and more Python code callbacks (as is currently done in "smtrace"), we need to come up with a better solution. I am sure Utkarsh has some ideas about this, as I saw on the VTK mail list that he faced the issue before. On our side we will try to come with a clear idea on how this could be done to satisfy both needs, and then suggest something. Best regards, Adrien Bruneton. | ||||||||
Tags | No tags attached. | ||||||||
Project | Kitware | ||||||||
Topic Name | |||||||||
Type | incorrect functionality | ||||||||
Attached Files | |||||||||
Relationships | |
Relationships |
Notes | |
(0038465) Kitware Robot (administrator) 2016-08-12 09:59 |
Resolving issue as `moved`. This issue tracker is no longer used. Further discussion of this issue may take place in the current ParaView Issues page linked in the banner at the top of this page. |
Notes |
Issue History | |||
Date Modified | Username | Field | Change |
2013-11-14 09:06 | Alejandro Ribes | New Issue | |
2013-11-14 09:06 | Alejandro Ribes | Assigned To | => Utkarsh Ayachit |
2016-08-12 09:59 | Kitware Robot | Note Added: 0038465 | |
2016-08-12 09:59 | Kitware Robot | Status | backlog => closed |
2016-08-12 09:59 | Kitware Robot | Resolution | open => moved |
2016-08-24 08:25 | Kitware Robot | View Status | private => public |
Issue History |
Copyright © 2000 - 2018 MantisBT Team |