View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007640ParaViewBugpublic2008-09-09 11:572009-09-11 14:10
ReporterUtkarsh Ayachit 
Assigned ToUtkarsh Ayachit 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version3.6.2Fixed in Version3.6.2 
Summary0007640: MPI Communication cleanup
DescriptionRunning ParaView as 16 processes cluster with openmpi (amber) is excruciatingly slow esp. in creating sources/filters/representations.
Additional InformationSuggested fixes:
* Don't use MPI_Ssend by default. Make it a CMake variable which can be set on clusters with buggy MPI_Send implementations.
* When broadcasting to all processess, the root node sequentially sends the message to all the satellite. Instead we should use a binary-tree like communication hierarchy to broadcast messages.
* Use buffering for SendStream(). The buffer will be flushed when GatherInformation or GetLastResult() is called. This will avoid several small messages being sent to the servers.
TagsNo tags attached.
Project
Topic Name
Type
Attached Files

 Relationships

  Notes
(0015039)
Utkarsh Ayachit (administrator)
2009-02-17 13:28

 (2) and (3) are already fixed. We just need to add a Cmake option to enable ssend.
(0017241)
Utkarsh Ayachit (administrator)
2009-08-31 08:56

 /cvsroot/ParaView3/ParaView3/vtkPVConfig.h.in,v <-- vtkPVConfig.h.in
new revision: 1.9; previous revision: 1.8
(0017239)
Utkarsh Ayachit (administrator)
2009-08-31 09:41
edited on: 2009-08-31 09:43

 Adding new advanced cmake variable PARAVIEW_USE_MPI_SSEND
(off by default) to make ParaView use MPI_Ssend, instead of MPI_Send.

/cvsroot/ParaView3/ParaView3/CMakeLists.txt,v <-- CMakeLists.txt
new revision: 1.110; previous revision: 1.109
/cvsroot/ParaView3/ParaView3/Servers/Common/vtkProcessModule.cxx,v <-- Servers/Common/vtkProcessModule.cxx
new revision: 1.95; previous revision: 1.94


Pending commit to 3.6 branch.
(0017240)
Utkarsh Ayachit (administrator)
2009-08-31 09:43

 Reminder sent to: Alan Scott

Alan,

Make sure that you set the new PARAVIEW_USE_MPI_SSEND to ON before building your binaries on the machines with MPI_Send buffer overflow issues.
(0017273)
Utkarsh Ayachit (administrator)
2009-09-01 09:55

 /cvsroot/ParaView3/ParaView3/CMakeLists.txt,v <-- CMakeLists.txt
new revision: 1.97.2.7; previous revision: 1.97.2.6
/cvsroot/ParaView3/ParaView3/Servers/Common/vtkProcessModule.cxx,v <-- Servers/Common/vtkProcessModule.cxx
new revision: 1.92.2.2; previous revision: 1.92.2.1

/cvsroot/ParaView3/ParaView3/vtkPVConfig.h.in,v <-- vtkPVConfig.h.in
new revision: 1.5.2.1; previous revision: 1.5
(0017379)
Alan Scott (manager)
2009-09-11 14:10

 Blackrose seems fine. I tested on Dave's large CTH dataset. If we get hangs on larger stuff, be sure to test this first. Tested remote server, 64 pvservers.

 Issue History
Date Modified Username Field Change
2008-09-09 11:57 Utkarsh Ayachit New Issue
2009-02-17 13:27 Utkarsh Ayachit Status backlog => tabled
2009-02-17 13:27 Utkarsh Ayachit Assigned To => Utkarsh Ayachit
2009-02-17 13:27 Utkarsh Ayachit Category Development => 3.6
2009-02-17 13:28 Utkarsh Ayachit Note Added: 0015039
2009-02-17 13:28 Utkarsh Ayachit Assigned To Utkarsh Ayachit => Burlen
2009-04-22 13:23 Utkarsh Ayachit Assigned To Burlen => Yumin Yuan
2009-05-13 13:45 Utkarsh Ayachit Target Version => 3.6
2009-08-25 12:17 Ken Moreland Category => Bug
2009-08-25 12:17 Ken Moreland Target Version 3.6 => 3.6.2
2009-08-31 08:56 Utkarsh Ayachit Note Added: 0017241
2009-08-31 09:36 Utkarsh Ayachit Assigned To Yumin Yuan => Utkarsh Ayachit
2009-08-31 09:41 Utkarsh Ayachit Note Added: 0017239
2009-08-31 09:43 Utkarsh Ayachit Note Added: 0017240
2009-08-31 09:43 Utkarsh Ayachit Note Edited: 0017239
2009-09-01 09:55 Utkarsh Ayachit Note Added: 0017273
2009-09-01 09:55 Utkarsh Ayachit Status tabled => @80@
2009-09-01 09:55 Utkarsh Ayachit Fixed in Version => 3.6.2
2009-09-01 09:55 Utkarsh Ayachit Resolution open => fixed
2009-09-11 14:10 Alan Scott Note Added: 0017379
2009-09-11 14:10 Alan Scott Status @80@ => closed

Copyright © 2000 - 2018 MantisBT Team