<br><br><div class="gmail_quote">2008/11/21 David E DeMarle <span dir="ltr"><<a href="mailto:dave.demarle@kitware.com">dave.demarle@kitware.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div class="Wj3C7c">On Thu, Nov 20, 2008 at 9:07 PM, Paul Harris <<a href="mailto:harris.pc@gmail.com">harris.pc@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> I am just starting out with VTK, and I want to know how to properly support<br>
> multicore desktop/laptop PCs...<br>
><br>
> I have done a lot of reading, I've read the VTK OO book, and now I'm<br>
> checking out ParaView...<br>
><br>
> Am I correct in thinking that MPI is the main (only?) way that people are<br>
> processing their data in parallel?<br>
><br>
> I have tried without luck to search for info and messages about executing on<br>
> a shared memory system (ie a multicore desktop), or even threaded (seems to<br>
> complicate things a lot).<br>
> Is the solution to use MPI, but on the one node?<br>
> Or is there some sort of decent shared-memory support?<br>
> From what I read on vtkDataObject, it looks like threading is a path to a<br>
> mine field, as a lot of classes may not be thread-safe, right?<br>
><br>
><br>
> Am I also correct to say that ParaView adds a lot of parallel processing<br>
> support to VTK ? Support that is not available to VTK ?<br>
><br>
><br>
> I have lots more questions, but I'll stop there for now...<br>
> I eagerly await your responses!<br>
><br>
> thanks for your time,<br>
> Paul<br>
><br>
><br>
</div></div>> _______________________________________________<br>
> This is the private VTK discussion list.<br>
> Please keep messages on-topic. Check the FAQ at:<br>
> <a href="http://www.vtk.org/Wiki/VTK_FAQ" target="_blank">http://www.vtk.org/Wiki/VTK_FAQ</a><br>
> Follow this link to subscribe/unsubscribe:<br>
> <a href="http://www.vtk.org/mailman/listinfo/vtkusers" target="_blank">http://www.vtk.org/mailman/listinfo/vtkusers</a><br>
><br>
><br>
<br>
Many of VTK's imaging filters support thread level parallelism. When<br>
VTK is compiled with threading support, those filters in the Imaging<br>
kit will automatically spawn threads to take advantage of the<br>
available parallelism near to the innermost loop. When processing this<br>
type of data, threading is fairly easy to do and scales, in terms of<br>
execution time, very well.<br>
<br>
ParaView heavily uses VTK's process level parallelism to make all data<br>
types (images and unstructured types) scale in terms of memory size.<br>
That is, given enough aggregate RAM, any size problem can be<br>
processed. This is available in VTK through the Parallel toolkit. For<br>
the most part it is only accessible through MPI and parallelizes near<br>
the outermost loop.<br>
<br>
To make a long story short, if you are working with image data, VTK<br>
has built in multicore support. If you are working with unstructured<br>
data, your best bet is to use mpi on your multicore machine. If you<br>
want to use VTK with threading otherwise, it can be done but as you<br>
say you are heading into a minefield.<br>
</blockquote><div><br><br>Thanks for the reply David :)<br><br>One major goal of mine is to allow data to be processed in the background, and for it to "appear" on the screen when it is available.<br><br>To cut a long story short (that long story is below if you keep reading), I want to know if VTK's pipeline can be part-push and part-pull ?<br>
<br><br>longer story...<br><br>I already have this working with a different 3D system... I have a data-push system that farms out jobs to a thread pool, and I had thought about reimplementing vtkDataDrivenPipeline so that certain jobs could be done in the background and when the job is complete, the data becomes available to the pipeline.<br>
<br>But, I see a few problems with that...<br><br>1) I get the impression that when an actor or filter demands data, it expects to have the data after the call has completed. This can't be done in my thread-job world... the data is ready when its ready, and its pushed through the outputs, not pulled.<br>
<br>2) I am afraid that I'll lose the ability to use MPI to distribute load via the streaming system. In my system, the job is processed fully before the data is passed on (as a read-only reference or as a copy). Although that might be possible if I choose a filter at the end to put in a threaded job, and assign all its inputs into MPI executives. Then it'll execute in a thread but stream the data from its (MPI) inputs. That will benefit from task-level parallelism and better memory usage.<br>
Note that memory usage is not currently a problem, but things change...<br><br>3) This is doing things in threads. As we have said, its bad. I could potentially farm out the jobs to an MPI processor pool, which seems to be what VTK would do, but I am still stuck with a pull-based pipeline which freezes the GUI while it waits for processing to complete.<br>
<br><br>What could I do?<br><br>thanks,<br>Paul<br><br></div></div>