<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Robert,<br>
<br>
meanwhile we worked out an implementation solving the problem.<br>
After discovering that the EnSightGoldReader (ASCII-version) uses
a caching system for the time steps we ported this in a slightly
modified way to the EnSightGoldBinaryReader. In addition we added
functionality to fill the cache in advance in case a file index is
available. <br>
This circumvents the problem with changing geometries.<br>
Attached are two patch files, the first one implements the caching
for the binary reader and the second one implements the use of the
file index. It would be great if this could be included in VTK. If
you find any problems with the implementation, please let us know
so we can find a solution.<br>
<br>
Thank you very much in advance<br>
<br>
Karl & Georg<br>
<br>
<br>
On 10/02/14 17:55, Georg Hammerl wrote:<br>
</div>
<blockquote cite="mid:52F90488.3080802@lnm.mw.tum.de" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<div class="moz-cite-prefix">Hi Robert,<br>
<br>
it is been a while since we had this problem but we are running
into it again. Therefore I would like to reopen the discussion
on making use of the file index in the
vtkEnSightGoldBinaryReader. I expect two advantages by doing so,
firstly it is more efficient and secondly, it can be used to
enable changing geometries over time (variable number of
points). At the moment, time steps are skipped (when reading
vectors) by assuming a constant number of points in the mesh
over all time steps which is of course not correct for variable
number of points.<br>
<br>
I implemented a preliminary version of reading the file index.
It works fine (and is fast :-)) and enables us to visualize
changing geometries. The implementation is not yet in a style
that could be included in VTK, I therefore wanted to get some
pointers what is the best way to include this feature into VTK.<br>
<br>
I have thought of the following scenarios:<br>
<br>
1)<br>
when reading Scalars, Vectors, Tensors, ... the blocks of data
until timeStep-1 are being skipped one by one assuming a
constant geometry. To skip this we look for a file index and, if
present, this->IFile is set to the correct position in the
file.<br>
This has the advantage of being very easy to implement, however
it has to be implemented for every type of field separately.
Also one has to jump to the end of the file for every field in
every file whenever a new time step is requested.<br>
<br>
2)<br>
provide a method in the vtkEnSightGoldBinaryReader, that
implements the reading of the file index and setting
this->IFile to the correct position (for example a
SkipTimestepsUsingFileindex(int timestep) ) or that returns the
address(es) of a(ll) time step(s), which can then be used in the
Read{Scalars,Vectors,..}.. methods.<br>
This way only one implementation that actually reads the file
index is needed, but it still requires the file index to be read
for every field in every file whenever a new time step is
requested.<br>
<br>
3)<br>
Read the file index of all files when the reader is initialized
and store the timestep positions per file in a member of the
vtkEnSightGoldBinaryReader. This way the index has to be read
only once and timesteps can be skipped by referring to the
member. Drawback of this method is, that I need some hints on
how this could be implemented consistent with the rest of the
reader (i.e. map<std::string, std::vectors<long> >
vs. string[] and long[][] or anything in between).<br>
<br>
There are some lines of code attached that show the proof of
concept implementation. I am looking forward to getting some
suggestions.<br>
<br>
Cheers,<br>
<br>
Georg<br>
<br>
<br>
On 19/04/13 14:51, Robert Maynard wrote:<br>
</div>
<blockquote
cite="mid:CAFzjYVq6HW9MHg9cOiwnMrFGXh-YPiJ8kj9iKqThL+8kKbpN=Q@mail.gmail.com"
type="cite">
<meta http-equiv="Context-Type" content="text/html;
charset=windows-1252">
<div dir="ltr">The file index is required by Ensight for all
transient single file formats. The documentation also states
that Ensight automatically writes out the file index
information, so I would consider any transient single file
format that doesn't have it be malformed.</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, Apr 19, 2013 at 7:57 AM,
Georg Hammerl <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:hammerl@lnm.mw.tum.de" target="_blank">hammerl@lnm.mw.tum.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote">
<div>
<div>Hi Robert,<br>
<br>
thanks for the hint with the file index. The ensight
gold files I have available come with a file index. I
have tried to use it in Paraview, however it seems
that the VTK/IO/Ensight/vtkEnSightGoldBinaryReader
does not make use of the index. <br>
Unfortunately, I do not have enough insight into the
inner workings of VTK, so I cannot see myself creating
a clean implementation for using the file index.<br>
<br>
As the file index is just optional, I think it should
be worth implementing an efficient way of reading
ensight gold format even without the file index.<br>
<br>
Cheers,<br>
<br>
Georg
<div>
<div class="h5"><span><br>
</span><br>
<br>
On 18/04/13 20:06, Robert Maynard wrote:<br>
</div>
</div>
</div>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">Hi Georg,
<div><br>
</div>
<div>You are correct that a
more efficient solution is possible. After
reading the Ensight Gold binary format ( <a
moz-do-not-send="true"
href="http://www-vis.lbl.gov/NERSC/Software/ensight/doc/OnlineHelp/UM-C11.pdf"
target="_blank">http://www-vis.lbl.gov/NERSC/Software/ensight/doc/OnlineHelp/UM-C11.pdf</a>
) I have discovered how you can do this
more efficiently.</div>
<div><br>
</div>
<div>If you are start at 11-37 you will see that
they state:</div>
<div><br>
</div>
<blockquote>
<div>
<div>"Note 3: Efficient reading of each file
(especially binary) is facilitated by </div>
</div>
<div>
<div>appending each file with a file index.
A file index contains appropriate </div>
</div>
<div>
<div>information to access the file byte
positions of each time step in the file.
(EnSight </div>
</div>
<div>
<div>automatically appends a file index to
each file when exporting in transient
single </div>
</div>
<div>
<div>file format.) If used, the file index
must follow the last END TIME STEP</div>
</div>
<div>
<div>wrapper in each file."</div>
</div>
<div>
<div><br>
</div>
</div>
<div>
<div>File Index Usage:</div>
</div>
<div>
<div>ASCII, Binary, Item, Description</div>
</div>
<div>
<div>“%20d\n”, sizeof(int), n, Total number
of data time steps in the file.</div>
</div>
<div>
<div>“%20d\n”, sizeof(long), fb1, File byte
loc for contents of 1st time step*</div>
</div>
<div>
<div>“%20d\n”, sizeof(long), fb2, File byte
loc for contents of 2nd time step*</div>
</div>
<div>
<div>
<div>. . ., . . ., . . ., . . .</div>
</div>
</div>
<div>
<div>
<div>“%20d\n”, sizeof(long), fbn, File
byte loc for contents of nth time step*</div>
</div>
</div>
<div>
<div>
<div>“%20d\n”, sizeof(int), flag,
Miscellaneous flag (= 0 for now)</div>
</div>
</div>
<div>
<div>
<div>“%20d\n”, sizeof(long), fb, of item n
File byte loc for Item n above</div>
</div>
</div>
<div>
<div>
<div>“%s\n”, sizeof(char)*80,
“FILE_INDEX”, File index keyword</div>
</div>
</div>
<div>
<div>* Each file byte location is the first
byte that follows the BEGIN TIME STEP
record</div>
</div>
</blockquote>
<div><br>
</div>
<div>To me the best way is to read the file in
reverse looking for FILE_INDEX as the last 80
characters. From that you parse the previous
long as</div>
<div>the new seek position which jumps you to
the start of the file index records. After
reading the file index records you have a
lookup table for all seek positions of the
time steps in that ensight file.</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Mar 6, 2013 at
4:23 AM, Georg Hammerl <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:hammerl@lnm.mw.tum.de"
target="_blank">hammerl@lnm.mw.tum.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote">Hello,<br>
<br>
I have already posted this issue on the
paraview mailing list, but then I realized
that it is more of a VTK problem.<br>
I can load the first time step of my results
but when I switch to the next step (in which
4 points are added), paraview freezes and
htop shows 100% load for this process.
Surprisingly, this only happens when I use
binary ensight gold format. The same results
written in ascii ensight gold format work.
The ens_checker tells me for both cases that
I have valid output files.<br>
<br>
Meanwhile I had a look into the source code
and discovered the issue. Whenever the
variable-files are read for a time steps
greater than 1, Paraview has to jump in the
binary file to the desired timestep. This is
done by partially parsing and skipping
blocks until the correct "BEGIN TIME STEP"
is found. In order to skip the correct block
length, Paraview uses the number of points
(numPts) from the current geometry step.
This breaks with a varying number of points
per timestep.<br>
As an example my first step contains 2
points and the second step 6 points,
therefore when Paraview skips the first time
step, it skips a block length equivalent to
6 points. The correct number would be 2
points. It therefore skips past the next
"BEGIN TIME STEP".<br>
<br>
In order to verify my theory I have
implemented a little hack which looks for
the next "BEGIN TIME STEP" by brute force.
In case someone wants to try this I have
attached a patch which can be used in
conjunction with the attached examples.<br>
<br>
A clean solution would be to skip the
correct number of points for every given
timestep. Unfortunately I do not have enough
insight into the VTK ensight reader to
implement this and would appreciate any help
in finding a clean solution to this.<br>
<br>
Cheers,<br>
<br>
Georg<span><br>
<br>
-- <br>
Dipl.-Ing. Georg Hammerl<br>
Lehrstuhl für Numerische Mechanik<br>
Technische Universität München<br>
Boltzmannstrasse 15, D-85747 Garching b.
München<br>
phone <a moz-do-not-send="true"
href="tel:%2B49%2089%20289%2015237"
value="+498928915237" target="_blank">+49
89 289 15237</a><br>
fax <a moz-do-not-send="true"
href="tel:%2B49%2089%20289%2015301"
value="+498928915301" target="_blank">+49
89 289 15301</a><br>
<a moz-do-not-send="true"
href="mailto:hammerl@lnm.mw.tum.de"
target="_blank">hammerl@lnm.mw.tum.de</a><br>
<a moz-do-not-send="true"
href="http://www.lnm.mw.tum.de"
target="_blank">http://www.lnm.mw.tum.de</a><br>
<br>
</span><br>
_______________________________________________<br>
Powered by <a moz-do-not-send="true"
href="http://www.kitware.com"
target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at
<a moz-do-not-send="true"
href="http://www.kitware.com/opensource/opensource.html"
target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the
VTK FAQ at: <a moz-do-not-send="true"
href="http://www.vtk.org/Wiki/VTK_FAQ"
target="_blank">http://www.vtk.org/Wiki/VTK_FAQ</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a moz-do-not-send="true"
href="http://www.vtk.org/mailman/listinfo/vtkusers"
target="_blank">http://www.vtk.org/mailman/listinfo/vtkusers</a><br>
<br>
</blockquote>
</div>
<br>
<br>
<div><br>
</div>
-- <br>
Robert Maynard </div>
</blockquote>
<br>
<br>
<pre cols="72">--
Dipl.-Ing. Georg Hammerl
Lehrstuhl für Numerische Mechanik
Technische Universität München
Boltzmannstrasse 15, D-85747 Garching b. München
phone <a moz-do-not-send="true" href="tel:%2B49%2089%20289%2015237" value="+498928915237" target="_blank">+49 89 289 15237</a>
fax <a moz-do-not-send="true" href="tel:%2B49%2089%20289%2015301" value="+498928915301" target="_blank">+49 89 289 15301</a>
<a moz-do-not-send="true" href="mailto:hammerl@lnm.mw.tum.de" target="_blank">hammerl@lnm.mw.tum.de</a>
<a moz-do-not-send="true" href="http://www.lnm.mw.tum.de" target="_blank">http://www.lnm.mw.tum.de</a>
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br>
<div><br>
</div>
-- <br>
Robert Maynard </div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Dipl.-Ing. Georg Hammerl
Lehrstuhl für Numerische Mechanik
Technische Universität München
Boltzmannstrasse 15, D-85747 Garching b. München
phone +49 89 289 15237
fax +49 89 289 15301
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:hammerl@lnm.mw.tum.de">hammerl@lnm.mw.tum.de</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.lnm.mw.tum.de">http://www.lnm.mw.tum.de</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Dipl.-Ing. Georg Hammerl
Lehrstuhl für Numerische Mechanik
Technische Universität München
Boltzmannstrasse 15, D-85747 Garching b. München
phone +49 89 289 15237
fax +49 89 289 15301
<a class="moz-txt-link-abbreviated" href="mailto:hammerl@lnm.mw.tum.de">hammerl@lnm.mw.tum.de</a>
<a class="moz-txt-link-freetext" href="http://www.lnm.mw.tum.de">http://www.lnm.mw.tum.de</a>
</pre>
</body>
</html>